Choose native executable packaging

Choose native executable packaging

Type: grilling Status: resolved Blocked by: 01, 08, 09 Parent: Find the ASPL MVP specification

Question

Which feasible ELF packaging design should aspl build adopt so one C-written tool can combine the VM and bytecode into a native program, preserve required shared-library behavior, avoid external build tools, and leave an inspectable and versioned artifact contract?

Answer

aspl build copies a release-owned x86-64 glibc PIE VM template and extends its final read-only load segment with a thin native envelope followed by unchanged canonical ASPL bytecode. It does not reopen the executable at runtime, reserve a fixed bytecode capacity, rewrite dynamic-linker metadata, or invoke a compiler, assembler, or linker.

Release template

Each ASPL release embeds one normalized VM template and a manifest in the distributed aspl executable. Contributors may use the platform C compiler and linker to produce this release asset. The end-user build path only reads and writes bytes in-process.

The manifest records the complete template byte length, its CRC-32/ISO-HDLC value, the payload program-header index, the payload file offset and virtual address, its alignment, and the expected immutable ELF fields. aspl build checks the CRC and parses the template instead of casting file bytes to C structs.

A valid template has all of these properties:

• ELF64, little-endian, x86-64, and ET_DYN PIE;
• the release's fixed PT_INTERP, native dependencies, and GNU build ID;
• no section table, expressed by zero e_shoff and e_shnum values and SHN_UNDEF in e_shstrndx;
• no bytes after the initial payload segment;
• no writable-executable load segment;
• a non-executable PT_GNU_STACK and a valid PT_GNU_RELRO region; and
• a final PT_LOAD segment with the highest virtual address, PF_R as its only permission, and initial p_filesz and p_memsz values of 24.

The payload segment's file offset and virtual address are congruent modulo its recorded alignment. Its initial range does not overlap any earlier load segment after page rounding. Since it is the final load segment and final file region, the builder can extend it without moving another region or changing an offset.

The exact glibc floor belongs to Define distribution and installation. Once a release chooses that floor, the template fixes its interpreter path, glibc symbol requirements, and native DT_NEEDED entries.

Native payload envelope

The payload segment starts with this 24-byte little-endian header:

Canonical ASPL bytecode starts immediately at offset 24. The segment contains no padding or trailing bytes, so its exact size is 24 + bytecode length. The builder and runtime perform this addition in a widened integer type. The bytecode length must fit its u32 field and the bytecode and host limits already fixed by Choose the register bytecode contract.

The native envelope does not copy the bytecode version, total length, or CRC. The inner bytecode envelope remains the sole owner of those values. This avoids conflicting copies and lets the native and browser VMs consume the same serialized bytes.

Native packaging version 1 uses exact-version acceptance. A change to the header layout, a field's meaning, or the native payload discovery and validation rules requires a new native packaging version. A bytecode-only change requires a new bytecode version instead. Changing VM code without changing either contract changes the GNU build ID but does not force a format-version change.

The native envelope is documented so tools can inspect it, but it is private. It is not a stable extension API. Later ASPL tools do not have to read or rewrite executables produced by older releases. An already-built native program carries its own VM and remains independently runnable while its required system and program libraries remain available.

Build procedure

For a bytecode artifact that has passed the canonical bytecode validator, aspl build performs these steps:

1. Validate the embedded template CRC, manifest, ELF identity, program-header bounds, segment ordering, alignments, permissions, non-overlap, and exact file ending.
2. Check the bytecode and resulting segment sizes against the serialized limits and native host policy without narrowing arithmetic.
3. Copy the template, write the 24-byte envelope at the payload offset, and write the bytecode immediately after it.
4. Patch only the payload program header's p_filesz and p_memsz fields to 24 + bytecode length.
5. Set the completed file length to payload file offset + patched p_filesz.
6. Parse and validate the completed ELF and its embedded bytecode before handing the result to the CLI publication path.

No other ELF header, program header, dynamic table, relocation, symbol, or interpreter field may change. A mismatched template is a broken release asset, not input for a repair path.

For the same ASPL release, source, and build options, this procedure produces byte-identical ELF contents. The artifact contains no timestamp, destination name, absolute source path, random padding, or other host-specific build data. File naming, modes, temporary-file handling, atomic publication, and replacement policy belong to Define the CLI and build behavior.

The template's native dependencies stay fixed. Foreign library names remain bytecode data. Native execution loads them through the eager dlopen and dlsym procedure from Define dynamic C interoperability. Checking and building never require those program libraries to be installed, and the emitter never adds them to DT_NEEDED.

Native startup and validation

Normal glibc PIE startup reaches the embedded VM. The VM uses dl_iterate_phdr to inspect the main executable's in-memory program headers and the linker-defined payload-start symbol to identify the payload segment. Before reading variable-length data, it requires:

• one matching final PT_LOAD segment whose relocated start address equals the payload symbol;
• PF_R as the segment's only permission;
• equal p_filesz and p_memsz values of at least 24;
• the exact magic, native version, header size, and zero flags; and
• a widened 24 + bytecode length calculation equal to the mapped segment size.

The VM then passes the unchanged bytecode range to the bytecode loader. That loader checks its exact version, structure, limits, CRC, instructions, types, control flow, and definite assignment before constructing VmProgram. Only after both validation layers succeed may native execution prepare foreign imports or call main.

Identity, integrity, and failures

The template CRC detects accidental damage or a release-generation mismatch before aspl build emits a program. The inner bytecode CRC covers the complete serialized bytecode. The native wrapper adds no duplicate checksum.

All programs produced by one ASPL release retain the template's GNU build ID. That ID names the native VM code for crash symbolization. It is not the identity or checksum of the complete generated program. A checksum over the full ELF identifies one complete artifact.

Neither CRC authenticates code or protects against an attacker who can rewrite the executable. Release and download checksums own that security boundary.

The outcome model separates failures as follows:

• An invalid embedded template, bad manifest, or invalid completed output is an internal aspl tool failure. The CLI must not publish the incomplete result.
• An invalid native envelope or invalid embedded bytecode is a load failure. Native execution stops before opening program libraries or entering main.
• A violated VM invariant after successful validation is a VM failure, not a load failure or ASPL trap.

Define diagnostics and safety owns stable diagnostic wording. Define the CLI and build behavior owns numeric process statuses and output publication behavior.