Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

37. The native-module rebuild mechanism: a generated project, korrin-cli as a library, and a git-rev-pinned korrin

  • Status: Accepted
  • Date: 2026-09-05

Context

ADR 0035 settled that native modules load by compile-time linking. This ADR is the mechanical question that decision leaves open, with zero precedent anywhere in this codebase to draw on: given a third-party crate implementing NativeModule (ADR 0036), how does "a crate implements a trait" become "the korrin binary a developer actually runs knows about it"?

There is no dynamic loading, so this has to resolve at Rust compile time — concretely, a second cargo build, separate from however the generic korrin binary itself was built, producing a binary specific to one project's set of native dependencies.

Decision

korrin-cli gains a library target

crates/korrin-cli is bin-only today (Cargo.toml has [[bin]] name = "korrin", doc = false to avoid colliding with the korrin library crate's own docs, no [lib]). It gains src/lib.rs:

pub fn main_with_native_modules(
    native_modules: Vec<(String, Box<dyn korrin::NativeModule>)>,
) -> std::process::ExitCode { /* today's main() body, parameterized */ }

main.rs shrinks to fn main() -> ExitCode { korrin_cli::main_with_native_modules(Vec::new()) }. The generic korrin binary's behavior is unchanged — an empty vec means the existing three-way import dispatch never reaches the native-module branch at all.

This was weighed against hand-writing a second, slimmed-down CLI inside the generated project (below) and rejected: two implementations of "parse argv, run a file, print diagnostics" that must agree forever is exactly the failure ADR 0021 already refused to create for builtins across engines. Reusing korrin-cli as a library means a project's custom binary supports run --vm, tokens, ast, bytecode, pkg, and the REPL identically to the generic one, for free.

The generated project

korrin pkg install/add/remove/update, on finding a native = true dependency, generate (or refresh) a project at .korrin/native-runner/ — gitignored, @generated by \korrin pkg` — do not edit by hand, the same convention korrin.lock` already established:

# .korrin/native-runner/Cargo.toml
[package]
name = "korrin-native-runner"
publish = false
[[bin]]
name = "korrin"
path = "src/main.rs"
[dependencies]
korrin-cli = { git = "<embedded, see below>", rev = "<embedded, see below>" }
gamekorr    = { git = "https://github.com/korrin-lang/gamekorr", tag = "v0.1.0" }

// .korrin/native-runner/src/main.rs
fn main() -> std::process::ExitCode {
    korrin_cli::main_with_native_modules(vec![
        ("gamekorr".to_string(), gamekorr::native_module()),
    ])
}

One line per native dependency, in manifest order, generated by hand as a plain Vec literal — this is ADR 0036's "hardcoded dispatch," just materialized as generated data instead of hand-typed source, exactly as stdlib::load's own match is hand-typed data today. linkme/inventory-style compile-time-registration crates (populating a registry via custom linker sections) were considered and rejected: neither literally requires a user-written unsafe block, but both depend on linker/section behavior outside what unsafe_code = "forbid"'s own stated purpose — auditability of this workspace's own memory-safety surface — was ever meant to police, and neither buys anything a five-line generated Vec literal doesn't already do more simply and more legibly for a reviewer.

Pinning which korrin the generated project builds against

The generated project must depend on a korrin/korrin-cli source- compatible with whatever korrin binary is doing the generating (the same NativeModule/Value shapes). Two real options:

  • A crates.io version pin (korrin-cli = "0.x"). Clean, but requires publishing korrin/korrin-cli to crates.io on some cadence — every existing cross-crate dependency in this workspace is a path dependency; this project has never needed to publish anything, and no other ADR anywhere commits to doing so.
  • A git-rev pin, embedded at build time. A build.rs in korrin-cli runs git rev-parse HEAD (when building from a real checkout) and exposes it via env!; that rev is what the generated Cargo.toml pins against.

Decision: the git-rev pin, for exactly the reason above — Korrin is pre-1.0 with no crates.io publication commitment anywhere else in this project. Revisit once/if Korrin ever does publish; the crates.io pin becomes strictly better then (faster resolution, no full git clone of the language itself as a build dependency of every native-module project).

The pinned URL comes from the checkout's own origin remote, not from Cargo.toml's static repository field. The same build.rs that reads git rev-parse HEAD also reads git remote get-url origin, falling back to Cargo.toml's repository field only when a checkout has no origin remote configured. This matters concretely, not just in principle: a fork's korrin pkg build must link the generated project against that fork's commit, not silently pull the upstream korrin-lang/korrin repository at a revision the fork's own changes were never pushed to, which is exactly what a hardcoded canonical URL would do. Both values are exposed as korrin_cli::GIT_URL/GIT_REV, mirroring each other in shape and origin.

What triggers the rebuild

Not every korrin pkg install — a cargo build pulling in korrin-cli's full dependency tree plus a native crate (e.g. minifb) is genuinely slow, and the overwhelming majority of installs (plain .kor packages, which remain the common case) must stay exactly as fast as they are today. The rebuild — exposed as its own subcommand, korrin pkg build — runs automatically only when the set of native dependencies actually changed since korrin.lock last recorded one. A stale pinned-korrin-cli-rev (the generated project's pin no longer matching the currently-installed korrin) is surfaced as a plain warning to re-run korrin pkg build manually, not an automatic rebuild — the same "explicit update step, not automatic tracking" philosophy ADR 0031 already chose for branch-pinned dependencies going stale.

Manifest and korrin-pkg changes

A native = true boolean on an existing [dependencies] entry, not a second table:

[dependencies]
gamekorr = { git = "https://github.com/korrin-lang/gamekorr", tag = "v0.1.0", native = true }

Weighed against a Cargo-[build-dependencies]-style separate table and rejected: native-vs-plain dependencies never need independent ordering or a different resolution algorithm — they still fetch identically (git/ tag/branch/rev), only what happens after fetching differs. A boolean is proportionate to the actual distinction. Dependency and LockedPackage each gain pub native: bool, default false, omitted (not written as false) for a plain package — every existing korrin.toml and test stays byte-for-byte unaffected, and no lockfile version bump is needed.

check_no_builtin_collision needs no change — a native module's local name lives in the same [dependencies] table a plain package's does, so the existing stdlib::module_names().contains(&name) check already covers it, run once per dependency regardless of native. This mirrors ADR 0032's own finding that package/built-in disjointness needed no new machinery beyond one existing check.

validate_fetched_package gains a parallel native path: requires Cargo.toml at the checkout root (the one thing checkable without invoking rustc); still applies the leaf-only rule uniformly if the fetched crate carries its own korrin.toml. It does not attempt to verify the crate actually exposes pub fn native_module() -> Box<dyn korrin::NativeModule> — Rust has no reflection to check that without compiling; a mismatch surfaces as an ordinary compile failure from korrin pkg build instead.

Errors

PkgError gains NativeModuleMissingCargoToml, CargoNotFound (a new environmental assumption this milestone adds on top of M5's "git on PATH": a working Rust toolchain), and NativeBuildFailed { stderr } — cargo's own stderr printed verbatim under the existing korrin pkg: ... prefix, never re-parsed, since that output belongs to cargo/rustc, not to Korrin. No new Korrin diagnostic (E0xxx) code is needed — import "gamekorr" through the generic binary (which was never given this module) fails exactly the way an unresolved package import already does, E0314, the same category ADR 0031/ 0032 already unified.

Consequences

  • A native-module dependency is the first thing in this project that requires a second, real cargo build as part of korrin pkg's own work — a genuinely new kind of operation, slower and more failure-prone (real compile errors) than anything M5's package system ever needed to handle.
  • Every project with a native-module dependency effectively vendors a pinned copy of korrin-cli's source (via the git-rev pin) inside its own build — real disk/bandwidth cost per project, accepted the same way ADR 0031 already accepted "no shared package cache" as a real cost of a simpler design.
  • korrin-cli being both a binary and a library is a small, permanent addition to its public surface — its lib.rs API (main_with_native_ modules) is now something other crates (specifically, every generated .korrin/native-runner/) depend on and must not break carelessly.
  • See ADR 0038 for how a project's generated binary actually gets invoked.

Alternatives considered

  • A crates.io pin instead of a git-rev pin. See "Pinning" above — not rejected forever, just premature given no crates.io publication commitment exists anywhere else in this project yet.
  • Rebuilding on every korrin pkg install, unconditionally. Simpler, but makes the common case (installing a plain .kor package) pay a multi-minute cargo build tax it never needed before. Rejected for cost alone.
  • Hand-writing a slimmed CLI for the generated project instead of reusing korrin-cli as a library. See "the generated project" above — rejected to avoid two CLIs that must agree forever.