danos/docs/build-packages-plan.md

Plan: packages — hierarchical builds for libraries and binaries

Status: complete (branch claude/build-packages-plan-174144). Phase 0 (build-support), phase 1 (all six library domains), phase 2 (every binary — the pci-bus pilot first, then services, drivers, and test fixtures in waves; multi-binary directories like ps2-bus and usb-hid are one package exporting several artifacts, and the acpi/fdt discovery pair each export an artifact named "discovery" that the root's -Ddiscovery picks between), and phase 3 (the root split into build/images.zig + build/qemu.zig; the root build.zig is ~460 lines of orchestration, down from ~1,250). Every phase landed green: unit tests, the QEMU suite at parity with main, boot-image file list unchanged. The lazyDependency payoff (What-this-buys #4) is in too: the /test fixtures and the unselected discovery package are lazy — a build loads and compiles only what it ships. And imports are exact: the pre-wired default set is gone; every binary names precisely the modules its source imports and carries only those domains in its manifest (rule 1 below).

Why

build.zig was ~1,250 lines, growing by three hand-written stanzas per binary; at a driver per device family that does not scale. More fundamentally: in one monolithic build every binary compiles against library source, so a library interface break is silently absorbed by whoever edits everything in one commit — the interface never has to be honest. danos is about isolation; the build should mirror it.

A package here is a build-time unit only — a directory owning a build.zig (recipe: what it exports, how to test it) and a build.zig.zon (manifest: name

• dependencies). Binaries remain fully static freestanding ELFs; packages change who declares what, not what links to what. Source code is untouched: @import uses module names ("pci", "service") exactly as today — only build files know where anything lives.

Target shape

build-support/                 package: the danos build API (userBinary(), defaultImports(), targets)
library/kernel/                package "kernel":   modules abi, ipc, service, memory, process, logging, time, ...   (depends on protocol)
library/device/                package "device":   modules driver, pci, usb-abi, model, ...   (depends on kernel, protocol, csv)
library/protocol/              package "protocol": the wire protocols
library/client/                package "client"    (depends on kernel, protocol)
library/csv/                   package "csv"
library/xkeyboard-config/      package "xkeyboard-config"
system/services/<name>/        one package per binary: ~15-line build.zig + zon
system/drivers/<name>/         one package per binary
build.zig (root)               orchestrator: dependency() per binary, image assembly, QEMU, test steps

The three shared contracts: boot-handoff stays a root module (only the loader↔kernel pair speaks it); abi is exported by the kernel package from ../../system/abi.zig (the source stays with the kernel; userspace's one view of it lives in the package, so every consumer names the same module instance); device-abi is exported by device. Reaching outside the package root means the kernel package is valid only as an in-repo path dependency — it could never be fetched by hash — which is fine: path dependencies are the only way any of these packages is consumed.

Rules:

• Imports are exact and per binary. A binary's build.zig names precisely the modules its source @imports — the moral equivalent of a C file's include list — and its zon names only the domains those modules come from (plus build-support and kernel, which is implicit in every binary: the root shim and user link script live there). Nothing is pre-wired: an undeclared @import is a compile error, and build-support resolves each name by searching the packages the zon declares — the domains' own addModule exports are the single statement of who owns what, with no name table anywhere to drift. Availability never meant bloat — Zig only compiles what a program actually imports — but exactness makes the declared interface honest and machine-checked.
• Modules export source, not artifacts — each consumer compiles libraries with its own flags, so per-binary optimization choices keep working; Zig's cache deduplicates.
• Zon paths are relative and that is accepted. Binaries sit exactly three levels deep, so the ../../../ prefix is a constant idiom; a library-domain move is a rare, already-breaking event fixed by one sed across manifests, and a stale path fails loudly before anything compiles.
• Cross-cutting build changes live in build-support only — that is the contract that keeps per-binary build files declarative.

What this buys

1. Library interfaces become machine-checked: a consumer can only import what it declared — per binary, down to the single module — and each domain's zon declares what it needs (claim-before-touch, applied to source). A keyboard driver carries xkeyboard-config in its manifest; nothing else does.
2. Each library domain gets a standalone zig build test — runtime-library stability testing in isolation.
3. Adding a binary = adding a directory (source + two small files), not editing three places in a 1,250-line file.
4. lazyDependency lets an image target build only what it ships: the /test fixtures resolve only under -Dtest-case, and only the -Ddiscovery-selected discovery package ever loads.

Phases

Each phase ends green: zig build test passes (88/88 QEMU) and the boot image's file list is unchanged. Byte-identical binaries are expected but not required (module reorganization can perturb symbol order); file list is the hard gate.

Phase 0 — build-support. Extract addUserBinary/addThreadedUserBinary, the freestanding target setup, and the default-import wiring into the build-support package. Root build consumes it; nothing else moves. This is the cross-cutting-change home, so it lands first.

Phase 1 — library domains become packages. In dependency order: protocol and csv (the roots) → kernel (depends on protocol: file-system speaks vfs-protocol) → device, client; xkeyboard-config stands alone. Each gets build.zig + zon + a standalone test step (client's is empty until its modules grow host tests — kept for uniformity, since the root aggregate depends on every domain's test step). The root build swaps its createModule calls for b.dependency("<domain>").module("<name>"). No binary moves in this phase — the root build is the pilot consumer, which proves the packages without touching 30 binaries.

Phase 2 — binaries become packages, in waves. The template was shaken out by the pci-bus pilot (see Status). Wave A: services (done). Wave B: the remaining drivers (done). Wave C: test fixtures (done). Root build shrank to orchestration per wave. init's -Dserial heartbeat flag rides a dependency option; a directory with several binaries (ps2-bus, usb-hid) is one package exporting several artifacts.

Phase 3 — root cleanup (done). What remained of the root build split into build/images.zig (the FHS install tree, boot manifest + capsule, FAT32 images, release ISO, check steps) and build/qemu.zig (the run steps + OVMF probing), imported by a short root build.zig.

Afterwards (outside this plan): the intel-uhd-graphics-750 driver is (re)created as a greenfield package. The new-driver checklist's build step (docs/device-driver-development/new-driver-checklist.md, step 2) is already rewritten against the package template.

Execution notes (the finished shape)

• The shared recipe lives in build-support/build.zig: userBinary (what every binary package calls; each named import resolves by searching the packages the binary's zon declares) and programModule (for per-binary addOptions modules). The start root shim and user.ld are named through the kernel package (Dependency.path).
• Adding a binary = adding a directory with source + a ~15-line build.zig + zon (copy any existing binary package, e.g. system/drivers/pci-bus/build.zig) listing exactly the modules the source imports and the domains they come from, then one dependency + one bundled entry in the root build.zig and one zon line.
• The boot-tree array in the root (search "etc/init.csv" or .getEmittedBin()) is the image file list — the authoritative comparison target for any future build change.
• Package unit tests live in each package's own test step; the root aggregate depends on every test-bearing package's step, so zig build test at the root still runs everything.

Verification per phase:

• Unit tests: zig build test.
• QEMU integration suite: python3 test/qemu_test.py (docs/testing.md; the full suite, all cases must pass).
• Image file list: the boot-tree array is the source of truth — snapshot it (paths only) before phase 0 and diff after each phase; zig build check-fat-image must also stay green.

Context a fresh session should read first: this doc, docs/testing.md, docs/coding-standards.md (kebab-case names, no abbreviations), and the userBinary/userBinaryFromImports bodies in build-support/build.zig. Commit style: no Co-Authored-By trailers.

Risks / notes

• Zig version churn: the package API (b.dependency, zon schema) has moved between releases; the work pins against the repo's current Zig and any upgrade lands separately, never mid-phase.
• The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol struct sizes) — they moved into their binaries' packages with their waves, discharging the carry-along obligation.
• Doc updates ride each phase: docs/README.md (repo layout + source map), docs/device-driver-development/new-driver-checklist.md (step 2) and devices-csv.md ("Adding a driver"), and the docs that cite the build recipe (driver-model.md, threading.md, system-requirements.md) reference build shapes that keep changing.