Plan: packages — hierarchical builds for libraries and binaries

Status: proposed, awaiting sign-off. No implementation yet.

Why

build.zig is ~1,250 lines and grows by three hand-written stanzas per binary; at a driver per device family that does not scale. More fundamentally: in one monolithic build every binary compiles against library source, so a library interface break is silently absorbed by whoever edits everything in one commit — the interface never has to be honest. danos is about isolation; the build should mirror it.

A package here is a build-time unit only — a directory owning a build.zig (recipe: what it exports, how to test it) and a build.zig.zon (manifest: name

dependencies). Binaries remain fully static freestanding ELFs; packages change who declares what, not what links to what. Source code is untouched: @import uses module names ("pci", "service") exactly as today — only build files know where anything lives.

Target shape

build-support/                 package: the danos build API (userBinary(), targets, default imports)
library/kernel/                package "kernel":   modules ipc, service, memory, process, logging, time, ...
library/device/                package "device":   modules driver, pci, usb-abi, model, ...   (depends on kernel)
library/protocol/              package "protocol": the wire protocols
library/client/                package "client"    (depends on kernel, protocol)
library/csv/                   package "csv"
library/xkeyboard-config/      package "xkeyboard-config"
system/services/<name>/        one package per binary: ~10-line build.zig + zon
system/drivers/<name>/         one package per binary
build.zig (root)               orchestrator: dependency() per binary, image assembly, QEMU, test steps

Rules:

Dependencies are declared at domain level (a binary's zon names kernel, device), imports stay module-level (@import("pci")). build-support pre-wires the core set every binary uses (service, ipc, memory, process, logging); per-binary build files name only extras.
Modules export source, not artifacts — each consumer compiles libraries with its own flags, so per-binary optimization choices keep working; Zig's cache deduplicates.
Zon paths are relative and that is accepted. Binaries sit exactly three levels deep, so the ../../../ prefix is a constant idiom; a library-domain move is a rare, already-breaking event fixed by one sed across manifests, and a stale path fails loudly before anything compiles.
Cross-cutting build changes live in build-support only — that is the contract that keeps per-binary build files declarative.

What this buys

Library interfaces become machine-checked: a consumer can only import what a domain exports, and each domain's zon declares what it needs (claim-before- touch, applied to source).
Each library domain gets a standalone zig build test — runtime-library stability testing in isolation.
Adding a binary = adding a directory (source + two small files), not editing three places in a 1,250-line file.
Later: lazyDependency lets an image target build only what it ships.

Phases

Each phase ends green: zig build test passes (88/88 QEMU) and the boot image's file list is unchanged. Byte-identical binaries are expected but not required (module reorganization can perturb symbol order); file list is the hard gate.

Phase 0 — build-support. Extract addUserBinary/addThreadedUserBinary, the freestanding target setup, and the default-import wiring into the build-support package. Root build consumes it; nothing else moves. This is the cross-cutting-change home, so it lands first.

Phase 1 — library domains become packages. In dependency order: kernel (no deps) → csv, protocol → device, client → xkeyboard-config. Each gets build.zig + zon + a standalone test step. The root build swaps its createModule calls for b.dependency("<domain>").module("<name>"). No binary moves in this phase — the root build is the pilot consumer, which proves the packages without touching 30 binaries.

Phase 2 — binaries become packages, in waves. Wave A: two small services (e.g. logger, display-demo) to shake out the template. Wave B: remaining services. Wave C: drivers. Wave D: test fixtures. Root build shrinks to orchestration per wave.

Phase 3 — root cleanup. Split what remains of the root build into build/images.zig, build/qemu.zig, imported by a short root build.zig.

Afterwards (outside this plan): the intel-uhd-graphics-750 driver is (re)created as a greenfield package — the new-driver checklist's step 2 gets rewritten against the package template at that point.

Execution notes (for whichever session runs this)

Anchors in today's root build.zig (~1,250 lines):

addUserBinary / addThreadedUserBinary / addUserBinaryImpl and the default_imports plumbing start around line 63 — this is what phase 0 extracts into build-support. addUserBinaryImpl also wires the start root shim from default_imports; that trick must survive the move.
programModule(<exe>).addImport(...) calls (search programModule) are the per-binary extra imports — the data for each binary's future build.zig.
The boot-tree array (search "etc/init.csv" or .getEmittedBin()) is the image file list — the authoritative before/after comparison target.
Protocol/module definitions (search createModule) map module names to library/... source paths — the data for each domain package's exports.
The QEMU size-check test hardcodes source paths (search virtio-gpu-protocol.zig near line 1149) — moves with phase 2 wave C.

Verification per phase:

Unit tests: zig build test.
QEMU integration suite: python3 test/qemu_test.py (docs/testing.md; the full suite, all cases must pass).
Image file list: the boot-tree array is the source of truth — snapshot it (paths only) before phase 0 and diff after each phase; zig build check-fat-image must also stay green.

Context a fresh session should read first: this doc, docs/testing.md, docs/coding-standards.md (kebab-case names, no abbreviations), and the addUserBinaryImpl body. Commit style: no Co-Authored-By trailers.

Risks / notes

Zig version churn: the package API (b.dependency, zon schema) has moved between releases; the work pins against the repo's current Zig and any upgrade lands separately, never mid-phase.
The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol struct sizes in the root build) — phase 2 wave C must carry those along.
Doc updates ride each phase: drivers.md, new-driver-checklist.md, and docs/README.md reference build steps that will change shape.

6.7 KiB Raw Blame History