docs: add the build-packages plan
This commit is contained in:
Daniel Samson
@@ -0,0 +1,132 @@
|
||||
# Plan: packages — hierarchical builds for libraries and binaries
|
||||
|
||||
**Status: proposed, awaiting sign-off. No implementation yet.**
|
||||
|
||||
## Why
|
||||
|
||||
`build.zig` is ~1,250 lines and grows by three hand-written stanzas per binary;
|
||||
at a driver per device family that does not scale. More fundamentally: in one
|
||||
monolithic build every binary compiles against library *source*, so a library
|
||||
interface break is silently absorbed by whoever edits everything in one commit —
|
||||
the interface never has to be honest. danos is about isolation; the build should
|
||||
mirror it.
|
||||
|
||||
A **package** here is a build-time unit only — a directory owning a `build.zig`
|
||||
(recipe: what it exports, how to test it) and a `build.zig.zon` (manifest: name
|
||||
+ dependencies). Binaries remain fully static freestanding ELFs; packages change
|
||||
who declares what, not what links to what. Source code is untouched: `@import`
|
||||
uses module names (`"pci"`, `"service"`) exactly as today — only build files
|
||||
know where anything lives.
|
||||
|
||||
## Target shape
|
||||
|
||||
```
|
||||
build-support/ package: the danos build API (userBinary(), targets, default imports)
|
||||
library/kernel/ package "kernel": modules ipc, service, memory, process, logging, time, ...
|
||||
library/device/ package "device": modules driver, pci, usb-abi, model, ... (depends on kernel)
|
||||
library/protocol/ package "protocol": the wire protocols
|
||||
library/client/ package "client" (depends on kernel, protocol)
|
||||
library/csv/ package "csv"
|
||||
library/xkeyboard-config/ package "xkeyboard-config"
|
||||
system/services/<name>/ one package per binary: ~10-line build.zig + zon
|
||||
system/drivers/<name>/ one package per binary
|
||||
build.zig (root) orchestrator: dependency() per binary, image assembly, QEMU, test steps
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- **Dependencies are declared at domain level** (a binary's zon names `kernel`,
|
||||
`device`), **imports stay module-level** (`@import("pci")`). `build-support`
|
||||
pre-wires the core set every binary uses (service, ipc, memory, process,
|
||||
logging); per-binary build files name only extras.
|
||||
- **Modules export source, not artifacts** — each consumer compiles libraries
|
||||
with its own flags, so per-binary optimization choices keep working; Zig's
|
||||
cache deduplicates.
|
||||
- **Zon paths are relative and that is accepted.** Binaries sit exactly three
|
||||
levels deep, so the `../../../` prefix is a constant idiom; a library-domain
|
||||
move is a rare, already-breaking event fixed by one sed across manifests, and
|
||||
a stale path fails loudly before anything compiles.
|
||||
- **Cross-cutting build changes live in `build-support` only** — that is the
|
||||
contract that keeps per-binary build files declarative.
|
||||
|
||||
## What this buys
|
||||
|
||||
1. Library interfaces become machine-checked: a consumer can only import what a
|
||||
domain exports, and each domain's zon declares what it needs (claim-before-
|
||||
touch, applied to source).
|
||||
2. Each library domain gets a standalone `zig build test` — runtime-library
|
||||
stability testing in isolation.
|
||||
3. Adding a binary = adding a directory (source + two small files), not editing
|
||||
three places in a 1,250-line file.
|
||||
4. Later: `lazyDependency` lets an image target build only what it ships.
|
||||
|
||||
## Phases
|
||||
|
||||
Each phase ends green: `zig build test` passes (88/88 QEMU) and the boot
|
||||
image's file list is unchanged. Byte-identical binaries are expected but not
|
||||
required (module reorganization can perturb symbol order); file list is the
|
||||
hard gate.
|
||||
|
||||
**Phase 0 — `build-support`.** Extract `addUserBinary`/`addThreadedUserBinary`,
|
||||
the freestanding target setup, and the default-import wiring into the
|
||||
`build-support` package. Root build consumes it; nothing else moves. This is
|
||||
the cross-cutting-change home, so it lands first.
|
||||
|
||||
**Phase 1 — library domains become packages.** In dependency order: `kernel`
|
||||
(no deps) → `csv`, `protocol` → `device`, `client` → `xkeyboard-config`. Each
|
||||
gets build.zig + zon + a standalone test step. The root build swaps its
|
||||
`createModule` calls for `b.dependency("<domain>").module("<name>")`. **No
|
||||
binary moves in this phase** — the root build is the pilot consumer, which
|
||||
proves the packages without touching 30 binaries.
|
||||
|
||||
**Phase 2 — binaries become packages, in waves.** Wave A: two small services
|
||||
(e.g. logger, display-demo) to shake out the template. Wave B: remaining
|
||||
services. Wave C: drivers. Wave D: test fixtures. Root build shrinks to
|
||||
orchestration per wave.
|
||||
|
||||
**Phase 3 — root cleanup.** Split what remains of the root build into
|
||||
`build/images.zig`, `build/qemu.zig`, imported by a short root `build.zig`.
|
||||
|
||||
**Afterwards** (outside this plan): the intel-uhd-graphics-750 driver is
|
||||
(re)created as a greenfield package — the new-driver checklist's step 2 gets
|
||||
rewritten against the package template at that point.
|
||||
|
||||
## Execution notes (for whichever session runs this)
|
||||
|
||||
Anchors in today's root `build.zig` (~1,250 lines):
|
||||
|
||||
- `addUserBinary` / `addThreadedUserBinary` / `addUserBinaryImpl` and the
|
||||
`default_imports` plumbing start around line 63 — this is what phase 0
|
||||
extracts into `build-support`. `addUserBinaryImpl` also wires the `start`
|
||||
root shim from `default_imports`; that trick must survive the move.
|
||||
- `programModule(<exe>).addImport(...)` calls (search `programModule`) are the
|
||||
per-binary extra imports — the data for each binary's future build.zig.
|
||||
- The boot-tree array (search `"etc/init.csv"` or `.getEmittedBin()`) is the
|
||||
image file list — the authoritative before/after comparison target.
|
||||
- Protocol/module definitions (search `createModule`) map module names to
|
||||
`library/...` source paths — the data for each domain package's exports.
|
||||
- The QEMU size-check test hardcodes source paths (search
|
||||
`virtio-gpu-protocol.zig` near line 1149) — moves with phase 2 wave C.
|
||||
|
||||
Verification per phase:
|
||||
|
||||
- Unit tests: `zig build test`.
|
||||
- QEMU integration suite: `python3 test/qemu_test.py` (docs/testing.md; the
|
||||
full suite, all cases must pass).
|
||||
- Image file list: the boot-tree array is the source of truth — snapshot it
|
||||
(paths only) before phase 0 and diff after each phase; `zig build
|
||||
check-fat-image` must also stay green.
|
||||
|
||||
Context a fresh session should read first: this doc, docs/testing.md,
|
||||
docs/coding-standards.md (kebab-case names, no abbreviations), and the
|
||||
`addUserBinaryImpl` body. Commit style: no Co-Authored-By trailers.
|
||||
|
||||
## Risks / notes
|
||||
|
||||
- Zig version churn: the package API (`b.dependency`, zon schema) has moved
|
||||
between releases; the work pins against the repo's current Zig and any
|
||||
upgrade lands separately, never mid-phase.
|
||||
- The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol
|
||||
struct sizes in the root build) — phase 2 wave C must carry those along.
|
||||
- Doc updates ride each phase: drivers.md, new-driver-checklist.md, and
|
||||
docs/README.md reference build steps that will change shape.
|
||||
Reference in New Issue
Block a user