docs: catch the docs up with the finished package split

The plan doc's status records completion (all waves + phase 3) and its
execution notes describe the finished shape; the fresh-session pointer
names build-support's userBinary instead of the deleted
addUserBinaryImpl, and the size-check carry-along note is discharged.
README gains the build/ directory in the layout tree and splits the
source-map row across root build.zig / build/images.zig /
build/qemu.zig. testing.md points at the distributed per-package test
steps; threading.md, threading-plan.md, driver-model.md, efi.md,
display.md, system-requirements.md, and devices-csv.md's adding-a-
driver checklist stop describing the pre-package build.
This commit is contained in:
Daniel Samson
2026-07-30 04:24:34 +01:00
parent 3f9b6813f7
commit d03942b543
10 changed files with 73 additions and 53 deletions
+8 -4
View File
@@ -266,14 +266,16 @@ test/ → /test the test tree: the QEMU harness (qemu_test.py, h
build-support/ the danos build API (build-time only, nothing on the image): build-support/ the danos build API (build-time only, nothing on the image):
the shared user-binary recipe + default-import wiring every the shared user-binary recipe + default-import wiring every
build file consumes (docs/build-packages-plan.md) build file consumes (docs/build-packages-plan.md)
build/ root-build helpers: image assembly (images.zig) + the QEMU
run steps (qemu.zig)
tools/ host-side build scripts tools/ host-side build scripts
``` ```
**Builds are packages** (docs/build-packages-plan.md): each `library/` domain owns a **Builds are packages** (docs/build-packages-plan.md): each `library/` domain owns a
`build.zig`/`build.zig.zon` exporting its modules (with a standalone `zig build test`), `build.zig`/`build.zig.zon` exporting its modules (with a standalone `zig build test`),
binaries are converting one directory at a time to ~15-line package builds (`pci-bus` every binary directory is a ~15-line package build, and the root `build.zig`
is the first), and the root `build.zig` orchestrates — image assembly, QEMU, the orchestrates — the kernel + loader, what ships, and the aggregate test step — with
aggregate test step. image assembly in `build/images.zig` and the QEMU run steps in `build/qemu.zig`.
**Wire protocols live in `library/protocol/`**, one module per directory **Wire protocols live in `library/protocol/`**, one module per directory
(`library/protocol/vfs/vfs-protocol.zig` is the `vfs-protocol` module), imported by module (`library/protocol/vfs/vfs-protocol.zig` is the `vfs-protocol` module), imported by module
@@ -329,5 +331,7 @@ exception in [coding-standards.md](coding-standards.md) applies to that seam.
| System services (init, the `fat` filesystem, the device-manager) | `system/services/` | | System services (init, the `fat` filesystem, the device-manager) | `system/services/` |
| Device drivers, one sub-project each (`pci-bus`, `ps2-bus`, `usb-xhci-bus` bus drivers) | `system/drivers/` | | Device drivers, one sub-project each (`pci-bus`, `ps2-bus`, `usb-xhci-bus` bus drivers) | `system/drivers/` |
| On-image test fixtures for the QEMU cases (`vfs-test`, `crash-test`, `thread-test`, …) → `/test/system/services` | `test/system/services/` | | On-image test fixtures for the QEMU cases (`vfs-test`, `crash-test`, `thread-test`, …) → `/test/system/services` | `test/system/services/` |
| Build orchestration + `run-x86-64` (QEMU/OVMF) + `release-x86-64` (the flashable ISO) | `build.zig` (root; the shared user-binary recipe is `build-support/`, and each `library/` domain + packaged binary carries its own `build.zig`) | | Build orchestration (kernel + loader, what ships, the aggregate test step) | `build.zig` (root; the shared user-binary recipe is `build-support/`, and each `library/` domain + binary package carries its own `build.zig`) |
| Image assembly + `release-x86-64` (the flashable ISO) | `build/images.zig` |
| `run-x86-64` / `run-x86-64-gpu` (QEMU/OVMF) | `build/qemu.zig` |
| QEMU integration test harness | `test/qemu_test.py` | | QEMU integration test harness | `test/qemu_test.py` |
+40 -32
View File
@@ -1,13 +1,15 @@
# Plan: packages — hierarchical builds for libraries and binaries # Plan: packages — hierarchical builds for libraries and binaries
**Status: in progress.** Implemented on branch `claude/build-packages-plan-174144`: **Status: complete** (branch `claude/build-packages-plan-174144`). Phase 0
phase 0 (`build-support`), phase 1 (all six library domains as packages, root as (`build-support`), phase 1 (all six library domains), phase 2 (every binary —
the pilot consumer), and the first phase-2 binary package — `pci-bus`, a the pci-bus pilot first, then services, drivers, and test fixtures in waves;
deliberate deviation from wave A's two-small-services opener, because a driver multi-binary directories like ps2-bus and usb-hid are one package exporting
with per-binary extras (device-manager-protocol, pci-class) exercises the several artifacts, and the acpi/fdt discovery pair each export an artifact
template harder than a plain service. Every phase landed green (unit tests, the named "discovery" that the root's -Ddiscovery picks between), and phase 3 (the
QEMU suite at parity with main, boot-image file list unchanged). Remaining: root split into `build/images.zig` + `build/qemu.zig`; the root `build.zig` is
waves B–D of phase 2, then phase 3. ~460 lines of orchestration, down from ~1,250). Every phase landed green: unit
tests, the QEMU suite at parity with main, boot-image file list unchanged.
Still future: `lazyDependency` for image-specific builds (What-this-buys #4).
## Why ## Why
@@ -99,36 +101,40 @@ every domain's test step). The root build swaps its `createModule` calls for
touching 30 binaries. touching 30 binaries.
**Phase 2 — binaries become packages, in waves.** The template was shaken out **Phase 2 — binaries become packages, in waves.** The template was shaken out
by the pci-bus pilot (see Status). Wave A: services. Wave B: the remaining by the pci-bus pilot (see Status). Wave A: services (done). Wave B: the
drivers. Wave C: test fixtures. Root build shrinks to orchestration per wave. remaining drivers (done). Wave C: test fixtures (done). Root build shrank to
orchestration per wave. init's `-Dserial` heartbeat flag rides a dependency
option; a directory with several binaries (ps2-bus, usb-hid) is one package
exporting several artifacts.
**Phase 3 — root cleanup.** Split what remains of the root build into **Phase 3 — root cleanup (done).** What remained of the root build split into
`build/images.zig`, `build/qemu.zig`, imported by a short root `build.zig`. `build/images.zig` (the FHS install tree, boot manifest + capsule, FAT32
images, release ISO, check steps) and `build/qemu.zig` (the run steps + OVMF
probing), imported by a short root `build.zig`.
**Afterwards** (outside this plan): the intel-uhd-graphics-750 driver is **Afterwards** (outside this plan): the intel-uhd-graphics-750 driver is
(re)created as a greenfield package — the "Adding a driver" checklist's build (re)created as a greenfield package — the "Adding a driver" checklist's build
step (docs/device-driver-development/devices-csv.md, step 1) gets rewritten step (docs/device-driver-development/devices-csv.md, step 1) gets rewritten
against the package template at that point. against the package template at that point.
## Execution notes (for whichever session runs the remaining waves) ## Execution notes (the finished shape)
Anchors in the root `build.zig` as it stands after the pilot: - The shared recipe lives in `build-support/build.zig`: `userBinary` (what
every binary package calls), `userBinaryFromImports` (the underlying
- The shared recipe lives in `build-support/build.zig`: `userBinary` (domains recipe), and `defaultImports` — the ONE list of default modules. The `start`
form, what binary packages call), `userBinaryFromImports` (the underlying root shim and `user.ld` are named through the kernel package
recipe root's stanzas still use), and `defaultImports` — the ONE list of (Dependency.path).
default modules; root and the packages both draw from it. The `start` root - Adding a binary = adding a directory with source + a ~15-line build.zig +
shim and `user.ld` are named through the kernel package (Dependency.path). zon (copy any existing binary package, e.g.
- `programModule(<exe>).addImport(...)` calls in root (search `programModule`) `system/drivers/pci-bus/build.zig`), then one dependency + one bundled
are the per-binary extra imports — the data for each binary's future entry in the root build.zig and one zon line. Per-binary extras go through
build.zig. `system/drivers/pci-bus/build.zig` is the template to copy. `build_support.programModule(exe).addImport(...)` inside the package.
- The boot-tree array (search `"etc/init.csv"` or `.getEmittedBin()`) is the - The boot-tree array in the root (search `"etc/init.csv"` or
image file list — the authoritative before/after comparison target. A `.getEmittedBin()`) is the image file list — the authoritative comparison
converted binary's stanza becomes target for any future build change.
`b.dependency("<name>", .{}).artifact("<name>")` plus a zon entry. - Package unit tests live in each package's own `test` step; the root
- The QEMU size-check tests hardcode source paths (search aggregate depends on every test-bearing package's step, so `zig build test`
`virtio-gpu-protocol.zig` in the root test list) — they move with their at the root still runs everything.
binaries' waves.
Verification per phase: Verification per phase:
@@ -141,7 +147,8 @@ Verification per phase:
Context a fresh session should read first: this doc, docs/testing.md, Context a fresh session should read first: this doc, docs/testing.md,
docs/coding-standards.md (kebab-case names, no abbreviations), and the docs/coding-standards.md (kebab-case names, no abbreviations), and the
`addUserBinaryImpl` body. Commit style: no Co-Authored-By trailers. `userBinary`/`userBinaryFromImports` bodies in build-support/build.zig. Commit
style: no Co-Authored-By trailers.
## Risks / notes ## Risks / notes
@@ -149,7 +156,8 @@ docs/coding-standards.md (kebab-case names, no abbreviations), and the
between releases; the work pins against the repo's current Zig and any between releases; the work pins against the repo's current Zig and any
upgrade lands separately, never mid-phase. upgrade lands separately, never mid-phase.
- The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol - The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol
struct sizes in the root build) — phase 2 wave C must carry those along. struct sizes) — they moved into their binaries' packages with their waves,
discharging the carry-along obligation.
- Doc updates ride each phase: docs/README.md (repo layout + source map), - Doc updates ride each phase: docs/README.md (repo layout + source map),
docs/device-driver-development/devices-csv.md ("Adding a driver", step 1), docs/device-driver-development/devices-csv.md ("Adding a driver", step 1),
and the docs that cite the build recipe (driver-model.md, threading.md, and the docs that cite the build recipe (driver-model.md, threading.md,
@@ -96,7 +96,12 @@ in different namespaces — against the right `bus` column.
## Adding a driver ## Adding a driver
1. Build the driver binary and bundle it at `/system/drivers/<name>` (build.zig). 1. Create `system/drivers/<name>/` with the driver source plus a ~15-line
package `build.zig` + `build.zig.zon` (copy an existing driver package,
e.g. `system/drivers/pci-bus/`; per-driver extras go through
`build_support.programModule`). Then bundle it at `/system/drivers/<name>`:
one dependency + one bundled entry in the root `build.zig`, one line in the
root `build.zig.zon`.
2. Add a row to `etc/devices.csv` naming the identity it binds and its full path. 2. Add a row to `etc/devices.csv` naming the identity it binds and its full path.
No device-manager change is required — the registry is the seam. No device-manager change is required — the registry is the seam.
+1 -1
View File
@@ -29,7 +29,7 @@ which one you're holding decides what you can do.
- **The PCI class-0x03 device is the raw controller** — BARs, config space, registers, - **The PCI class-0x03 device is the raw controller** — BARs, config space, registers,
IO ports. It is what you actually *own* after boot. On QEMU's emulated adapter IO ports. It is what you actually *own* after boot. On QEMU's emulated adapter
([`-device VGA,edid=on`](../../build.zig), the Bochs VBE/DISPI model) the `base` GOP handed ([`-device VGA,edid=on`](../../build/qemu.zig), the Bochs VBE/DISPI model) the `base` GOP handed
you *is* that device's linear-framebuffer BAR — the same physical memory, seen through you *is* that device's linear-framebuffer BAR — the same physical memory, seen through
a different door. On a real discrete GPU, GOP's `base` is an aperture inside the GPU's a different door. On a real discrete GPU, GOP's `base` is an aperture inside the GPU's
VRAM BAR. danos already decodes this device VRAM BAR. danos already decodes this device
@@ -145,9 +145,8 @@ The build side of this has since landed: the shared recipe in
device/service clients (`driver`, `block`, `display`, `input`), plus `mmio`, device/service clients (`driver`, `block`, `display`, `input`), plus `mmio`,
`xkeyboard-config`, `acpi-ids` — into every user binary, and per-binary extras — `xkeyboard-config`, `acpi-ids` — into every user binary, and per-binary extras —
protocol modules, bus logic — are added with `programModule(exe).addImport(...)`. protocol modules, bus logic — are added with `programModule(exe).addImport(...)`.
Most binaries are still built by the root `build.zig`'s stanzas through that recipe; Every binary owns a package with its own ~15-line `build.zig` calling that recipe
a binary can instead own a package with its own ~15-line `build.zig` (pci-bus is the (see [build-packages-plan.md](../build-packages-plan.md)). That's the *entire*
first — see [build-packages-plan.md](../build-packages-plan.md)). That's the *entire*
mechanism — Zig modules already give you everything else. mechanism — Zig modules already give you everything else.
The discipline that makes this work: **a class driver must not import a bus's *hardware* The discipline that makes this work: **a class driver must not import a bus's *hardware*
+3 -2
View File
@@ -24,8 +24,9 @@ EFI/BOOT/BOOTX64.efi <- the "removable media" default for x86-64
``` ```
The boot volume is **FHS-shaped** (see the repository-layout note in The boot volume is **FHS-shaped** (see the repository-layout note in
[README.md](../README.md)): `build.zig` installs `boot/efi.zig` (built for the `uefi` [README.md](../README.md)): the root `build.zig` compiles `boot/efi.zig` (built
target) at `EFI/BOOT/BOOTX64.efi` — the one path UEFI firmware fixes — and lays for the `uefi` target) and `build/images.zig` places it at
`EFI/BOOT/BOOTX64.efi` — the one path UEFI firmware fixes — and lays
the rest out by FHS path: the kernel at `system/kernel`, init at the rest out by FHS path: the kernel at `system/kernel`, init at
`system/services/init`, the pre-packed boot capsule at `boot/system.img` `system/services/init`, the pre-packed boot capsule at `boot/system.img`
([system-image.md](system-image.md)). ([system-image.md](system-image.md)).
+6 -5
View File
@@ -22,11 +22,12 @@ lands on its own and ends in a **verifiable gate** — shaped for a `/loop` run,
## Conventions ## Conventions
Follow [coding-standards.md](../coding-standards.md): spell out non-acronym abbreviations, Follow [coding-standards.md](../coding-standards.md): spell out non-acronym abbreviations,
kebab-case file names, no `Co-Authored-By` trailers. New user binaries go through kebab-case file names, no `Co-Authored-By` trailers. New user binaries are
`addUserBinary` (with the new `threaded` flag where a binary spawns threads) and get packages whose build.zig calls `build_support.userBinary` (with `.threaded =
packed into the initial-ramdisk; new syscalls extend [abi.zig](../../system/abi.zig) true` where a binary spawns threads) and get packed into the initial-ramdisk;
`SystemCall` + a `library/runtime` wrapper; test services live beside the code they new syscalls extend [abi.zig](../../system/abi.zig) `SystemCall` + a
exercise and register a `ServiceId` if they must be looked up. `library/kernel` wrapper; test services live beside the code they exercise and
register a `ServiceId` if they must be looked up.
## How to verify along the way ## How to verify along the way
+3 -3
View File
@@ -237,9 +237,9 @@ see the intro). Two scoped pieces, as built:
### Build: multi-threaded codegen, opt-in ### Build: multi-threaded codegen, opt-in
A binary opts in with `addThreadedUserBinary` in the root `build.zig` (or A binary opts in with `.threaded = true` in its package's
`.threaded = true` in a binary package's `build_support.userBinary` call) — the `build_support.userBinary` call — the shared recipe in build-support then builds it
shared recipe in build-support then builds it `single_threaded = false` — so atomics `single_threaded = false` — so atomics
and (later) TLS are real. Threads and atomics are unsound in a `single_threaded` image, and (later) TLS are real. Threads and atomics are unsound in a `single_threaded` image,
so a binary must opt in **before** it may call `Thread.spawn`. Everyone else so a binary must opt in **before** it may call `Thread.spawn`. Everyone else
stays single-threaded and lean. stays single-threaded and lean.
+1 -1
View File
@@ -108,7 +108,7 @@ hypervisor configured for UEFI firmware and an xHCI USB controller.
- **UEFI only.** A custom UEFI application loader is installed to - **UEFI only.** A custom UEFI application loader is installed to
`\EFI\BOOT\BOOTX64.efi`. There is **no BIOS, multiboot, or limine** path. The `\EFI\BOOT\BOOTX64.efi`. There is **no BIOS, multiboot, or limine** path. The
loader tolerates UEFI Class-3 machines with no legacy PIC/PIT. loader tolerates UEFI Class-3 machines with no legacy PIC/PIT.
(`build.zig:246`, `boot/efi.zig`) (`build/images.zig` — the EFI/BOOT install — and `boot/efi.zig`)
- **ACPI is the hardware-discovery mechanism.** The RSDP is taken from the UEFI - **ACPI is the hardware-discovery mechanism.** The RSDP is taken from the UEFI
configuration table (ACPI 2.0 GUID preferred, 1.0 fallback). Without a valid configuration table (ACPI 2.0 GUID preferred, 1.0 fallback). Without a valid
RSDP there is **no device discovery** — no SMP, no IOAPIC routing, no PCI/USB. RSDP there is **no device discovery** — no SMP, no IOAPIC routing, no PCI/USB.
+3 -1
View File
@@ -12,7 +12,9 @@ There are two layers:
`system/abi.zig`, `library/device/model/device-abi.zig`) now spans ~26 modules: `system/abi.zig`, `library/device/model/device-abi.zig`) now spans ~26 modules:
protocol and on-wire definitions (VFS, USB, virtio-gpu), the FAT engine, the protocol and on-wire definitions (VFS, USB, virtio-gpu), the FAT engine, the
display compositor, PS/2 and HID decoding, the kernel log ring, and the display compositor, PS/2 and HID decoding, the kernel log ring, and the
runtime's `time`/`thread` — the full list is the test step in `build.zig`. runtime's `time`/`thread` — the list is distributed across the library-domain
and binary packages' own `test` steps, which the root `zig build test`
aggregates (docs/build-packages-plan.md).
These compile for the host and run natively. These compile for the host and run natively.
- **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel - **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel
and check its behaviour. This is the interesting part. and check its behaviour. This is the interesting part.