diff --git a/docs/README.md b/docs/README.md index e862288..c1412d1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -266,14 +266,16 @@ test/ → /test the test tree: the QEMU harness (qemu_test.py, h build-support/ the danos build API (build-time only, nothing on the image): the shared user-binary recipe + default-import wiring every build file consumes (docs/build-packages-plan.md) +build/ root-build helpers: image assembly (images.zig) + the QEMU + run steps (qemu.zig) tools/ host-side build scripts ``` **Builds are packages** (docs/build-packages-plan.md): each `library/` domain owns a `build.zig`/`build.zig.zon` exporting its modules (with a standalone `zig build test`), -binaries are converting one directory at a time to ~15-line package builds (`pci-bus` -is the first), and the root `build.zig` orchestrates — image assembly, QEMU, the -aggregate test step. +every binary directory is a ~15-line package build, and the root `build.zig` +orchestrates — the kernel + loader, what ships, and the aggregate test step — with +image assembly in `build/images.zig` and the QEMU run steps in `build/qemu.zig`. **Wire protocols live in `library/protocol/`**, one module per directory (`library/protocol/vfs/vfs-protocol.zig` is the `vfs-protocol` module), imported by module @@ -329,5 +331,7 @@ exception in [coding-standards.md](coding-standards.md) applies to that seam. | System services (init, the `fat` filesystem, the device-manager) | `system/services/` | | Device drivers, one sub-project each (`pci-bus`, `ps2-bus`, `usb-xhci-bus` bus drivers) | `system/drivers/` | | On-image test fixtures for the QEMU cases (`vfs-test`, `crash-test`, `thread-test`, …) → `/test/system/services` | `test/system/services/` | -| Build orchestration + `run-x86-64` (QEMU/OVMF) + `release-x86-64` (the flashable ISO) | `build.zig` (root; the shared user-binary recipe is `build-support/`, and each `library/` domain + packaged binary carries its own `build.zig`) | +| Build orchestration (kernel + loader, what ships, the aggregate test step) | `build.zig` (root; the shared user-binary recipe is `build-support/`, and each `library/` domain + binary package carries its own `build.zig`) | +| Image assembly + `release-x86-64` (the flashable ISO) | `build/images.zig` | +| `run-x86-64` / `run-x86-64-gpu` (QEMU/OVMF) | `build/qemu.zig` | | QEMU integration test harness | `test/qemu_test.py` | diff --git a/docs/build-packages-plan.md b/docs/build-packages-plan.md index 7b8a646..6e00dcf 100644 --- a/docs/build-packages-plan.md +++ b/docs/build-packages-plan.md @@ -1,13 +1,15 @@ # Plan: packages — hierarchical builds for libraries and binaries -**Status: in progress.** Implemented on branch `claude/build-packages-plan-174144`: -phase 0 (`build-support`), phase 1 (all six library domains as packages, root as -the pilot consumer), and the first phase-2 binary package — `pci-bus`, a -deliberate deviation from wave A's two-small-services opener, because a driver -with per-binary extras (device-manager-protocol, pci-class) exercises the -template harder than a plain service. Every phase landed green (unit tests, the -QEMU suite at parity with main, boot-image file list unchanged). Remaining: -waves B–D of phase 2, then phase 3. +**Status: complete** (branch `claude/build-packages-plan-174144`). Phase 0 +(`build-support`), phase 1 (all six library domains), phase 2 (every binary — +the pci-bus pilot first, then services, drivers, and test fixtures in waves; +multi-binary directories like ps2-bus and usb-hid are one package exporting +several artifacts, and the acpi/fdt discovery pair each export an artifact +named "discovery" that the root's -Ddiscovery picks between), and phase 3 (the +root split into `build/images.zig` + `build/qemu.zig`; the root `build.zig` is +~460 lines of orchestration, down from ~1,250). Every phase landed green: unit +tests, the QEMU suite at parity with main, boot-image file list unchanged. +Still future: `lazyDependency` for image-specific builds (What-this-buys #4). ## Why @@ -99,36 +101,40 @@ every domain's test step). The root build swaps its `createModule` calls for touching 30 binaries. **Phase 2 — binaries become packages, in waves.** The template was shaken out -by the pci-bus pilot (see Status). Wave A: services. Wave B: the remaining -drivers. Wave C: test fixtures. Root build shrinks to orchestration per wave. +by the pci-bus pilot (see Status). Wave A: services (done). Wave B: the +remaining drivers (done). Wave C: test fixtures (done). Root build shrank to +orchestration per wave. init's `-Dserial` heartbeat flag rides a dependency +option; a directory with several binaries (ps2-bus, usb-hid) is one package +exporting several artifacts. -**Phase 3 — root cleanup.** Split what remains of the root build into -`build/images.zig`, `build/qemu.zig`, imported by a short root `build.zig`. +**Phase 3 — root cleanup (done).** What remained of the root build split into +`build/images.zig` (the FHS install tree, boot manifest + capsule, FAT32 +images, release ISO, check steps) and `build/qemu.zig` (the run steps + OVMF +probing), imported by a short root `build.zig`. **Afterwards** (outside this plan): the intel-uhd-graphics-750 driver is (re)created as a greenfield package — the "Adding a driver" checklist's build step (docs/device-driver-development/devices-csv.md, step 1) gets rewritten against the package template at that point. -## Execution notes (for whichever session runs the remaining waves) +## Execution notes (the finished shape) -Anchors in the root `build.zig` as it stands after the pilot: - -- The shared recipe lives in `build-support/build.zig`: `userBinary` (domains - form, what binary packages call), `userBinaryFromImports` (the underlying - recipe root's stanzas still use), and `defaultImports` — the ONE list of - default modules; root and the packages both draw from it. The `start` root - shim and `user.ld` are named through the kernel package (Dependency.path). -- `programModule().addImport(...)` calls in root (search `programModule`) - are the per-binary extra imports — the data for each binary's future - build.zig. `system/drivers/pci-bus/build.zig` is the template to copy. -- The boot-tree array (search `"etc/init.csv"` or `.getEmittedBin()`) is the - image file list — the authoritative before/after comparison target. A - converted binary's stanza becomes - `b.dependency("", .{}).artifact("")` plus a zon entry. -- The QEMU size-check tests hardcode source paths (search - `virtio-gpu-protocol.zig` in the root test list) — they move with their - binaries' waves. +- The shared recipe lives in `build-support/build.zig`: `userBinary` (what + every binary package calls), `userBinaryFromImports` (the underlying + recipe), and `defaultImports` — the ONE list of default modules. The `start` + root shim and `user.ld` are named through the kernel package + (Dependency.path). +- Adding a binary = adding a directory with source + a ~15-line build.zig + + zon (copy any existing binary package, e.g. + `system/drivers/pci-bus/build.zig`), then one dependency + one bundled + entry in the root build.zig and one zon line. Per-binary extras go through + `build_support.programModule(exe).addImport(...)` inside the package. +- The boot-tree array in the root (search `"etc/init.csv"` or + `.getEmittedBin()`) is the image file list — the authoritative comparison + target for any future build change. +- Package unit tests live in each package's own `test` step; the root + aggregate depends on every test-bearing package's step, so `zig build test` + at the root still runs everything. Verification per phase: @@ -141,7 +147,8 @@ Verification per phase: Context a fresh session should read first: this doc, docs/testing.md, docs/coding-standards.md (kebab-case names, no abbreviations), and the -`addUserBinaryImpl` body. Commit style: no Co-Authored-By trailers. +`userBinary`/`userBinaryFromImports` bodies in build-support/build.zig. Commit +style: no Co-Authored-By trailers. ## Risks / notes @@ -149,7 +156,8 @@ docs/coding-standards.md (kebab-case names, no abbreviations), and the between releases; the work pins against the repo's current Zig and any upgrade lands separately, never mid-phase. - The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol - struct sizes in the root build) — phase 2 wave C must carry those along. + struct sizes) — they moved into their binaries' packages with their waves, + discharging the carry-along obligation. - Doc updates ride each phase: docs/README.md (repo layout + source map), docs/device-driver-development/devices-csv.md ("Adding a driver", step 1), and the docs that cite the build recipe (driver-model.md, threading.md, diff --git a/docs/device-driver-development/devices-csv.md b/docs/device-driver-development/devices-csv.md index bceede4..44a66f4 100644 --- a/docs/device-driver-development/devices-csv.md +++ b/docs/device-driver-development/devices-csv.md @@ -96,7 +96,12 @@ in different namespaces — against the right `bus` column. ## Adding a driver -1. Build the driver binary and bundle it at `/system/drivers/` (build.zig). +1. Create `system/drivers//` with the driver source plus a ~15-line + package `build.zig` + `build.zig.zon` (copy an existing driver package, + e.g. `system/drivers/pci-bus/`; per-driver extras go through + `build_support.programModule`). Then bundle it at `/system/drivers/`: + one dependency + one bundled entry in the root `build.zig`, one line in the + root `build.zig.zon`. 2. Add a row to `etc/devices.csv` naming the identity it binds and its full path. No device-manager change is required — the registry is the seam. diff --git a/docs/device-driver-development/display.md b/docs/device-driver-development/display.md index 23602dc..05b6b54 100644 --- a/docs/device-driver-development/display.md +++ b/docs/device-driver-development/display.md @@ -29,7 +29,7 @@ which one you're holding decides what you can do. - **The PCI class-0x03 device is the raw controller** — BARs, config space, registers, IO ports. It is what you actually *own* after boot. On QEMU's emulated adapter - ([`-device VGA,edid=on`](../../build.zig), the Bochs VBE/DISPI model) the `base` GOP handed + ([`-device VGA,edid=on`](../../build/qemu.zig), the Bochs VBE/DISPI model) the `base` GOP handed you *is* that device's linear-framebuffer BAR — the same physical memory, seen through a different door. On a real discrete GPU, GOP's `base` is an aperture inside the GPU's VRAM BAR. danos already decodes this device diff --git a/docs/device-driver-development/driver-model.md b/docs/device-driver-development/driver-model.md index 1e41a86..8fcf1ea 100644 --- a/docs/device-driver-development/driver-model.md +++ b/docs/device-driver-development/driver-model.md @@ -145,9 +145,8 @@ The build side of this has since landed: the shared recipe in device/service clients (`driver`, `block`, `display`, `input`), plus `mmio`, `xkeyboard-config`, `acpi-ids` — into every user binary, and per-binary extras — protocol modules, bus logic — are added with `programModule(exe).addImport(...)`. -Most binaries are still built by the root `build.zig`'s stanzas through that recipe; -a binary can instead own a package with its own ~15-line `build.zig` (pci-bus is the -first — see [build-packages-plan.md](../build-packages-plan.md)). That's the *entire* +Every binary owns a package with its own ~15-line `build.zig` calling that recipe +(see [build-packages-plan.md](../build-packages-plan.md)). That's the *entire* mechanism — Zig modules already give you everything else. The discipline that makes this work: **a class driver must not import a bus's *hardware* diff --git a/docs/os-development/efi.md b/docs/os-development/efi.md index 41217e2..5f4781a 100644 --- a/docs/os-development/efi.md +++ b/docs/os-development/efi.md @@ -24,8 +24,9 @@ EFI/BOOT/BOOTX64.efi <- the "removable media" default for x86-64 ``` The boot volume is **FHS-shaped** (see the repository-layout note in -[README.md](../README.md)): `build.zig` installs `boot/efi.zig` (built for the `uefi` -target) at `EFI/BOOT/BOOTX64.efi` — the one path UEFI firmware fixes — and lays +[README.md](../README.md)): the root `build.zig` compiles `boot/efi.zig` (built +for the `uefi` target) and `build/images.zig` places it at +`EFI/BOOT/BOOTX64.efi` — the one path UEFI firmware fixes — and lays the rest out by FHS path: the kernel at `system/kernel`, init at `system/services/init`, the pre-packed boot capsule at `boot/system.img` ([system-image.md](system-image.md)). diff --git a/docs/os-development/threading-plan.md b/docs/os-development/threading-plan.md index f2f5481..09180e6 100644 --- a/docs/os-development/threading-plan.md +++ b/docs/os-development/threading-plan.md @@ -22,11 +22,12 @@ lands on its own and ends in a **verifiable gate** — shaped for a `/loop` run, ## Conventions Follow [coding-standards.md](../coding-standards.md): spell out non-acronym abbreviations, -kebab-case file names, no `Co-Authored-By` trailers. New user binaries go through -`addUserBinary` (with the new `threaded` flag where a binary spawns threads) and get -packed into the initial-ramdisk; new syscalls extend [abi.zig](../../system/abi.zig) -`SystemCall` + a `library/runtime` wrapper; test services live beside the code they -exercise and register a `ServiceId` if they must be looked up. +kebab-case file names, no `Co-Authored-By` trailers. New user binaries are +packages whose build.zig calls `build_support.userBinary` (with `.threaded = +true` where a binary spawns threads) and get packed into the initial-ramdisk; +new syscalls extend [abi.zig](../../system/abi.zig) `SystemCall` + a +`library/kernel` wrapper; test services live beside the code they exercise and +register a `ServiceId` if they must be looked up. ## How to verify along the way diff --git a/docs/os-development/threading.md b/docs/os-development/threading.md index eaa94f4..5b7d3a7 100644 --- a/docs/os-development/threading.md +++ b/docs/os-development/threading.md @@ -237,9 +237,9 @@ see the intro). Two scoped pieces, as built: ### Build: multi-threaded codegen, opt-in -A binary opts in with `addThreadedUserBinary` in the root `build.zig` (or -`.threaded = true` in a binary package's `build_support.userBinary` call) — the -shared recipe in build-support then builds it `single_threaded = false` — so atomics +A binary opts in with `.threaded = true` in its package's +`build_support.userBinary` call — the shared recipe in build-support then builds it +`single_threaded = false` — so atomics and (later) TLS are real. Threads and atomics are unsound in a `single_threaded` image, so a binary must opt in **before** it may call `Thread.spawn`. Everyone else stays single-threaded and lean. diff --git a/docs/system-requirements.md b/docs/system-requirements.md index 2df286f..854ef7e 100644 --- a/docs/system-requirements.md +++ b/docs/system-requirements.md @@ -108,7 +108,7 @@ hypervisor configured for UEFI firmware and an xHCI USB controller. - **UEFI only.** A custom UEFI application loader is installed to `\EFI\BOOT\BOOTX64.efi`. There is **no BIOS, multiboot, or limine** path. The loader tolerates UEFI Class-3 machines with no legacy PIC/PIT. - (`build.zig:246`, `boot/efi.zig`) + (`build/images.zig` — the EFI/BOOT install — and `boot/efi.zig`) - **ACPI is the hardware-discovery mechanism.** The RSDP is taken from the UEFI configuration table (ACPI 2.0 GUID preferred, 1.0 fallback). Without a valid RSDP there is **no device discovery** — no SMP, no IOAPIC routing, no PCI/USB. diff --git a/docs/testing.md b/docs/testing.md index a442577..966f2f8 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -12,7 +12,9 @@ There are two layers: `system/abi.zig`, `library/device/model/device-abi.zig`) now spans ~26 modules: protocol and on-wire definitions (VFS, USB, virtio-gpu), the FAT engine, the display compositor, PS/2 and HID decoding, the kernel log ring, and the - runtime's `time`/`thread` — the full list is the test step in `build.zig`. + runtime's `time`/`thread` — the list is distributed across the library-domain + and binary packages' own `test` steps, which the root `zig build test` + aggregates (docs/build-packages-plan.md). These compile for the host and run natively. - **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel and check its behaviour. This is the interesting part.