docs: catch the docs up with the finished package split
The plan doc's status records completion (all waves + phase 3) and its execution notes describe the finished shape; the fresh-session pointer names build-support's userBinary instead of the deleted addUserBinaryImpl, and the size-check carry-along note is discharged. README gains the build/ directory in the layout tree and splits the source-map row across root build.zig / build/images.zig / build/qemu.zig. testing.md points at the distributed per-package test steps; threading.md, threading-plan.md, driver-model.md, efi.md, display.md, system-requirements.md, and devices-csv.md's adding-a- driver checklist stop describing the pre-package build.
This commit is contained in:
+8
-4
@@ -266,14 +266,16 @@ test/ → /test the test tree: the QEMU harness (qemu_test.py, h
|
|||||||
build-support/ the danos build API (build-time only, nothing on the image):
|
build-support/ the danos build API (build-time only, nothing on the image):
|
||||||
the shared user-binary recipe + default-import wiring every
|
the shared user-binary recipe + default-import wiring every
|
||||||
build file consumes (docs/build-packages-plan.md)
|
build file consumes (docs/build-packages-plan.md)
|
||||||
|
build/ root-build helpers: image assembly (images.zig) + the QEMU
|
||||||
|
run steps (qemu.zig)
|
||||||
tools/ host-side build scripts
|
tools/ host-side build scripts
|
||||||
```
|
```
|
||||||
|
|
||||||
**Builds are packages** (docs/build-packages-plan.md): each `library/` domain owns a
|
**Builds are packages** (docs/build-packages-plan.md): each `library/` domain owns a
|
||||||
`build.zig`/`build.zig.zon` exporting its modules (with a standalone `zig build test`),
|
`build.zig`/`build.zig.zon` exporting its modules (with a standalone `zig build test`),
|
||||||
binaries are converting one directory at a time to ~15-line package builds (`pci-bus`
|
every binary directory is a ~15-line package build, and the root `build.zig`
|
||||||
is the first), and the root `build.zig` orchestrates — image assembly, QEMU, the
|
orchestrates — the kernel + loader, what ships, and the aggregate test step — with
|
||||||
aggregate test step.
|
image assembly in `build/images.zig` and the QEMU run steps in `build/qemu.zig`.
|
||||||
|
|
||||||
**Wire protocols live in `library/protocol/`**, one module per directory
|
**Wire protocols live in `library/protocol/`**, one module per directory
|
||||||
(`library/protocol/vfs/vfs-protocol.zig` is the `vfs-protocol` module), imported by module
|
(`library/protocol/vfs/vfs-protocol.zig` is the `vfs-protocol` module), imported by module
|
||||||
@@ -329,5 +331,7 @@ exception in [coding-standards.md](coding-standards.md) applies to that seam.
|
|||||||
| System services (init, the `fat` filesystem, the device-manager) | `system/services/` |
|
| System services (init, the `fat` filesystem, the device-manager) | `system/services/` |
|
||||||
| Device drivers, one sub-project each (`pci-bus`, `ps2-bus`, `usb-xhci-bus` bus drivers) | `system/drivers/` |
|
| Device drivers, one sub-project each (`pci-bus`, `ps2-bus`, `usb-xhci-bus` bus drivers) | `system/drivers/` |
|
||||||
| On-image test fixtures for the QEMU cases (`vfs-test`, `crash-test`, `thread-test`, …) → `/test/system/services` | `test/system/services/` |
|
| On-image test fixtures for the QEMU cases (`vfs-test`, `crash-test`, `thread-test`, …) → `/test/system/services` | `test/system/services/` |
|
||||||
| Build orchestration + `run-x86-64` (QEMU/OVMF) + `release-x86-64` (the flashable ISO) | `build.zig` (root; the shared user-binary recipe is `build-support/`, and each `library/` domain + packaged binary carries its own `build.zig`) |
|
| Build orchestration (kernel + loader, what ships, the aggregate test step) | `build.zig` (root; the shared user-binary recipe is `build-support/`, and each `library/` domain + binary package carries its own `build.zig`) |
|
||||||
|
| Image assembly + `release-x86-64` (the flashable ISO) | `build/images.zig` |
|
||||||
|
| `run-x86-64` / `run-x86-64-gpu` (QEMU/OVMF) | `build/qemu.zig` |
|
||||||
| QEMU integration test harness | `test/qemu_test.py` |
|
| QEMU integration test harness | `test/qemu_test.py` |
|
||||||
|
|||||||
+40
-32
@@ -1,13 +1,15 @@
|
|||||||
# Plan: packages — hierarchical builds for libraries and binaries
|
# Plan: packages — hierarchical builds for libraries and binaries
|
||||||
|
|
||||||
**Status: in progress.** Implemented on branch `claude/build-packages-plan-174144`:
|
**Status: complete** (branch `claude/build-packages-plan-174144`). Phase 0
|
||||||
phase 0 (`build-support`), phase 1 (all six library domains as packages, root as
|
(`build-support`), phase 1 (all six library domains), phase 2 (every binary —
|
||||||
the pilot consumer), and the first phase-2 binary package — `pci-bus`, a
|
the pci-bus pilot first, then services, drivers, and test fixtures in waves;
|
||||||
deliberate deviation from wave A's two-small-services opener, because a driver
|
multi-binary directories like ps2-bus and usb-hid are one package exporting
|
||||||
with per-binary extras (device-manager-protocol, pci-class) exercises the
|
several artifacts, and the acpi/fdt discovery pair each export an artifact
|
||||||
template harder than a plain service. Every phase landed green (unit tests, the
|
named "discovery" that the root's -Ddiscovery picks between), and phase 3 (the
|
||||||
QEMU suite at parity with main, boot-image file list unchanged). Remaining:
|
root split into `build/images.zig` + `build/qemu.zig`; the root `build.zig` is
|
||||||
waves B–D of phase 2, then phase 3.
|
~460 lines of orchestration, down from ~1,250). Every phase landed green: unit
|
||||||
|
tests, the QEMU suite at parity with main, boot-image file list unchanged.
|
||||||
|
Still future: `lazyDependency` for image-specific builds (What-this-buys #4).
|
||||||
|
|
||||||
## Why
|
## Why
|
||||||
|
|
||||||
@@ -99,36 +101,40 @@ every domain's test step). The root build swaps its `createModule` calls for
|
|||||||
touching 30 binaries.
|
touching 30 binaries.
|
||||||
|
|
||||||
**Phase 2 — binaries become packages, in waves.** The template was shaken out
|
**Phase 2 — binaries become packages, in waves.** The template was shaken out
|
||||||
by the pci-bus pilot (see Status). Wave A: services. Wave B: the remaining
|
by the pci-bus pilot (see Status). Wave A: services (done). Wave B: the
|
||||||
drivers. Wave C: test fixtures. Root build shrinks to orchestration per wave.
|
remaining drivers (done). Wave C: test fixtures (done). Root build shrank to
|
||||||
|
orchestration per wave. init's `-Dserial` heartbeat flag rides a dependency
|
||||||
|
option; a directory with several binaries (ps2-bus, usb-hid) is one package
|
||||||
|
exporting several artifacts.
|
||||||
|
|
||||||
**Phase 3 — root cleanup.** Split what remains of the root build into
|
**Phase 3 — root cleanup (done).** What remained of the root build split into
|
||||||
`build/images.zig`, `build/qemu.zig`, imported by a short root `build.zig`.
|
`build/images.zig` (the FHS install tree, boot manifest + capsule, FAT32
|
||||||
|
images, release ISO, check steps) and `build/qemu.zig` (the run steps + OVMF
|
||||||
|
probing), imported by a short root `build.zig`.
|
||||||
|
|
||||||
**Afterwards** (outside this plan): the intel-uhd-graphics-750 driver is
|
**Afterwards** (outside this plan): the intel-uhd-graphics-750 driver is
|
||||||
(re)created as a greenfield package — the "Adding a driver" checklist's build
|
(re)created as a greenfield package — the "Adding a driver" checklist's build
|
||||||
step (docs/device-driver-development/devices-csv.md, step 1) gets rewritten
|
step (docs/device-driver-development/devices-csv.md, step 1) gets rewritten
|
||||||
against the package template at that point.
|
against the package template at that point.
|
||||||
|
|
||||||
## Execution notes (for whichever session runs the remaining waves)
|
## Execution notes (the finished shape)
|
||||||
|
|
||||||
Anchors in the root `build.zig` as it stands after the pilot:
|
- The shared recipe lives in `build-support/build.zig`: `userBinary` (what
|
||||||
|
every binary package calls), `userBinaryFromImports` (the underlying
|
||||||
- The shared recipe lives in `build-support/build.zig`: `userBinary` (domains
|
recipe), and `defaultImports` — the ONE list of default modules. The `start`
|
||||||
form, what binary packages call), `userBinaryFromImports` (the underlying
|
root shim and `user.ld` are named through the kernel package
|
||||||
recipe root's stanzas still use), and `defaultImports` — the ONE list of
|
(Dependency.path).
|
||||||
default modules; root and the packages both draw from it. The `start` root
|
- Adding a binary = adding a directory with source + a ~15-line build.zig +
|
||||||
shim and `user.ld` are named through the kernel package (Dependency.path).
|
zon (copy any existing binary package, e.g.
|
||||||
- `programModule(<exe>).addImport(...)` calls in root (search `programModule`)
|
`system/drivers/pci-bus/build.zig`), then one dependency + one bundled
|
||||||
are the per-binary extra imports — the data for each binary's future
|
entry in the root build.zig and one zon line. Per-binary extras go through
|
||||||
build.zig. `system/drivers/pci-bus/build.zig` is the template to copy.
|
`build_support.programModule(exe).addImport(...)` inside the package.
|
||||||
- The boot-tree array (search `"etc/init.csv"` or `.getEmittedBin()`) is the
|
- The boot-tree array in the root (search `"etc/init.csv"` or
|
||||||
image file list — the authoritative before/after comparison target. A
|
`.getEmittedBin()`) is the image file list — the authoritative comparison
|
||||||
converted binary's stanza becomes
|
target for any future build change.
|
||||||
`b.dependency("<name>", .{}).artifact("<name>")` plus a zon entry.
|
- Package unit tests live in each package's own `test` step; the root
|
||||||
- The QEMU size-check tests hardcode source paths (search
|
aggregate depends on every test-bearing package's step, so `zig build test`
|
||||||
`virtio-gpu-protocol.zig` in the root test list) — they move with their
|
at the root still runs everything.
|
||||||
binaries' waves.
|
|
||||||
|
|
||||||
Verification per phase:
|
Verification per phase:
|
||||||
|
|
||||||
@@ -141,7 +147,8 @@ Verification per phase:
|
|||||||
|
|
||||||
Context a fresh session should read first: this doc, docs/testing.md,
|
Context a fresh session should read first: this doc, docs/testing.md,
|
||||||
docs/coding-standards.md (kebab-case names, no abbreviations), and the
|
docs/coding-standards.md (kebab-case names, no abbreviations), and the
|
||||||
`addUserBinaryImpl` body. Commit style: no Co-Authored-By trailers.
|
`userBinary`/`userBinaryFromImports` bodies in build-support/build.zig. Commit
|
||||||
|
style: no Co-Authored-By trailers.
|
||||||
|
|
||||||
## Risks / notes
|
## Risks / notes
|
||||||
|
|
||||||
@@ -149,7 +156,8 @@ docs/coding-standards.md (kebab-case names, no abbreviations), and the
|
|||||||
between releases; the work pins against the repo's current Zig and any
|
between releases; the work pins against the repo's current Zig and any
|
||||||
upgrade lands separately, never mid-phase.
|
upgrade lands separately, never mid-phase.
|
||||||
- The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol
|
- The QEMU size-check tests hardcode source paths (e.g. virtio-gpu protocol
|
||||||
struct sizes in the root build) — phase 2 wave C must carry those along.
|
struct sizes) — they moved into their binaries' packages with their waves,
|
||||||
|
discharging the carry-along obligation.
|
||||||
- Doc updates ride each phase: docs/README.md (repo layout + source map),
|
- Doc updates ride each phase: docs/README.md (repo layout + source map),
|
||||||
docs/device-driver-development/devices-csv.md ("Adding a driver", step 1),
|
docs/device-driver-development/devices-csv.md ("Adding a driver", step 1),
|
||||||
and the docs that cite the build recipe (driver-model.md, threading.md,
|
and the docs that cite the build recipe (driver-model.md, threading.md,
|
||||||
|
|||||||
@@ -96,7 +96,12 @@ in different namespaces — against the right `bus` column.
|
|||||||
|
|
||||||
## Adding a driver
|
## Adding a driver
|
||||||
|
|
||||||
1. Build the driver binary and bundle it at `/system/drivers/<name>` (build.zig).
|
1. Create `system/drivers/<name>/` with the driver source plus a ~15-line
|
||||||
|
package `build.zig` + `build.zig.zon` (copy an existing driver package,
|
||||||
|
e.g. `system/drivers/pci-bus/`; per-driver extras go through
|
||||||
|
`build_support.programModule`). Then bundle it at `/system/drivers/<name>`:
|
||||||
|
one dependency + one bundled entry in the root `build.zig`, one line in the
|
||||||
|
root `build.zig.zon`.
|
||||||
2. Add a row to `etc/devices.csv` naming the identity it binds and its full path.
|
2. Add a row to `etc/devices.csv` naming the identity it binds and its full path.
|
||||||
|
|
||||||
No device-manager change is required — the registry is the seam.
|
No device-manager change is required — the registry is the seam.
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ which one you're holding decides what you can do.
|
|||||||
|
|
||||||
- **The PCI class-0x03 device is the raw controller** — BARs, config space, registers,
|
- **The PCI class-0x03 device is the raw controller** — BARs, config space, registers,
|
||||||
IO ports. It is what you actually *own* after boot. On QEMU's emulated adapter
|
IO ports. It is what you actually *own* after boot. On QEMU's emulated adapter
|
||||||
([`-device VGA,edid=on`](../../build.zig), the Bochs VBE/DISPI model) the `base` GOP handed
|
([`-device VGA,edid=on`](../../build/qemu.zig), the Bochs VBE/DISPI model) the `base` GOP handed
|
||||||
you *is* that device's linear-framebuffer BAR — the same physical memory, seen through
|
you *is* that device's linear-framebuffer BAR — the same physical memory, seen through
|
||||||
a different door. On a real discrete GPU, GOP's `base` is an aperture inside the GPU's
|
a different door. On a real discrete GPU, GOP's `base` is an aperture inside the GPU's
|
||||||
VRAM BAR. danos already decodes this device
|
VRAM BAR. danos already decodes this device
|
||||||
|
|||||||
@@ -145,9 +145,8 @@ The build side of this has since landed: the shared recipe in
|
|||||||
device/service clients (`driver`, `block`, `display`, `input`), plus `mmio`,
|
device/service clients (`driver`, `block`, `display`, `input`), plus `mmio`,
|
||||||
`xkeyboard-config`, `acpi-ids` — into every user binary, and per-binary extras —
|
`xkeyboard-config`, `acpi-ids` — into every user binary, and per-binary extras —
|
||||||
protocol modules, bus logic — are added with `programModule(exe).addImport(...)`.
|
protocol modules, bus logic — are added with `programModule(exe).addImport(...)`.
|
||||||
Most binaries are still built by the root `build.zig`'s stanzas through that recipe;
|
Every binary owns a package with its own ~15-line `build.zig` calling that recipe
|
||||||
a binary can instead own a package with its own ~15-line `build.zig` (pci-bus is the
|
(see [build-packages-plan.md](../build-packages-plan.md)). That's the *entire*
|
||||||
first — see [build-packages-plan.md](../build-packages-plan.md)). That's the *entire*
|
|
||||||
mechanism — Zig modules already give you everything else.
|
mechanism — Zig modules already give you everything else.
|
||||||
|
|
||||||
The discipline that makes this work: **a class driver must not import a bus's *hardware*
|
The discipline that makes this work: **a class driver must not import a bus's *hardware*
|
||||||
|
|||||||
@@ -24,8 +24,9 @@ EFI/BOOT/BOOTX64.efi <- the "removable media" default for x86-64
|
|||||||
```
|
```
|
||||||
|
|
||||||
The boot volume is **FHS-shaped** (see the repository-layout note in
|
The boot volume is **FHS-shaped** (see the repository-layout note in
|
||||||
[README.md](../README.md)): `build.zig` installs `boot/efi.zig` (built for the `uefi`
|
[README.md](../README.md)): the root `build.zig` compiles `boot/efi.zig` (built
|
||||||
target) at `EFI/BOOT/BOOTX64.efi` — the one path UEFI firmware fixes — and lays
|
for the `uefi` target) and `build/images.zig` places it at
|
||||||
|
`EFI/BOOT/BOOTX64.efi` — the one path UEFI firmware fixes — and lays
|
||||||
the rest out by FHS path: the kernel at `system/kernel`, init at
|
the rest out by FHS path: the kernel at `system/kernel`, init at
|
||||||
`system/services/init`, the pre-packed boot capsule at `boot/system.img`
|
`system/services/init`, the pre-packed boot capsule at `boot/system.img`
|
||||||
([system-image.md](system-image.md)).
|
([system-image.md](system-image.md)).
|
||||||
|
|||||||
@@ -22,11 +22,12 @@ lands on its own and ends in a **verifiable gate** — shaped for a `/loop` run,
|
|||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
Follow [coding-standards.md](../coding-standards.md): spell out non-acronym abbreviations,
|
Follow [coding-standards.md](../coding-standards.md): spell out non-acronym abbreviations,
|
||||||
kebab-case file names, no `Co-Authored-By` trailers. New user binaries go through
|
kebab-case file names, no `Co-Authored-By` trailers. New user binaries are
|
||||||
`addUserBinary` (with the new `threaded` flag where a binary spawns threads) and get
|
packages whose build.zig calls `build_support.userBinary` (with `.threaded =
|
||||||
packed into the initial-ramdisk; new syscalls extend [abi.zig](../../system/abi.zig)
|
true` where a binary spawns threads) and get packed into the initial-ramdisk;
|
||||||
`SystemCall` + a `library/runtime` wrapper; test services live beside the code they
|
new syscalls extend [abi.zig](../../system/abi.zig) `SystemCall` + a
|
||||||
exercise and register a `ServiceId` if they must be looked up.
|
`library/kernel` wrapper; test services live beside the code they exercise and
|
||||||
|
register a `ServiceId` if they must be looked up.
|
||||||
|
|
||||||
## How to verify along the way
|
## How to verify along the way
|
||||||
|
|
||||||
|
|||||||
@@ -237,9 +237,9 @@ see the intro). Two scoped pieces, as built:
|
|||||||
|
|
||||||
### Build: multi-threaded codegen, opt-in
|
### Build: multi-threaded codegen, opt-in
|
||||||
|
|
||||||
A binary opts in with `addThreadedUserBinary` in the root `build.zig` (or
|
A binary opts in with `.threaded = true` in its package's
|
||||||
`.threaded = true` in a binary package's `build_support.userBinary` call) — the
|
`build_support.userBinary` call — the shared recipe in build-support then builds it
|
||||||
shared recipe in build-support then builds it `single_threaded = false` — so atomics
|
`single_threaded = false` — so atomics
|
||||||
and (later) TLS are real. Threads and atomics are unsound in a `single_threaded` image,
|
and (later) TLS are real. Threads and atomics are unsound in a `single_threaded` image,
|
||||||
so a binary must opt in **before** it may call `Thread.spawn`. Everyone else
|
so a binary must opt in **before** it may call `Thread.spawn`. Everyone else
|
||||||
stays single-threaded and lean.
|
stays single-threaded and lean.
|
||||||
|
|||||||
@@ -108,7 +108,7 @@ hypervisor configured for UEFI firmware and an xHCI USB controller.
|
|||||||
- **UEFI only.** A custom UEFI application loader is installed to
|
- **UEFI only.** A custom UEFI application loader is installed to
|
||||||
`\EFI\BOOT\BOOTX64.efi`. There is **no BIOS, multiboot, or limine** path. The
|
`\EFI\BOOT\BOOTX64.efi`. There is **no BIOS, multiboot, or limine** path. The
|
||||||
loader tolerates UEFI Class-3 machines with no legacy PIC/PIT.
|
loader tolerates UEFI Class-3 machines with no legacy PIC/PIT.
|
||||||
(`build.zig:246`, `boot/efi.zig`)
|
(`build/images.zig` — the EFI/BOOT install — and `boot/efi.zig`)
|
||||||
- **ACPI is the hardware-discovery mechanism.** The RSDP is taken from the UEFI
|
- **ACPI is the hardware-discovery mechanism.** The RSDP is taken from the UEFI
|
||||||
configuration table (ACPI 2.0 GUID preferred, 1.0 fallback). Without a valid
|
configuration table (ACPI 2.0 GUID preferred, 1.0 fallback). Without a valid
|
||||||
RSDP there is **no device discovery** — no SMP, no IOAPIC routing, no PCI/USB.
|
RSDP there is **no device discovery** — no SMP, no IOAPIC routing, no PCI/USB.
|
||||||
|
|||||||
+3
-1
@@ -12,7 +12,9 @@ There are two layers:
|
|||||||
`system/abi.zig`, `library/device/model/device-abi.zig`) now spans ~26 modules:
|
`system/abi.zig`, `library/device/model/device-abi.zig`) now spans ~26 modules:
|
||||||
protocol and on-wire definitions (VFS, USB, virtio-gpu), the FAT engine, the
|
protocol and on-wire definitions (VFS, USB, virtio-gpu), the FAT engine, the
|
||||||
display compositor, PS/2 and HID decoding, the kernel log ring, and the
|
display compositor, PS/2 and HID decoding, the kernel log ring, and the
|
||||||
runtime's `time`/`thread` — the full list is the test step in `build.zig`.
|
runtime's `time`/`thread` — the list is distributed across the library-domain
|
||||||
|
and binary packages' own `test` steps, which the root `zig build test`
|
||||||
|
aggregates (docs/build-packages-plan.md).
|
||||||
These compile for the host and run natively.
|
These compile for the host and run natively.
|
||||||
- **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel
|
- **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel
|
||||||
and check its behaviour. This is the interesting part.
|
and check its behaviour. This is the interesting part.
|
||||||
|
|||||||
Reference in New Issue
Block a user