build: lazy dependencies — a build loads only what it ships

The 13 /test fixtures and the acpi/fdt discovery pair are .lazy in the
root zon, resolved with b.lazyDependency only when a build actually
bundles them: a plain `zig build` neither compiles the fixtures nor
loads their build files, and only the -Ddiscovery-selected package ever
loads. Fixture packages are uniform (dependency name = artifact name =
boot-path leaf), so the bundled list shrinks to a name loop. Production
manifest byte-identical; the -Dtest-case manifest carries the same 13
entries in the same order; -Ddiscovery=fdt exercises the lazy fdt path.
Discharges the plan's deferred what-this-buys #4.
This commit is contained in:
Daniel Samson
2026-07-30 06:26:22 +01:00
parent d27670ec39
commit c621b649f6
3 changed files with 63 additions and 65 deletions
+39 -48
View File
@@ -221,11 +221,12 @@ pub fn build(b: *std.Build) void {
// image leaves it out). // image leaves it out).
const init_exe = b.dependency("init", .{ .serial = serial }).artifact("init"); const init_exe = b.dependency("init", .{ .serial = serial }).artifact("init");
// --- the rest of the boot tree: /system services and drivers, /test fixtures --- // --- the rest of the boot tree: /system services and drivers ---
// Each is built by the same user-binary recipe and laid out at its FHS path on // Each is built by the same user-binary recipe and laid out at its FHS path on
// the boot volume (see `bundled` below). The EFI loader walks the tree at boot // the boot volume (see `bundled` below). The EFI loader walks the tree at boot
// and hands the kernel an in-RAM initial_ramdisk of it (system/initial-ramdisk.zig). // and hands the kernel an in-RAM initial_ramdisk of it (system/initial-ramdisk.zig).
const vfstest_exe = b.dependency("vfs-test", .{}).artifact("vfs-test"); // (The /test fixtures are lazy dependencies, resolved further down only
// for a -Dtest-case build.)
// The drivers, each directory its own package: the PS/2 bus family (bus + // The drivers, each directory its own package: the PS/2 bus family (bus +
// keyboard + mouse from one package), the xHCI bus driver, the USB HID // keyboard + mouse from one package), the xHCI bus driver, the USB HID
// class drivers, and USB mass storage. Their unit tests ride along. // class drivers, and USB mass storage. Their unit tests ride along.
@@ -248,47 +249,31 @@ pub fn build(b: *std.Build) void {
const display_demo_exe = b.dependency("display-demo", .{}).artifact("display-demo"); const display_demo_exe = b.dependency("display-demo", .{}).artifact("display-demo");
const virtio_gpu_package = b.dependency("virtio-gpu", .{}); const virtio_gpu_package = b.dependency("virtio-gpu", .{});
const virtio_gpu_exe = virtio_gpu_package.artifact("virtio-gpu"); const virtio_gpu_exe = virtio_gpu_package.artifact("virtio-gpu");
const shared_memory_server_exe = b.dependency("shared-memory-server", .{}).artifact("shared-memory-server");
const shared_memory_client_exe = b.dependency("shared-memory-client", .{}).artifact("shared-memory-client");
const fat_test_exe = b.dependency("fat-test", .{}).artifact("fat-test");
// The first binary package (docs/build-packages-plan.md, phase 2): pci-bus // The first binary package (docs/build-packages-plan.md, phase 2): pci-bus
// builds itself against the domain packages; the root build just takes the // builds itself against the domain packages; the root build just takes the
// artifact for the boot image. // artifact for the boot image.
const pci_bus_exe = b.dependency("pci-bus", .{}).artifact("pci-bus"); const pci_bus_exe = b.dependency("pci-bus", .{}).artifact("pci-bus");
// crash-test is a fixture, not a real driver: it hellos to the device
// manager, then faults — what the driver-restart scenario drives the
// crash-loop cap with. pci-cap-test and iommu-fault-test exercise the
// driver-side PCI library surface and the VT-d rogue-DMA negative proof.
const crash_test_exe = b.dependency("crash-test", .{}).artifact("crash-test");
const device_list_exe = b.dependency("device-list", .{}).artifact("device-list");
const pci_cap_test_exe = b.dependency("pci-cap-test", .{}).artifact("pci-cap-test");
const iommu_fault_test_exe = b.dependency("iommu-fault-test", .{}).artifact("iommu-fault-test");
// The discovery service: one swappable process per firmware // The discovery service: one swappable process per firmware
// (docs/discovery.md), bundled under the neutral ramdisk name // (docs/discovery.md), bundled under the neutral ramdisk name
// "discovery" so the device manager never learns which firmware it is on. // "discovery" so the device manager never learns which firmware it is on.
// x86 boots describe hardware with ACPI; the Raspberry Pis hand over a // x86 boots describe hardware with ACPI; the Raspberry Pis hand over a
// flattened device tree — the aarch64 target flips the default when it // flattened device tree — the aarch64 target flips the default when it
// lands (docs/arm.md). Each firmware's service is its own package; both // lands (docs/arm.md). Each firmware's service is its own LAZY package;
// export an artifact named "discovery", and this option picks which one // both export an artifact named "discovery", and this option picks which
// ships (only the chosen one is compiled). // one ships — the unselected package's build file is never even loaded.
// (lazyDependency returns null only for an unfetched remote package; these
// are in-repo path dependencies, so a null means the directory is gone.)
const Discovery = enum { acpi, fdt }; const Discovery = enum { acpi, fdt };
const discovery = b.option(Discovery, "discovery", "Which discovery service fills the ramdisk's 'discovery' slot (default: acpi)") orelse Discovery.acpi; const discovery = b.option(Discovery, "discovery", "Which discovery service fills the ramdisk's 'discovery' slot (default: acpi)") orelse Discovery.acpi;
const discovery_exe = switch (discovery) { const discovery_exe = switch (discovery) {
.acpi => b.dependency("acpi", .{}).artifact("discovery"), .acpi => (b.lazyDependency("acpi", .{}) orelse @panic("system/services/acpi is missing")).artifact("discovery"),
.fdt => b.dependency("fdt", .{}).artifact("discovery"), .fdt => (b.lazyDependency("fdt", .{}) orelse @panic("system/services/fdt is missing")).artifact("discovery"),
}; };
const device_manager_exe = b.dependency("device-manager", .{}).artifact("device-manager"); const device_manager_exe = b.dependency("device-manager", .{}).artifact("device-manager");
// The input service and its exercisers: the fan-out server, a hardware-free synthetic // The input service and its exercisers: the fan-out server, a hardware-free synthetic
// source, and a subscriber that doubles as the `input` test's oracle. See docs/input.md. // source, and a subscriber that doubles as the `input` test's oracle. See docs/input.md.
const input_exe = b.dependency("input", .{}).artifact("input"); const input_exe = b.dependency("input", .{}).artifact("input");
const input_source_exe = b.dependency("input-source", .{}).artifact("input-source");
const input_test_exe = b.dependency("input-test", .{}).artifact("input-test");
const args_echo_exe = b.dependency("args-echo", .{}).artifact("args-echo");
const process_test_exe = b.dependency("process-test", .{}).artifact("process-test");
const logger_exe = b.dependency("logger", .{}).artifact("logger"); const logger_exe = b.dependency("logger", .{}).artifact("logger");
// The first multi-threaded binary: exercises runtime.Thread over the thread ABI
// (docs/threading.md). Its package opts into threading (real atomics/TLS).
const thread_test_exe = b.dependency("thread-test", .{}).artifact("thread-test");
// Every user binary and its FHS home on the boot volume. There is no packed // Every user binary and its FHS home on the boot volume. There is no packed
// ramdisk artifact any more: make-fat-image.py lays each binary out at this // ramdisk artifact any more: make-fat-image.py lays each binary out at this
@@ -327,32 +312,38 @@ pub fn build(b: *std.Build) void {
.{ .path = "system/drivers/virtio-gpu", .binary = virtio_gpu_exe.getEmittedBin() }, .{ .path = "system/drivers/virtio-gpu", .binary = virtio_gpu_exe.getEmittedBin() },
.{ .path = "system/drivers/pci-bus", .binary = pci_bus_exe.getEmittedBin() }, .{ .path = "system/drivers/pci-bus", .binary = pci_bus_exe.getEmittedBin() },
}; };
// The userspace test fixtures under /test. A plain `zig build` produces a clean
// image WITHOUT them; they are bundled only for a test build — which the QEMU
// harness signals by passing -Dtest-case=<name> for every scenario, exactly when
// these fixtures must be on the boot volume. Merely building this array never
// forces a compile: the fixture exes build only if `bundled` (below) includes them.
const test_bundled = [_]images.BundledBinary{
.{ .path = "test/system/services/vfs-test", .binary = vfstest_exe.getEmittedBin() },
.{ .path = "test/system/services/fat-test", .binary = fat_test_exe.getEmittedBin() },
.{ .path = "test/system/services/shared-memory-server", .binary = shared_memory_server_exe.getEmittedBin() },
.{ .path = "test/system/services/shared-memory-client", .binary = shared_memory_client_exe.getEmittedBin() },
.{ .path = "test/system/services/crash-test", .binary = crash_test_exe.getEmittedBin() },
.{ .path = "test/system/services/device-list", .binary = device_list_exe.getEmittedBin() },
.{ .path = "test/system/services/pci-cap-test", .binary = pci_cap_test_exe.getEmittedBin() },
.{ .path = "test/system/services/iommu-fault-test", .binary = iommu_fault_test_exe.getEmittedBin() },
.{ .path = "test/system/services/input-source", .binary = input_source_exe.getEmittedBin() },
.{ .path = "test/system/services/input-test", .binary = input_test_exe.getEmittedBin() },
.{ .path = "test/system/services/args-echo", .binary = args_echo_exe.getEmittedBin() },
.{ .path = "test/system/services/process-test", .binary = process_test_exe.getEmittedBin() },
.{ .path = "test/system/services/thread-test", .binary = thread_test_exe.getEmittedBin() },
};
// A no-option build assumes neither -Dtest-case nor -Ddiagnose: it ships the // A no-option build assumes neither -Dtest-case nor -Ddiagnose: it ships the
// production set only. Test fixtures join in only under -Dtest-case; the // production set only. The userspace test fixtures under /test join in only
// diagnose display-omission is already handled by init_csv_source above. // for a test build — which the QEMU harness signals by passing
// -Dtest-case=<name> for every scenario, exactly when they must be on the
// boot volume. They are LAZY dependencies: a plain build neither compiles
// them nor loads their build files (docs/build-packages-plan.md). Fixture
// packages are uniform — the dependency name, the artifact name, and the
// boot path's leaf all match the directory — so a name is a whole entry.
var bundled_list: std.ArrayListUnmanaged(images.BundledBinary) = .empty; var bundled_list: std.ArrayListUnmanaged(images.BundledBinary) = .empty;
bundled_list.appendSlice(b.allocator, &production_bundled) catch @panic("OOM"); bundled_list.appendSlice(b.allocator, &production_bundled) catch @panic("OOM");
if (test_case != null) bundled_list.appendSlice(b.allocator, &test_bundled) catch @panic("OOM"); if (test_case != null) for ([_][]const u8{
"vfs-test", // the user-space VFS round-trip client
"fat-test",
"shared-memory-server",
"shared-memory-client",
"crash-test", // hellos to the device manager, then faults — drives the crash-loop cap
"device-list",
"pci-cap-test", // exercises the driver-side PCI library against the pci-caps NIC
"iommu-fault-test", // fires the rogue DMA that VT-d must fault
"input-source",
"input-test",
"args-echo",
"process-test",
"thread-test", // the multi-threaded fixture (its package sets .threaded)
}) |fixture| {
const package = b.lazyDependency(fixture, .{}) orelse
@panic("a test fixture package is missing under test/system/services");
bundled_list.append(b.allocator, .{
.path = b.fmt("test/system/services/{s}", .{fixture}),
.binary = package.artifact(fixture).getEmittedBin(),
}) catch @panic("OOM");
};
const bundled = bundled_list.items; const bundled = bundled_list.items;
// Boot methods live in boot/, one per way of getting the kernel running. // Boot methods live in boot/, one per way of getting the kernel running.
+18 -15
View File
@@ -51,26 +51,29 @@
.@"device-manager" = .{ .path = "system/services/device-manager" }, .@"device-manager" = .{ .path = "system/services/device-manager" },
.input = .{ .path = "system/services/input" }, .input = .{ .path = "system/services/input" },
.logger = .{ .path = "system/services/logger" }, .logger = .{ .path = "system/services/logger" },
.acpi = .{ .path = "system/services/acpi" }, // The discovery pair and the /test fixtures are lazy: only what a
.fdt = .{ .path = "system/services/fdt" }, // given build actually ships gets its build file loaded and compiled
// (-Ddiscovery picks one of the pair; -Dtest-case pulls the fixtures).
.acpi = .{ .path = "system/services/acpi", .lazy = true },
.fdt = .{ .path = "system/services/fdt", .lazy = true },
.@"ps2-bus" = .{ .path = "system/drivers/ps2-bus" }, .@"ps2-bus" = .{ .path = "system/drivers/ps2-bus" },
.@"usb-xhci-bus" = .{ .path = "system/drivers/usb-xhci-bus" }, .@"usb-xhci-bus" = .{ .path = "system/drivers/usb-xhci-bus" },
.@"usb-hid" = .{ .path = "system/drivers/usb-hid" }, .@"usb-hid" = .{ .path = "system/drivers/usb-hid" },
.@"usb-storage" = .{ .path = "system/drivers/usb-storage" }, .@"usb-storage" = .{ .path = "system/drivers/usb-storage" },
.@"virtio-gpu" = .{ .path = "system/drivers/virtio-gpu" }, .@"virtio-gpu" = .{ .path = "system/drivers/virtio-gpu" },
.@"vfs-test" = .{ .path = "test/system/services/vfs-test" }, .@"vfs-test" = .{ .path = "test/system/services/vfs-test", .lazy = true },
.@"fat-test" = .{ .path = "test/system/services/fat-test" }, .@"fat-test" = .{ .path = "test/system/services/fat-test", .lazy = true },
.@"shared-memory-server" = .{ .path = "test/system/services/shared-memory-server" }, .@"shared-memory-server" = .{ .path = "test/system/services/shared-memory-server", .lazy = true },
.@"shared-memory-client" = .{ .path = "test/system/services/shared-memory-client" }, .@"shared-memory-client" = .{ .path = "test/system/services/shared-memory-client", .lazy = true },
.@"crash-test" = .{ .path = "test/system/services/crash-test" }, .@"crash-test" = .{ .path = "test/system/services/crash-test", .lazy = true },
.@"device-list" = .{ .path = "test/system/services/device-list" }, .@"device-list" = .{ .path = "test/system/services/device-list", .lazy = true },
.@"pci-cap-test" = .{ .path = "test/system/services/pci-cap-test" }, .@"pci-cap-test" = .{ .path = "test/system/services/pci-cap-test", .lazy = true },
.@"iommu-fault-test" = .{ .path = "test/system/services/iommu-fault-test" }, .@"iommu-fault-test" = .{ .path = "test/system/services/iommu-fault-test", .lazy = true },
.@"input-source" = .{ .path = "test/system/services/input-source" }, .@"input-source" = .{ .path = "test/system/services/input-source", .lazy = true },
.@"input-test" = .{ .path = "test/system/services/input-test" }, .@"input-test" = .{ .path = "test/system/services/input-test", .lazy = true },
.@"args-echo" = .{ .path = "test/system/services/args-echo" }, .@"args-echo" = .{ .path = "test/system/services/args-echo", .lazy = true },
.@"process-test" = .{ .path = "test/system/services/process-test" }, .@"process-test" = .{ .path = "test/system/services/process-test", .lazy = true },
.@"thread-test" = .{ .path = "test/system/services/thread-test" }, .@"thread-test" = .{ .path = "test/system/services/thread-test", .lazy = true },
// See `zig fetch --save <url>` for a command-line interface for adding dependencies. // See `zig fetch --save <url>` for a command-line interface for adding dependencies.
//.example = .{ //.example = .{
// // When updating this field to a new URL, be sure to delete the corresponding // // When updating this field to a new URL, be sure to delete the corresponding
+6 -2
View File
@@ -9,7 +9,9 @@ named "discovery" that the root's -Ddiscovery picks between), and phase 3 (the
root split into `build/images.zig` + `build/qemu.zig`; the root `build.zig` is root split into `build/images.zig` + `build/qemu.zig`; the root `build.zig` is
~460 lines of orchestration, down from ~1,250). Every phase landed green: unit ~460 lines of orchestration, down from ~1,250). Every phase landed green: unit
tests, the QEMU suite at parity with main, boot-image file list unchanged. tests, the QEMU suite at parity with main, boot-image file list unchanged.
Still future: `lazyDependency` for image-specific builds (What-this-buys #4). The `lazyDependency` payoff (What-this-buys #4) is in too: the /test fixtures
and the unselected discovery package are lazy — a build loads and compiles
only what it ships.
## Why ## Why
@@ -76,7 +78,9 @@ Rules:
stability testing in isolation. stability testing in isolation.
3. Adding a binary = adding a directory (source + two small files), not editing 3. Adding a binary = adding a directory (source + two small files), not editing
three places in a 1,250-line file. three places in a 1,250-line file.
4. Later: `lazyDependency` lets an image target build only what it ships. 4. `lazyDependency` lets an image target build only what it ships: the /test
fixtures resolve only under -Dtest-case, and only the -Ddiscovery-selected
discovery package ever loads.
## Phases ## Phases