Three services had each written the same thing and got it three different
ways: input polled the process list to notice a dead subscriber, and only
when someone else subscribed; the power service never noticed at all; the
device manager noticed drivers but not subscribers. The harness owns the
table now, driven by the events a protocol declares — it registers on the
reserved verb, frames each event once, posts to everyone interested without
waiting on any of them, and reclaims a slot when the kernel says its owner
died. Interest masks moved to the envelope, so a subscriber that wants only
mice asks the same way everywhere.
Two consequences the plan had not foreseen. The device manager now hears a
supervised child's death twice, once as its supervisor and once as a
subscriber, so restart backoff counted every crash twice and gave up after
half as many; it retires the id before counting. And the kernel's published
exit table had eight slots for what is now six subscriptions in a plain
boot, so it holds sixteen.
The other half is a hole the design named early and left standing: a
backend handed out a small integer and then honoured it from anyone. A
process that guessed a file's node id read another client's file; a display
layer had no owner at all, so any client could reconfigure or destroy any
layer; a USB device token was never checked against the client that opened
it. Each is now bound to the task that opened it, and a wrong owner gets
exactly what an unknown id gets — the refusal must not become the oracle
the identical answers elsewhere were designed to remove. Closing a file
changed with it: it used to succeed unconditionally, which would have told
a caller which ids existed.
Suite 111/111, with a new case in which one process holds a file and a
layer, hands both ids to a second process, and finds them untouched after
that process has tried everything with them.
The folded header stops being a rule in a document and becomes the layout
on the wire. Verbs number from sixteen, leaving describe, enumerate,
subscribe and unsubscribe reserved and answered the same way by every
provider — none of them writes a line to do it. What each protocol used to
carry in a field of its own now travels in the header: a vfs node and a
display layer are the packet's target, and a reply opens with a status the
envelope stamps rather than one each protocol spelled for itself.
Display gains the most. One forty-byte request had served eleven verbs, so
attach_scanout smuggled stride through x, refresh through y and format
through colour, and every coordinate crossed as a bitcast. Per-operation
structs end all three: the fields have their own names and their own signs,
and the tile payload grows to 224 bytes because the prefix shrank. Scanout
loses a message maximum of 64 it had no business declaring — it answers
calls, and the floor for a call is 256 — and virtio-gpu stops hard-coding
that number at its harness.
Two changes are semantic rather than notational. A directory now ends at an
entry with no name, because the fixed part of a reply always travels and a
zero-length reply no longer exists to mean anything. And input joins the
service harness, the last loop in the tree that answered no ping and heard
no terminate; its subscriber table, its pruning and its fan-out are the
same code, and a shutdown now asks it to stop instead of killing it.
A new conformance case reads the registry's own listing and asks every
protocol it finds for its name, its version and its verb count, then offers
a verb nobody defines and requires -ENOSYS — the envelope's promise,
checked against providers rather than against itself. What it cannot reach
in that boot it names on the serial line instead of passing quietly.
Suite 110/110.
The registry consults the open rows it has been parsing since P2, so
reaching a contract now takes a grant as well as a binding. A caller
without one is answered exactly as it would be for a name nobody ever
bound: same status, same empty reply, same absent capability, byte for
byte, and no log line on either path — klog_read is ungated, so a line on
one and not the other would be the oracle the design set out to remove.
Refusal and absence being one answer is what lets a supervisor later
narrow, fake or park a child's namespace without the child learning what
it was denied.
The manifest gains a third permission for a shape the plan did not
foresee: attestation is one hop, but the driver tree is three deep — the
PS/2 keyboard and mouse are spawned by ps2-bus, which the device manager
spawned — so no row could name them and PS/2 input would simply stop.
A supervise grant lets a delegate vouch for what its children *reach*,
never for what they claim; the bind path is untouched, and the laundering
deputy is still refused.
The review found the receive side of a rule this track had already
written down. Every process holds a sendable handle to the registrar —
resolve installs one for anyone who asks — and ipc_reply_wait never asked
who owned the endpoint, so a stranger could dequeue there: take the
provider endpoints riding bind requests, and answer other clients' opens
in the registrar's name. Receiving is the owner's privilege, like binding
a signal or a timer; sending remains anyone's.
Suite 109/109.
A protocol is reached by name now, not by a compile-time integer. Init is
PID 1 and already knows which binary it started, so init serves /protocol
as a vfs backend: bind claims a contract with the provider's endpoint
attached, open answers with that endpoint as the reply's capability, and
readdir lists what is bound with the task and binary behind it. The kernel
reserves the prefix — nothing may mount over it, under it, or unmount it —
and ServiceId, ipc_register and ipc_lookup are gone, their syscall numbers
left vacant.
A bind is authorized by who the caller *is*: the kernel-stamped binary
together with the supervising task's identity, matched against
/system/configuration/protocol.csv. Identity, not spelling — spawn is
ungated, so an attacker can run any bundled binary, and a name-only rule
would have let it launder grants through an init of its own making. A name
a live process holds is refused to everyone else; a dead one's is released.
Three review rounds against a hostile ring-3 process found what 108 green
tests could not, because the suite contains no attacker. Publishing init's
supervision endpoint as the registry put PID 1's mailbox in every process's
hands, where two forged bytes reached the shutdown path: privileged traffic
is now believed only from the task that holds the contract it speaks for.
A capability arriving on a request outlived every path that ignored it,
one handle per call until the table was full — in init, and in the harness
ten services share — so the arriving capability is owned by the turn and
released unless a handler says otherwise. And the kernel let anyone holding
an endpoint handle aim signals, timers, exit notices and interrupts at it:
binding now requires having created it.
Suite 108/108. The new protocol-registry case asserts eleven properties,
each one an attack that must fail.
A new user-memory module owns every kernel touch of a user buffer:
copyFromUser, the new copyToUser, and the resolve behind both. The walk
accumulates the U/S and writable bits down all four levels with the MMU's
own AND rule — folding a 2 MiB leaf in before it resolves and refusing a
1 GiB leaf outright — so a copy honours what ring 3 itself would be
allowed, closing the presence-only trust model the IPC layer carried since
bring-up. It then confirms the frame is physmap-backed, because that is how
the copy reaches it: an mmio_map'd BAR passes the permission walk and would
otherwise fault ring 0 on an alias the physmap never mapped, on the IPC path
as much as the new one.
The nine stragglers that dereferenced user pointers raw now route through
it, so a bad pointer returns -EFAULT where it used to fault the kernel.
The write direction restructures its callees around kernel bounce buffers:
scheduler and devices-broker enumerate from a slot cursor (a task exiting
between chunks can neither duplicate nor lose an entry), klog_read drains
the ring in chunks, and fs_node stages headers and names contiguously.
fs_resolve copies out before installing the endpoint handle, so a faulting
copy cannot strand a capability; its out-capacity bound no longer adds an
unbounded ring-3 length to the base, which wrapped and trapped the kernel's
own overflow check. debug_write reads the caller's message once.
Suite 107/107 (new user-memory case: seven bad pointers refused, each
paired with a sound call that must still succeed).
/etc/init.csv and /etc/devices.csv become /system/configuration/*.csv (the
repo's etc/ moves to system/configuration/, mirroring the runtime tree),
/var/log becomes /system/logs, and /mnt/usb becomes /volumes/usb. The
kernel VFS gains a carve-out so FAT may serve exactly /system/configuration
and /system/logs beneath the initrd-backed /system while /system and /test
themselves stay unshadowable; FAT's single /var mount splits into those two
rewritten mounts. The kvfs readdir check learns /system's third child and
the ramdisk spawn sweep skips the configuration tree.
Suite 106/106.
Each shipped binary was registered twice in the root build (a
dependency/artifact line plus a bundled row repeating its name and
path). The uniform rows — dependency name = artifact name = boot-path
leaf — collapse into production_ship, one line per binary via the
service()/driver()/driverArtifact() helpers; only the genuinely
non-uniform entries stay spelled out (init's -Dserial, the -Ddiscovery
pick, the /etc data files). Selecting what goes into a build is now
selecting table rows, and an unselected package's build file is never
loaded. The boot manifest is set-identical; its order shifts (discovery
and the /etc entries move) — every lookup is by name, and the
boot-order QEMU smoke passes.
The 13 /test fixtures and the acpi/fdt discovery pair are .lazy in the
root zon, resolved with b.lazyDependency only when a build actually
bundles them: a plain `zig build` neither compiles the fixtures nor
loads their build files, and only the -Ddiscovery-selected package ever
loads. Fixture packages are uniform (dependency name = artifact name =
boot-path leaf), so the bundled list shrinks to a name loop. Production
manifest byte-identical; the -Dtest-case manifest carries the same 13
entries in the same order; -Ddiscovery=fdt exercises the lazy fdt path.
Discharges the plan's deferred what-this-buys #4.
build/images.zig owns everything between built binaries and a bootable
volume: the FHS zig-out install tree, boot manifest + capsule, both
FAT32 images, the release ISO, and their check steps. build/qemu.zig
owns run-x86-64 / run-x86-64-gpu and the OVMF probing. The root
build.zig (461 lines, from 1,242 pre-split) now only decides what
ships: kernel + loader, the package list, the bundled boot tree, and
the aggregate test step. The stale commented-out run scaffold is gone.
Boot-image file list unchanged; check-fat-image and check-iso-image
both green.
All thirteen /test/system/services fixtures convert on the pci-bus
template (thread-test threaded; crash-test/device-list/pci-cap-test/
iommu-fault-test carrying their protocol and PCI extras). With the last
addUserBinary caller gone, the root build's wrapper functions and its
default-imports plumbing are deleted — every user binary now reaches
the shared recipe only through its own package. Production boot-image
file list unchanged; a -Dtest-case build bundles the fixtures exactly
as before.
ps2-bus (bus + keyboard + mouse artifacts from one package),
usb-xhci-bus, usb-hid (keyboard + mouse), usb-storage, and virtio-gpu
convert on the pci-bus template. Their unit tests — PS/2 decode, HID
boot reports, Bulk-Only Transport/SCSI encodings, and the virtio-gpu
size checks the plan flagged as a wave carry-along — move into their
packages; the root aggregate delegates. Boot-image file list
unchanged.
init, fat, display, display-demo, device-manager, input, logger, and
the two discovery fillers (acpi, fdt — each exporting an artifact named
"discovery"; the root -Ddiscovery picks which ships) convert to binary
packages on the pci-bus template. init's serial heartbeat flag rides
the dependency options (the root forwards its -Dserial). fat's and
display's unit tests move into their packages and the root aggregate
delegates to them. Boot-image file list unchanged.
Review findings from the pilot: the 17-module default import set was
maintained twice (root's array and build-support's userBinary) — drift
would silently make packaged binaries differ from root-built ones. It
now lives once, as build-support.defaultImports; root and userBinary
both draw from it, and root's user-binary wrapper names the shim +
link script through the kernel package like build-support does.
The root aggregate test step likewise duplicated the library domains'
14 test definitions; it now depends on each domain's own standalone
test step (host-only, so root's -Dtarget/-Doptimize deliberately do
not reach them), and the client domain gains an empty test step for
uniformity. Boot-image file list unchanged.
build-support gains the domains-based userBinary: the default import set
(the library/kernel concern modules, driver/service clients, mmio,
acpi-ids, xkeyboard-config) is assembled from the domain packages, and
the root shim + user link script come from the kernel package directory.
system/drivers/pci-bus is the first binary package: a ~15-line
declarative build.zig naming only its extras (device-manager-protocol,
pci-class); the root build consumes the artifact for the boot image and
the driver also builds standalone from its own directory. Boot-image
file list unchanged.
Each library domain (kernel, device, client, protocol, csv,
xkeyboard-config) now owns a build.zig + build.zig.zon that wires and
exports its modules, with a standalone `zig build test` per domain.
The kernel package also exports abi (source stays in system/abi.zig) so
every consumer names one module instance. The root build swaps its
createModule calls for b.dependency(...).module(...) — no binary moves;
the root is the pilot consumer (docs/build-packages-plan.md). Path
dependencies deduplicate by resolved location, so the diamond
(root -> device -> kernel, root -> kernel) yields a single instance of
each module. Boot-image file list unchanged.
The shared user-binary recipe (freestanding target, root-shim wiring,
link-script and image-base settings) moves out of the root build into
build-support/, the package that is the single home for cross-cutting
build changes (docs/build-packages-plan.md). The root build's
addUserBinary/addThreadedUserBinary keep their signatures and delegate;
nothing else moves. Boot-image file list unchanged.
Replaces L1's shared blanket identity domain with a private translation
domain per claimed PCI function. A device now reaches only:
- the DMA pool: every dma_alloc'd region, mapped into every claimed
device's domain (poolAdd/poolRemove, driven from the dma_alloc and
dma_free syscalls). This keeps the cross-process buffer handoff
working (fat's bounce buffer reaches the xHC) while blocking the
kernel, page tables, process heaps, MMIO, and unallocated RAM.
- its own firmware reserved region (RMRR), seeded at confine time.
The pool is the honest interim: devices can still reach one another's
DMA buffers. The DMA-region capability layer (next) narrows it to
per-grant reachability.
dma_free unmaps from every domain and invalidates BEFORE the frames
return to the allocator, closing the stale-IOTLB use-after-free window.
Driver death tears down its domains (detach + free tables) before the
broker claims and DMA frames are released.
New iommu_fault_drain syscall (+ driver.iommuFaultDrain) forces pending
fault records to the log on demand. The new iommu-fault case proves it:
a claimed e1000e is programmed to DMA-fetch its TX ring from an unmapped
page; VT-d faults the access (bdf 00:03.0 addr 0x1000 reason 0x6) and the
system stays alive. 104/104.
library/device/pci is now the complete generic floor a leaf PCI driver
needs, instead of just what virtio-gpu used:
- pci-class: capability IDs, MSI/MSI-X/power-management/PCI-Express
register layouts, extended-capability header decode (host-tested),
per-bit command constants, remaining header offsets.
- pci.Function: header accessors, disableBusMaster + interrupt-disable
helpers, findCapability, programMsi/disableMsi, MsiX vector-table
struct, ensurePowerStateD0, functionLevelReset (BAR save/restore),
extended-capability iterator.
Proven by the new pci-caps QEMU case: a pci-cap-test fixture claims an
extra e1000e (PM+MSI+PCIe+MSI-X, no danos driver) and readback-verifies
every surface, including the first driver-side use of msi_bind.
usb-xhci-bus converts from 8 ms event-ring polling to message-signalled
interrupts: plain MSI where offered (real Intel xHC), MSI-X entry 0
otherwise (qemu-xhci has no MSI capability), byte-identical polling as
fallback. The timer survives as a 250 ms port-reconcile/lost-edge tick —
real-hardware USB2 hub debounce still needs it. MSI setup runs BEFORE
controller bring-up: QEMU's xhci only registers the MSI-X vector as used
when IMAN.IE is written while MSI-X is already enabled; interrupts are
silently dropped otherwise (real hardware does not care about the order).
101/101 QEMU cases green; real-hardware smoke passed (mouse works,
boot 2026-07-23T174805Z, plain-MSI branch, vector 33).
A plain `zig build` was baking all 11 /test/system/services fixtures into the
production image. Split the bundle: the production set always, the userspace test
fixtures only when -Dtest-case is set — which the QEMU harness passes for every
scenario, exactly when the fixtures are needed on the boot volume. So `zig build`
with no options now ships a clean image (no test binaries in the manifest,
capsule, or FAT tree) and assumes neither -Dtest-case nor -Ddiagnose (the
diagnose display-omission was already gated by the init.csv selection).
Verified: a plain build has zero /test entries in the manifest; -Dtest-case=smoke
includes all fixtures; the args/vfs/device-list QEMU scenarios still pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Replace init's hardcoded boot_services array with an authoritative,
human-readable service list read at boot, mirroring /etc/devices.csv. Each row
is a service binary path followed by its argv; startup order is file order,
shutdown the reverse. There is no hardcoded fallback — a missing file starts
nothing (the no-ramdisk isolation behavior).
- library/csv: shared CSV helpers (comment strip, field iteration) with unit
tests; device-registry is refactored onto them so both /etc/*.csv files parse
through one place.
- init reads /etc/init.csv into fixed-max static tables (the same pattern as the
device registry) and passes each row's argv straight to spawnSupervised. This
also makes boot-time modes (e.g. device-manager test-usb-restart) expressible
as data rather than hardcoded.
- Diagnose mode (-Ddiagnose omits the display stack) becomes build-time file
selection between etc/init.csv and etc/init-diagnose.csv, so init carries no
comptime service logic; the diagnose build option is dropped from init.
- build.zig: csv module wired; /etc/init.csv bundled into the initrd; the
device-registry tests move to a dedicated block since they now import csv.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Replace the three hardcoded switch tables (pciDriverForIdentity, hidDriverFor,
usbDriverForIdentity) with an authoritative, human-readable device registry the
manager reads at boot. Matching is most-specific-wins across
base/subclass/prog_if/vendor/device/subsystem/hid, so a precise vendor:device
rule and a generic class rule coexist; an unmatched device is logged, never
guessed. This resolves docs' "matching stays code until the third bus".
- ABI: child_added and DeviceDescriptor gain vendor/device/subsystem; child_added
gains a bus discriminator (BusKind) so PCI and USB class triples match against
the right namespace.
- pci-bus reads vendor/device (config 0x00) and subsystem (0x2C, type-0) and
reports them.
- library/device/registry: freestanding CSV parser + matchDriver() with
specificity scoring; 5 unit tests wired into `zig build test`.
- etc/devices.csv bundled into the initrd; the kernel serves /etc directly, so
the manager reads it before any filesystem service is up (fat starts later).
- virtio-gpu: drop the now-redundant post-spawn 1AF4:1050 re-confirm, since the
registry binds this driver by exact identity.
- Remove the orphaned system/drivers/display driver (unreferenced by build or
registry).
- docs: new devices-csv.md; device-manager.md "matching stays code" resolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
ReleaseSafe kept full DWARF/symtab because nothing set strip on the
kernel module, bloating the flashable image to 2.4MB (~1.4MB of which
was debug info). Strip for every optimize mode except Debug, so
release builds shrink to ~900KB while debug builds keep symbols.
The 11 QEMU-suite fixtures lived mixed into system/services/ with their
binaries bundled at /system/tests/<name>. Now the repo path is the boot
path, like every real service: test/system/services/<name>. fat-test
moves out of the fat server's directory into its own; display-demo
stays a boot service.
- kernel VFS: setInitialRamdisk derives one read-only initrd mount per
top-level tree named by the ramdisk entry paths (/system, /test),
registers ancestors generically with self-parented roots, and refuses
backend shadowing of any initrd tree
- EFI loader: the fallback walk also enumerates \test (optional — a
volume without fixtures still boots); manifest and capsule unchanged
- path literals: vfs-test self-open + create probe, process-test
process_enumerate matches, the args-echo argv[0] expectation; the
kvfs case now covers the /test root end to end
- docs: DFHS /test rows, tree diagrams, loader prose, and the location
convention gain the third home; fixed the input-source link
100/100 QEMU cases pass.
With every consumer migrated (C2), nothing imports `runtime` or `system` any
longer. Removed both shim files and their build.zig module definitions, dropped
the `runtime` entry from `default_imports`, and rewrote the per-binary root shim
(library/kernel/root.zig) to import `start` (_start + panic) and `logging`
(std_options) directly instead of through `runtime`.
addUserBinary/addThreadedUserBinary/addUserBinaryImpl lose their `runtime_module`
parameter; the root module now pulls the two modules it needs out of
`default_imports` via a small findImport helper.
The runtime dumping ground is gone: the userspace library is library/kernel
(concern modules), library/device (driver/pci/usb/block/model/mmio/acpi data),
library/client (display, input service clients), and library/protocol (wire
contracts). Consumers @import concern modules by name.
Verified: zig build, zig build test, and QEMU (smoke, thread-spawn, thread-mutex,
logger, display-native, device-manager, usb-storage, input).
Every user binary and the two device-logic library modules (pci, usb) now
`@import` the concern modules directly instead of aliasing through `runtime`:
runtime.ipc/process/time/service/input/block/display -> @import("<module>")
runtime.device / runtime.device_manager -> @import("driver")
runtime.fs -> @import("file-system")
runtime.Thread -> @import("thread").Thread
runtime.system.{write,writeRecord,klog*} -> logging.*
runtime.system.{sleep,timerOnce,wallClock,clock} -> time.*
runtime.system.{spawn*,kill,exit,yield,processes,...}-> process.*
runtime.system.{mmap,munmap,PROT_*} -> memory.*
runtime.dma.* / runtime.shared_memory.* / runtime.allocator -> memory.*
Each consumer keeps its own alias name (e.g. `const device = @import("driver")`),
so call sites are unchanged and there are no collisions with local `driver`
variables. build.zig now injects the concern modules into every user binary via
`default_imports`; pci/usb module import lists were updated to match.
The `runtime` and `system` shims remain for one more step (root.zig still uses
runtime); they are deleted in C5. Nothing but root.zig imports `runtime` now.
Verified: zig build, zig build test, and 17 QEMU cases (smoke, device-manager,
logger, fat-mount, fat-mutations, usb-storage, usb-hid, display-native,
virtio-gpu, input, thread-spawn, thread-mutex, process-kill, shared-memory,
driver-restart, acpi-ps2, pci-scan).
The one giant `runtime` module (with a `system.zig` that was itself a dumping
ground of unrelated syscalls) is split into directly-importable, flat concern
modules under library/kernel/:
system-call ipc memory process thread time logging file-system service start
(+ the device/service clients: device, device-manager, block, display, input)
system.zig is dissolved — its functions moved to their concern home (mmap ->
memory, spawn/kill/exit -> process, sleep/clock -> time, write/klog -> logging,
fs* -> file-system). `memory` merges heap+dma+shared-memory behind one flat API
(memory.allocator/dmaAlloc/sharedCreate/mmap), keeping heap's state and malloc
export single. The memory<->thread dependency cycle (heap needs Thread.Mutex,
thread needs mmap) is broken by having thread allocate its own stack via the raw
mmap syscall, so the module graph is a DAG.
This is the atomic step: all 42 internal cross-imports flip from relative to
module imports at once. `runtime.zig` and `system.zig` become thin re-export
SHIMS so the ~38 consumers keep compiling on `runtime.*` untouched; they migrate
to direct imports in C2, after which the shims are deleted (C5).
zig build + zig build test green; 14 QEMU cases pass (smoke, process,
process-kill, thread-spawn/join, logger, vfs, fat-mount, display-native,
usb-storage, virtio-gpu, device-manager, input, power-button).
First step of splitting the runtime dumping ground. Pure directory rename
(git mv library/runtime library/kernel) + the six build.zig path references
repointed. The module is still named "runtime" for now; the next commits split
it into concern modules (ipc, memory, process, time, logging, file-system, ...),
dissolve system.zig, and delete the runtime aggregator.
The library/kernel name follows the kernel32 model: it is the userspace library
that wraps the private kernel ABI, distinct from system/kernel/ (the kernel).
zig build green.
Follow-up cleanup of the just-moved kernel device code:
- power.zig -> acpi.zig. Its reboot() is built entirely on the FADT reset
register (acpi.power_information) plus the legacy 0xCF9/8042 fallbacks — it is
ACPI reboot, so it becomes acpi.reboot (the "P" in ACPI). platform.reboot
still delegates; soft-off/S5 stays the ring-3 acpi service's job as before.
- device-tree.zig -> fdt.zig. It is a discovery *backend* (the ARM/FDT parser,
a sibling of acpi.zig), not part of the model — renaming it to its actual
subject kills the confusing device-tree / device-model DeviceTree name clash
and makes acpi.zig + fdt.zig read as the two parallel backends.
- Flatten: device-model.zig, acpi.zig, fdt.zig, platform.zig move out of the
system/kernel/devices/ subdir up into system/kernel/, joining devices-broker.zig
(already flat). The kernel dir is a flat pile by convention (only architecture/
is a subdir), so the subdir — and its poor "devices" name — is gone.
Pure restructure; "platform" module name unchanged, all cross-file deps are
relative siblings that moved together. zig build + test green; smoke, discovery,
acpi-parse, acpi-ps2, acpi-report, power-button, orderly-shutdown, reboot pass.
With the shared device data (device-abi, pci-class, usb-abi, usb-ids, acpi-ids,
aml) now in library/device/, what remained in system/devices/ was purely
kernel-internal: the firmware-discovery machinery and the rich pointer-based
device model (platform, device-model, acpi, device-tree, power), reached only
through the "platform" module by three kernel files. It belongs with the kernel.
Move it to system/kernel/devices/. A pure relocation: the "platform" module
name is unchanged and every cross-dir dependency is a module import, so only the
one b.path and some comments move. The top-level split is now clean —
system/kernel/ is the kernel, library/device/ the shared device libraries,
system/{drivers,services} the userspace. The runtime /system/devices concept
(the virtual device tree) is unaffected; system/kernel/devices/ is its
implementation.
Also fixes two comment refs that still pointed device-abi at its pre-Wave-1a
home (system/devices/) — it lives at library/device/model/ now.
zig build + test green; smoke, discovery, acpi-parse, acpi-ps2, acpi-report pass.
mmio is device-driver code, so it joins the other domains under
library/device/mmio/ (module name "mmio" unchanged — a pure relocation, only
the build paths move). And its abbreviated function names are spelled out per
docs/coding-standards.md:
read -> readRegister mb -> memoryBarrier
write -> writeRegister rmb -> readMemoryBarrier
wmb -> writeMemoryBarrier
All call sites updated (virtio-gpu, usb-xhci-library, pci.Function); the two
display-driver placeholders import mmio but use nothing, so they're untouched.
Docs (driver-model graph, README layout, drivers.md, the FHS note) follow the
new path and names.
zig build + test green; virtio-gpu, display-native, display-reattach, usb-hid,
usb-hub, usb-storage, pci-scan pass.
power-protocol -> library/protocol/power/power-protocol.zig. Its two consumers
(init and the acpi discovery service) import it directly; the
runtime.power_protocol re-export and the runtime module import are dropped (no
runtime client speaks it). The now-empty system/services/power/ is removed.
This completes library/protocol/: every driver<->service wire contract lives
there and is imported by module name, no protocol is re-exported through
runtime, and the alias chaos (dp/sp, mixed protocol/<x>_protocol) is resolved.
virtio-gpu-protocol stays a driver-private relative import (a hardware command
set, not a driver<->service seam — like virtio-pci.zig beside it).
zig build + test green; power-button, orderly-shutdown pass.
display-protocol -> library/protocol/display/, scanout-protocol ->
library/protocol/scanout/. Consumers import both directly, killing the cryptic
dp/sp aliases in virtio-gpu (now display_protocol / scanout_protocol) and the
runtime.display_protocol / runtime.scanout_protocol re-exports. runtime.display
(the client) still imports display-protocol by name; scanout has no runtime
client, so its runtime module import is dropped too.
zig build + test green; display-service, virtio-gpu, display-native,
display-reattach pass.
device-manager-protocol -> library/protocol/device-manager/. Its consumers
(pci-bus, usb-xhci-bus, the acpi discovery service, device-manager, device-list,
crash-test, and the not-yet-built intel-integrated display sub-driver) import the
module directly instead of through runtime.device_manager_protocol, which is
deleted. runtime.device_manager (the hello client) already imported the module
by name and is unchanged.
zig build + test green; device-manager, driver-restart, device-list, pci-scan pass.
input-protocol -> library/protocol/input/input-protocol.zig. Its five
consumers (usb-hid keyboard/mouse, ps2 keyboard/mouse, the input service) now
import the module directly and alias it input_protocol, replacing the
runtime.input_protocol re-export (deleted) and the inconsistent local `protocol`
aliases. runtime.input (the client) still imports the module by name.
zig build + test green; input, acpi-ps2, usb-hid pass.
vfs-protocol -> library/protocol/vfs/vfs-protocol.zig. fat, its only consumer,
now imports the module directly (const vfs_protocol = @import("vfs-protocol"))
instead of through runtime.vfs_protocol, and the runtime re-export is deleted.
The runtime's own client (runtime.fs) still imports the module by name.
zig build + test green; vfs, fat-mount pass.
Start the protocol tree: wire protocols move to library/protocol/<name>/,
entry file <name>-protocol.zig, module name unchanged. block and usb-transfer
are pure moves — every consumer already imports them by module name, so only
the b.addModule paths change and the empty system/services/block/ is removed.
block-protocol -> library/protocol/block/block-protocol.zig
usb-transfer-protocol -> library/protocol/usb-transfer/usb-transfer-protocol.zig
zig build + test green.
The USB class-driver transfer client was library/runtime/usb.zig, re-exported
as runtime.usb — which compiled the USB client and usb-transfer-protocol into
every user binary (init, fat, the compositor…), none of which speak USB. It is
bus-family logic, not core runtime.
Move it to its domain home, library/device/usb/usb.zig (module "usb"), alongside
the usb-abi and usb-ids data modules; fix its internal imports to go through the
runtime module; and re-export usb.abi / usb.ids so a class driver reaches the
whole USB domain through one import. runtime.usb and runtime's
usb-transfer-protocol import are removed; the three class drivers
(usb-hid keyboard/mouse, usb-storage) import module "usb" directly.
zig build + test green; usb-hid, usb-storage pass.
New pci logic module (library/device/pci/pci.zig): a device driver's view of
the one PCI function it has claimed — Function.map (config space = resource 0),
vendorId/deviceId/command/status, enableMemoryAndBusMaster, mapBar (BAR decode
+ resource correlation + mmio_map, cached), and a capabilities() iterator. The
generic PCI mechanics every leaf PCI driver used to re-derive inline.
The config-space layout it needs — header offsets, the command MEM|bus-master
bits, the status capabilities-list bit, the capability-pointer mask, and the
BAR bit fields — is named in the pci-class data module (a "Configuration-space
layout" section), so the bus enumerator can share the same constants later.
virtio-gpu is the first consumer: its inline mapBar + walkCapabilities + config
header reads are gone, replaced by pci.Function; only the virtio-specific
cfg_type dispatch (and the virtio common-config cfgRead/cfgWrite, which are NOT
PCI config space) stay in the driver. The generic display driver will use the
same module. pci-bus's enumerator (arbitrary-function probing) is untouched.
zig build + test green; virtio-gpu, display-native, display-reattach, pci-scan pass.
The kernel's only dependency on pci-class and acpi-ids was cosmetic: the
boot-time device-tree dump (DeviceTree.dump) decoded class codes and _HID
strings to human names. That is device decoding — a user-space concern in a
microkernel, and the last thing pinning two userspace taxonomies into the
kernel's compile.
dumpNode now prints raw values (the class tag, the raw _HID string, and the
packed PCI class code as hex); a user-space tool that owns the taxonomies
(the device manager already imports them) can pretty-print when wanted. The
pci-class / acpi-ids imports are dropped from device-model.zig and from the
platform module.
The kernel's sole remaining library/device/ import is now device-abi (pure
data — the device-model types the broker marshals across the syscall
boundary), the one intentional kernel->library crossing.
zig build + test green; smoke, discovery, acpi-parse, acpi-ps2 pass.
First step of the device-code reorganization (plan: group device code by
domain, split each domain into a shareable data module + a logic module).
This moves the pure-data modules — the enums, wire types, and taxonomies
that any layer including the kernel can import cheaply — out of
system/devices/ and into their domain home:
device-abi -> library/device/model/device-abi.zig
pci-class -> library/device/pci/pci-class.zig
usb-abi -> library/device/usb/usb-abi.zig
usb-ids -> library/device/usb/usb-ids.zig
acpi-ids -> library/device/acpi/acpi-ids.zig
aml/ -> library/device/acpi/aml/
Module names are unchanged, so this is a pure file move: only the
b.addModule paths and the host-test file list in build.zig change; no
importer is touched. system/devices/ now holds only the kernel-internal
device model (device-model, platform, acpi, device-tree, power).
The device *logic* (the pci Function helper, the usb transfer client) and
the kernel's cosmetic taxonomy dependency are handled in following commits.
zig build + zig build test green.
Every doc verified claim-by-claim against the code by parallel audit agents,
then fixed and adversarially re-verified. Two waves of staleness corrected:
the originally audited findings (higher-half boot handoff, kernel VFS
takeover, fault isolation + claim release + driver restart, AML/S5 moving to
ring 3, threading's shipped design, USB+FAT landing) and a second pass of
adjacent claims the verifiers caught (smp.md 'not built yet' intro,
system-requirements' PS/2-only and no-storage claims, halting.md's red-panic
and no-IDT text, testing.md's serial mirroring, router-era vfs-protocol
wording, capsule-first boot loading).
threading.md now documents the shared-fate gap explicitly: the design says a
process dies whole, the kernel today kills only the offending thread.
Also fixes three stale code comments (isr.s exceptionHandler, acpi.zig
sleepValue, build.zig boot-volume) — comments only, no behavior change.
The display service claiming the framebuffer suppresses the on-screen
boot transcript — correctly in normal operation, but on a serial-less
machine being debugged, the timeline vanishes just when it matters. A
diagnose image (zig build -Ddiagnose=true) has init skip the display
service and demo: the timestamped transcript stays on screen
indefinitely, and the power button still runs the orderly shutdown (so
the logger's files land when storage works).
First use immediately caught a real bug IN QEMU: an intermittent (~1
in 3) usb-storage READ CAPACITY failure after a ~21 s stall — after
which usb-storage and fat both exit cleanly and nothing retries: one
transient early-boot USB failure leaves the system permanently without
storage (and therefore without logs). That no-retry policy is a prime
suspect for real hardware never mounting /var, and is Track B's first
work item.
Real-firmware finding: the per-file /system tree walk boots in seconds
under OVMF but stalls for MINUTES on real firmware — the cost is not
bytes (USB 3 moves the ~4 MB instantly) but firmware filesystem
OPERATIONS: ~30 opens, each an uncached directory-chain walk in an
unoptimized firmware FAT driver. This is why every real OS loader
(winload, GRUB) reads many files through its own filesystem code over
Block I/O rather than the firmware's file protocol.
The loader now reads boot/system.img — the bundled binaries packed into
ONE v2 initial_ramdisk (tools/pack-system-image.py, derived from the
same bundled list in the same build graph, so tree and capsule cannot
drift) — with a single open + sequential read, the one firmware file
I/O shape that is fast everywhere. The manifest (open each listed path
by name) and the tree walk remain as fallbacks, so a hand-assembled
stick without the capsule still boots. The running system is identical
in all three cases: the kernel receives the same in-RAM table.
The load phase now brackets itself with unconditional on-screen
breadcrumbs ('EFI: loading the system...' / '...starting the kernel'),
because this phase stalling behind a silent black screen — kernel
status is serial-only by design — already cost a real-hardware
debugging session.
Direction (settled with the user): this capsule becomes the BOOTSTRAP
capsule — kernel + init + the storage-bring-up set — once a
spawn-from-memory syscall lets init and the device manager load
everything else from the stick's real file tree at runtime through
danos's own storage stack: file-granular updates (rebuild one binary,
copy one file), the initramfs shape.
Per docs/coding-standards.md (no Unix-abbreviation exception): syscalls
shared_memory_create/map/physical, kernel SharedMemoryObject + handlers,
runtime.shared_memory (library/runtime/shared-memory.zig), the
shared-memory-server/-client test services, the shared-memory QEMU case,
and docs incl. vdso.md's danos_shared_memory_*. 87/87 QEMU tests pass.
runtime.fs now routes every path through fs_resolve: kernel-served
/system nodes are read via fs_node (tokens, no open state); everything
under a userspace mount goes straight to the owning backend's endpoint
with the kernel-rewritten mount-relative path — one syscall of naming,
then the unchanged vfs-protocol rendezvous, public API untouched. mkdir/
unlink/rename resolve-then-forward (rename checks both paths land on
the SAME backend); mount is the fs_mount syscall.
The fat server mounts twice — /mnt/usb from the volume root and /var
from its /var subtree — so the logger now writes the FHS path
/var/log/<boot-stamp>/... and swapping the persistent medium later
touches only fat's two mount calls. With clients holding fat's node ids
directly, fat records each handle's owner, checks it, and sweeps a dead
client's handles via the published exit events (the old router's
pattern, now where the state actually lives).
The userspace vfs server and its router die; ServiceId.vfs=1 stays
reserved-retired; protocol.zig moves to system/vfs-protocol.zig (the
wire contract is backend-only now). vfs-test becomes the ring-3 proof
of the kernel VFS (own-binary ELF magic through /system, read-only
refusals, listing); vfs-client-death becomes the fat sweep test over
the full storage chain, with a ring-scanning check (the last-write
buffer is too racy under a chattering tree).
The logger service drains the tagged ring every 250 ms and demultiplexes
it into one file per process on the flash volume:
/mnt/usb/var/log/2026-07-21T150434Z/system/services/fat.log
[ 0.214] mounted FAT (fat32, 128992 clusters, partition lba 0)
The boot stamp is the wall-clock anchor from klog_status (FAT-safe, no
colons); the kernel's records go to kernel.log; each line carries the
record's monotonic timestamp and level. Storage is best-effort and
late — the ring buffers a whole boot until the mount appears, then the
first drain writes the backlog, storage-stack records included. Lost
records surface as '-- N records lost --' from sequence gaps. Files
close (= fat's device cache flush) after a 2 s quiet period, bounding
data-at-risk without per-record flush thrash. The logger announces
itself once — a periodic status line would feed the stream it drains.
init spawns the logger last, so the reverse-order shutdown stops it
first and its final drain runs over a live storage chain; log-flush and
init's own DANOS.LOG shutdown flush retire (superseded). runtime.fs
makePath treats components at or above a mount point as router names —
create is best-effort per prefix, the final verdict is exists(path).
Replace the linear keep-earliest RAM buffer with a 512 KiB ring of
framed records (log-ring.zig, host-tested): every debug_write becomes
one record per payload line, stamped by the kernel with the sender's
pid, task name (its binary path), level, per-boot sequence number, and
monotonic timestamp. Attribution is structural — a payload cannot forge
another sender's tag, and newline injection lands inside the forger's
own next record. Oldest records are overwritten when full; sequence
gaps make the loss countable.
debug_write gains a level argument (err/warn/info/debug/raw; old
two-arg callers clamp to raw). klog_read becomes a stream-offset read
that fails once the cursor falls behind the ring's tail; the new
klog_status (#45) returns the cursors plus the boot wall-clock anchor —
what the logger service will name per-boot log directories with.
The log now guards itself with a dedicated spinlock (BKL -> log lock
order, never the reverse); panic paths use a bounded try-acquire and
fall back to sinks-only. Serial rendering keeps the historical
transcript byte-identical for kernel and legacy raw output; leveled
records get a kernel-rendered name prefix. log-flush/init's interim
drains start at the ring tail and write framed records until the logger
service replaces them.
Retire the build-time ramdisk packer and the packed initial-ramdisk.img.
make-fat-image.py now lays every user binary out at its FHS path on the
boot volume (system/services, system/drivers, system/tests), and the EFI
loader walks \system at boot, packing what it finds into an in-RAM v2
initial_ramdisk whose entry names are full FHS paths. init rides the
table like every other binary: its dedicated handoff fields are gone and
the kernel spawns PID 1 via the same lookup as everyone else
(process.spawnBundled).
system_spawn resolves names by exact path first, then unique basename,
and normalizes argv[0] to the stored path — so task names (and, next,
the tagged log ring's attribution) are honest binary paths everywhere.
initrd v2 rejects the old magic so a stale image fails loudly.
Groundwork for per-process logging (/var/log/<boot-stamp>/<binary-path>.log)
and the kernel-VFS /system mount.
Daniel Samson
Daniel Samson