P2 of docs/establishment-planes-plan.md. usb-storage serves nameless — one
process per stick cannot share an exclusive bind, and a second stick used to
die silently on -EBUSY before ever helloing. Its one hello now moves both
directions at once: the block-serving endpoint up, its controller channel
down. fat finds its volume through the manager — a new `.consumer` role asks
for the channel of the driver BOUND TO a device (distinct from the device's
reporter), found by enumerating the tree for the mass-storage identity.
fat stays single-volume; the boot-volume-by-content choice is M21.
The conversion immediately caught a live truncation of exactly the audit's
shape: ChildEntry grew to 32 bytes, one enumerate reply holds ~7, and a real
tree carries a dozen ACPI nodes before the first USB child — the storage
entry silently never fit (the protocol comment already said "paging joins
the protocol if a tree ever outgrows one packet"). enumerate is now paged:
Header.target is the start cursor, a short page is the end; device-list's
page-0 read is unchanged.
Grant rows move with the code: the block bind and fat's block open die, fat
gains open device-manager. Gate: 18 cases green including the registry
trio, device-list, and both IOMMU storage variants.
P0 of docs/establishment-planes-plan.md; no behavior changes yet, nothing
sets the new flag or sends a hello capability.
- service.run gains a reply-capability out-slot (replyWithCapability), the
registry idiom init already uses, lifted into the harness; null stays the
untouched common path. Subscribers gains claimArrival() so a provider
handler can keep a turn capability through the same flag the reserved
subscribe uses.
- Hello wire struct: the padding byte becomes wants_channel — old callers
wire-compatibly say 0, and the manager nominates a reply capability ONLY
when asked, because a capability sent to a caller that never reads one is
a leaked slot in that caller's table.
- driver.helloExchange: one handshake can hand a serving endpoint up and
receive the device's provider channel down (usb-storage will need both at
once). No channel in the reply is retryable, never a verdict.
- The manager stores each instance's serving endpoint on its Driver entry,
routes consumer hellos by lineage (child -> reporter -> endpoint), replaces
on re-hello, and closes the stale handle on death - the 32-slot table is
the bound that makes forgetting this boot-fatal.
The name list and its predicate existed so drivers could move to delegation
one at a time with the suite green throughout. Every driver is delegated
now, so the manager simply hands over whatever device a driver was assigned.
Deleting it caught a real consequence: crash-test finally got delegated too,
and it was still claiming its device — so it got AlreadyClaimed because it
already held it, exited, and the restart drill had nothing to restart. Its
own comment named what the case was really checking: "the respawn only
reaches this line because the kernel released the previous instance's claim
at death". That property still holds, by a different mechanism — the device
reverts to the manager on death and is handed to the replacement, which is
the same guarantee without the race it used to rely on.
All four delegation paths verified: the xHCI controller, the PCI bridge, the
PS/2 two-node singleton, and virtio-gpu's restart re-attach.
Run 3 complete. Suite 118/118.
A device nobody holds can be claimed by anyone, so the manager now takes
every firmware-discovered device that carries mappable resources, whether or
not a driver wants it. The real gap was the HPET: an MMIO window, an IRQ, no
user-space driver, and there for the taking. Held by the manager it is
inert; unheld it was a way into physical memory.
Two deliberate exclusions. The loader's framebuffer, which the compositor
claims and which the manager must not take because it starts first. And
anything with no resources, which grants nothing worth holding.
Scope is the boot snapshot. A device reported later and matched to no driver
stays claimable — pci-cap-test and iommu-fault-test both reach an unmatched
NIC that way, so narrowing it is a separate change with those fixtures in
scope. Recorded in the plan rather than left implied.
The attacker fixture gains the assertion deferred since D2: after the system
settles, nothing with resources may be taken.
That assertion defeated itself twice before it worked, and both failures are
worth remembering. First it swept at 0.029 while the manager did not bind
its protocol until 0.047, so it reported a hole that closed a millisecond
later. The retry loop that "fixed" that was worse: the first pass TAKES the
device, so the second finds it unavailable because this process now holds
it, and concludes all is well — it passed with the manager's claiming
removed entirely. It now settles once and sweeps once, and fails when the
claiming is removed.
Suite 118/118.
The last claimant. The kernel seeds the acpi-tables node, so it sits in the
same boot snapshot the manager already scans to find the PCI host bridge —
there was never a bootstrap problem, only a lookup nobody had written. The
manager claims it and names it in the spawn; the service stops claiming.
Every driver in the system now receives its hardware rather than taking it.
Two failures on the way, both mine. addDriver puts the device id in argv[1],
and the acpi service read argv[1] as a self-verify device-count floor — so
handed device 7 it decided it was in test mode, printed "acpi-parse: ok",
and never reported a device. The test argument is now floor:N, which a bare
id cannot be mistaken for.
And acpi-parse spawns the service directly rather than through the manager,
so nothing handed it the node. That test now claims and transfers it exactly
as the manager does, which is the right shape: the test plays the manager's
role instead of the service reaching for hardware.
device_claim now has two callers left: the manager, which is the acquirer
and should have it, and the display service's GOP path. That is recorded as
question 10 — the framebuffer is not a device, so the answer is likely that
it leaves the device table rather than being exempted from its rules.
Suite 118/118.
The 8042 is a single controller described by two ACPI nodes — PNP0303
carries the 0x60/0x64 ports, PNP0F13 is the mouse — so it cannot be split
across two processes without them fighting over the same registers. That is
why ps2-bus is a singleton, and why it used to find and claim both nodes
itself.
The manager now gives it every matching node instead. The keyboard node
rides the spawn, because it holds the ports and is needed immediately; the
mouse node is transferred to the already-running instance. Late arrival is
safe here and the ordering is natural rather than lucky: the mouse is not
touched until after the controller handshakes and identify. Measured, the
handover lands at 0.336 and the driver reaches the mouse at 0.456.
Because the count is however many matched, a machine with no PS/2 ports or
only one works without a special case — which matters, since the bus is
mostly emulated now and machines vary.
ps2-bus claims nothing. irq_bind on the mouse node is the proof it holds it:
that call is ownership-gated, so a failure means the handover did not land
rather than a hardware fault, and the log says so.
acpi-ps2 asserts both delegations with the spawned device backreferenced, so
the node that rides the spawn must be the one the driver was spawned for.
Disabling the second delegation fails it.
Two things worth recording. The first discrimination patch was not valid Zig,
so nothing ran and a stale binary reported a pass — checked the build before
believing it. And with the second delegation disabled, acpi-ps2 fails while
input still passes: the mouse works without its IRQ binding, so exactly one
case covers that path.
Suite 118/118.
Third driver converted. It no longer claims the id from argv[1] — the
manager holds the device and names it in the call that creates the process,
so it is held before the driver's first instruction.
display-reattach is the case that matters here: it kills the driver and
watches the compositor re-attach to the fresh scanout. It passes, so the
restart path survives the fused grant — the manager re-takes the device when
the driver dies and hands it to the replacement.
ps2-bus and discovery are NOT converted, and the reason is recorded as open
question 9 rather than worked around. Both need a device nobody assigned
them. ps2-bus ignores its argv[1] entirely: it finds the controller by
walking the table for PNP0303, then claims a second device, the PNP0F13
mouse node, which it also finds itself — so it holds two devices and was
assigned at most one, while system_spawn carries one. discovery claims the
acpi-tables node it locates itself, because it is what produces the device
tree and there is nothing to assign at that point.
One thing worth checking before designing an answer: devices.csv maps both
PS/2 hardware ids to ps2-bus, so the manager may already be spawning two
instances where the driver expects one. If so the fix is smaller than it
looks.
D6 stays blocked — closing device_claim with these two still depending on it
would stop the machine booting.
Suite 118/118.
Delegation moves out of onHello and into the spawn itself. The manager holds
the hardware and names it in the call that creates the driver; the kernel
checks the device is the caller's to give, then hands it over as part of
making the child.
The reason is the window. A transfer after spawning always leaves an
interval in which the child is running and does not yet hold its device. It
would close on QEMU every time and open occasionally on a machine with
different core counts and timing — the exact failure shape this track exists
to delete, and not one worth introducing while removing the others. Fused
into the spawn there is no interval: the child does not exist until it holds
the device.
Ownership is checked BEFORE the child is created, so a refusal leaves
nothing running rather than a driver without the hardware it was spawned
for. The IOMMU confinement moves with the device, as it does on the transfer
path. systemCall6 is added for the sixth argument; r9 was free, and abi
gains a no_device sentinel matching the protocol's.
No driver had to change to receive a device, which is what makes this
better than requiring every driver to hello: ps2-bus keeps its legacy
status, and discovery — which has no assignment at all, since it is what
produces the device tree — is unaffected.
The attacker fixture now tries the spawn as a back door: name someone else's
device, and both the spawn and any child must be refused. Verifying that
assertion exposed a bug in the fixture itself. The kernel case's pass marker
was "device-authority: ok", which matches the FIRST per-assertion line, so
its wait loop exited before any failure was printed — the case would have
passed with failures in it, and had been able to since D2. The verdict lines
now carry a distinct VERDICT prefix, and with the ownership check removed
the case genuinely fails. A green test that cannot go red is worse than no
test.
Suite 118/118.
pci-bus joins usb-xhci-bus in receiving its device from the manager rather
than claiming the id it found in argv[1]. Its hello moves ahead of the ECAM
mapping, since that is where the bridge now arrives, and its hello was
already mandatory so nothing about its failure behaviour changes.
isDelegated compared whole strings, which silently missed this driver: the
manager records the boot-snapshot match as the bare "pci-bus" and a
devices.csv match as the full "/system/drivers/pci-bus". pci-bus was then
neither claiming nor delegated and died on "ECAM mmio_map failed". It now
matches on the last path component. Reintroducing the whole-string compare
breaks usb-xhci-bus instead of pci-bus — the two spellings swap which driver
loses — so usb-hid is the case that catches it, not pci-scan.
pci-scan asserts the delegation on the initial bring-up AND after the
restart drill, with the device id backreferenced so both must name the same
device. That is what proves the manager re-takes a device when its driver
dies and hands it to the replacement, which is the property the whole
supervision design rests on.
The remaining three claimants are NOT converted, and the plan records why
rather than working around it. ps2-bus and the acpi service never hello at
all, which device-manager.md states deliberately ("legacy drivers ... not
yet required to hello"), so delegating to them means either promoting them
out of legacy or giving the grant a delivery point that is not hello.
virtio-gpu hellos best-effort by design — "standalone bring-up has no
manager" — and delegation would make it mandatory. Both are decisions, not
mechanical steps.
Consequence: D6 is blocked, because device_claim cannot be closed off while
three claimants still depend on it. D7-D9 are unaffected — they concern what
the kernel stores and how its table is sized.
Suite 118/118.
The first driver to stop claiming its own hardware. The device manager holds
the controller and transfers it in the hello reply, so its matching becomes
authoritative instead of advisory — until now the driver claimed the id it
found in argv[1], and any process could have claimed the same integer first.
The manager claims before it spawns, so there is no window in which anything
else could take the device, and transfers in onHello using invocation.sender
— the kernel-stamped task id, which cannot be forged by the caller. hello is
synchronous, so the transfer has completed before the reply lands: no gap
between being told yes and holding the thing.
usb-xhci-bus's hello moves from after controller bring-up to before anything
that needs the device, which is the bring-up reorder the design predicted.
It is the first member of an explicit delegated set, so every unconverted
driver keeps claiming exactly as before and the suite stays green; the set
and device_claim both go at D6. D3 and D4 could not be separated and the
plan records why: the moment the manager claims, any driver still calling
device_claim is refused, and D3 applied to nothing changes no behaviour and
cannot be tested.
This step introduced a regression and the incremental conversion is what
caught it. confineDevice runs inside systemDeviceClaim, so a device arriving
by transfer was never confined for its new owner. Three IOMMU+USB cases
failed on the driver's DMA rings going unbound, and two worse consequences
were latent: a manager death would have torn down a domain a live driver was
using, and a driver death would have leaked one. iommu.reassign now moves
the confinement with the device, keeping the domain and its attachment
intact so it never translates through nothing. Converting all five drivers
at once would have produced the same three failures with five suspects.
A log line of mine claimed "holding controller device N" before anything
verified it — it printed even in the failure case, where the driver held
nothing. Reworded to state only what is known there: where the registers
are.
usb-hid asserts the delegation with the device id backreferenced, so the id
delegated and the id the driver ends up with must match. Emptying the
delegated set fails it with "hello acknowledged" then "mmio_map failed".
usb-hub failed once in a full run and has passed six times since (four
isolated, two full) — recorded in the plan as a suspected instance of the
known intermittent AP fault, not dismissed, since this step did shift boot
timing.
Suite 118/118.
Three services had each written the same thing and got it three different
ways: input polled the process list to notice a dead subscriber, and only
when someone else subscribed; the power service never noticed at all; the
device manager noticed drivers but not subscribers. The harness owns the
table now, driven by the events a protocol declares — it registers on the
reserved verb, frames each event once, posts to everyone interested without
waiting on any of them, and reclaims a slot when the kernel says its owner
died. Interest masks moved to the envelope, so a subscriber that wants only
mice asks the same way everywhere.
Two consequences the plan had not foreseen. The device manager now hears a
supervised child's death twice, once as its supervisor and once as a
subscriber, so restart backoff counted every crash twice and gave up after
half as many; it retires the id before counting. And the kernel's published
exit table had eight slots for what is now six subscriptions in a plain
boot, so it holds sixteen.
The other half is a hole the design named early and left standing: a
backend handed out a small integer and then honoured it from anyone. A
process that guessed a file's node id read another client's file; a display
layer had no owner at all, so any client could reconfigure or destroy any
layer; a USB device token was never checked against the client that opened
it. Each is now bound to the task that opened it, and a wrong owner gets
exactly what an unknown id gets — the refusal must not become the oracle
the identical answers elsewhere were designed to remove. Closing a file
changed with it: it used to succeed unconditionally, which would have told
a caller which ids existed.
Suite 111/111, with a new case in which one process holds a file and a
layer, hands both ids to a second process, and finds them untouched after
that process has tried everything with them.
These were the awkward ones. Each began with an operation packed into a
single byte — two of them with a version wedged in beside it — so there was
no wrapping them: the layouts had to be rebuilt. The device manager's own
enumerate and subscribe become the reserved verbs that mean the same thing
everywhere, its replies lose three status structs the envelope already
carries, and a device id becomes the packet's target. Power drops the
version it repeated on every request, because describe is the handshake,
and stops claiming a 64-byte ceiling it never needed for calls. USB moves a
control transfer's data to the packet tail in both directions, which makes
the status length the transferred length and retires a field that had been
saying the same thing twice.
The danger in this one was not the protocols but their readers. Init
recognised a power button by two bytes at the head of a message, the ACPI
service dispatched on the first byte, the xHCI driver read its operation
with a raw integer load, and the HID drivers reinterpreted a report
wholesale — none of which would have failed to compile once the layouts
moved. They would simply have stopped: no shutdown on the power button, no
reports from the keyboard. Every one of them now reads through the
generated types, and the shutdown gate that answers only a subscriber is
the same code it was.
Two sizes were decided by measuring rather than assuming. The child-added
message is both a request and the event broadcast to subscribers, and
alignment rounds it to 48 bytes, which puts its packet exactly on the
64-byte push floor — a test pins that, because a field added carelessly
would now overflow it. The interrupt report gives up eight bytes of inline
room to make space for the header; the two drivers that produce reports
send eight and four.
Suite 110/110.
A protocol is reached by name now, not by a compile-time integer. Init is
PID 1 and already knows which binary it started, so init serves /protocol
as a vfs backend: bind claims a contract with the provider's endpoint
attached, open answers with that endpoint as the reply's capability, and
readdir lists what is bound with the task and binary behind it. The kernel
reserves the prefix — nothing may mount over it, under it, or unmount it —
and ServiceId, ipc_register and ipc_lookup are gone, their syscall numbers
left vacant.
A bind is authorized by who the caller *is*: the kernel-stamped binary
together with the supervising task's identity, matched against
/system/configuration/protocol.csv. Identity, not spelling — spawn is
ungated, so an attacker can run any bundled binary, and a name-only rule
would have let it launder grants through an init of its own making. A name
a live process holds is refused to everyone else; a dead one's is released.
Three review rounds against a hostile ring-3 process found what 108 green
tests could not, because the suite contains no attacker. Publishing init's
supervision endpoint as the registry put PID 1's mailbox in every process's
hands, where two forged bytes reached the shutdown path: privileged traffic
is now believed only from the task that holds the contract it speaks for.
A capability arriving on a request outlived every path that ignored it,
one handle per call until the table was full — in init, and in the harness
ten services share — so the arriving capability is owned by the turn and
released unless a handler says otherwise. And the kernel let anyone holding
an endpoint handle aim signals, timers, exit notices and interrupts at it:
binding now requires having created it.
Suite 108/108. The new protocol-registry case asserts eleven properties,
each one an attack that must fail.
/etc/init.csv and /etc/devices.csv become /system/configuration/*.csv (the
repo's etc/ moves to system/configuration/, mirroring the runtime tree),
/var/log becomes /system/logs, and /mnt/usb becomes /volumes/usb. The
kernel VFS gains a carve-out so FAT may serve exactly /system/configuration
and /system/logs beneath the initrd-backed /system while /system and /test
themselves stay unshadowable; FAT's single /var mount splits into those two
rewritten mounts. The kvfs readdir check learns /system's third child and
the ramdisk spawn sweep skips the configuration tree.
Suite 106/106.
The module-to-domain table in build-support duplicated what each
domain's build.zig already states with its addModule exports. userBinary
now resolves each named import by searching the packages the binary
declared in its own build.zig.zon (b.available_deps), which also makes
the zon the literal include path: an import can only be satisfied by a
domain the binary claims, and naming a module whose domain is missing
fails the build graph with the domain to declare. build-support is down
to the recipe alone. All build variants green; manifest unchanged.
Every binary's build.zig now names precisely the modules its source
imports (derived by scanning each artifact's sources, transitively
through same-directory files), and its zon carries only the domains
those come from — kernel stays implicit (the root shim + link script
live there). build-support's userBinary resolves each name through one
module-to-domain table (module_homes); Domains/domains()/defaultImports
and the raw recipe entry point are deleted. An undeclared @import is a
compile error (verified: injecting @import("xkeyboard-config") into
logger fails with 'no module named ... available within module
program'), and e.g. xkeyboard-config now appears in exactly two
manifests — the two keyboard drivers. Availability never bloated the
emitted binaries (Zig compiles only what a program imports); this makes
the declared interfaces honest. Production and -Dtest-case manifests
byte-identical; all build variants and standalone package builds
green.
init, fat, display, display-demo, device-manager, input, logger, and
the two discovery fillers (acpi, fdt — each exporting an artifact named
"discovery"; the root -Ddiscovery picks which ships) convert to binary
packages on the pci-bus template. init's serial heartbeat flag rides
the dependency options (the root forwards its -Dserial). fat's and
display's unit tests move into their packages and the root aggregate
delegates to them. Boot-image file list unchanged.
Replace the three hardcoded switch tables (pciDriverForIdentity, hidDriverFor,
usbDriverForIdentity) with an authoritative, human-readable device registry the
manager reads at boot. Matching is most-specific-wins across
base/subclass/prog_if/vendor/device/subsystem/hid, so a precise vendor:device
rule and a generic class rule coexist; an unmatched device is logged, never
guessed. This resolves docs' "matching stays code until the third bus".
- ABI: child_added and DeviceDescriptor gain vendor/device/subsystem; child_added
gains a bus discriminator (BusKind) so PCI and USB class triples match against
the right namespace.
- pci-bus reads vendor/device (config 0x00) and subsystem (0x2C, type-0) and
reports them.
- library/device/registry: freestanding CSV parser + matchDriver() with
specificity scoring; 5 unit tests wired into `zig build test`.
- etc/devices.csv bundled into the initrd; the kernel serves /etc directly, so
the manager reads it before any filesystem service is up (fat starts later).
- virtio-gpu: drop the now-redundant post-spawn 1AF4:1050 re-confirm, since the
registry binds this driver by exact identity.
- Remove the orphaned system/drivers/display driver (unreferenced by build or
registry).
- docs: new devices-csv.md; device-manager.md "matching stays code" resolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Every user binary and the two device-logic library modules (pci, usb) now
`@import` the concern modules directly instead of aliasing through `runtime`:
runtime.ipc/process/time/service/input/block/display -> @import("<module>")
runtime.device / runtime.device_manager -> @import("driver")
runtime.fs -> @import("file-system")
runtime.Thread -> @import("thread").Thread
runtime.system.{write,writeRecord,klog*} -> logging.*
runtime.system.{sleep,timerOnce,wallClock,clock} -> time.*
runtime.system.{spawn*,kill,exit,yield,processes,...}-> process.*
runtime.system.{mmap,munmap,PROT_*} -> memory.*
runtime.dma.* / runtime.shared_memory.* / runtime.allocator -> memory.*
Each consumer keeps its own alias name (e.g. `const device = @import("driver")`),
so call sites are unchanged and there are no collisions with local `driver`
variables. build.zig now injects the concern modules into every user binary via
`default_imports`; pci/usb module import lists were updated to match.
The `runtime` and `system` shims remain for one more step (root.zig still uses
runtime); they are deleted in C5. Nothing but root.zig imports `runtime` now.
Verified: zig build, zig build test, and 17 QEMU cases (smoke, device-manager,
logger, fat-mount, fat-mutations, usb-storage, usb-hid, display-native,
virtio-gpu, input, thread-spawn, thread-mutex, process-kill, shared-memory,
driver-restart, acpi-ps2, pci-scan).
Finish the direct-import convention: a protocol is now aliased to its module
name in snake_case in every file that imports one — consumers and the runtime
client wrappers alike. This kills the last of the alias variety (the generic
`protocol`, plus `power` and `transfer`), so `device_manager_protocol.Hello`
means the same thing everywhere and grepping a module name finds all its uses.
Pure rename (identifier uses only; prose references left untouched via a
guarded pattern). 14 files, 299/299 lines.
zig build + test green; 16 QEMU cases pass (smoke, device-manager,
driver-restart, device-list, pci-scan, usb-hid, usb-storage, display-service,
display-native, display-reattach, input, acpi-ps2, vfs, fat-mount,
power-button, orderly-shutdown).
device-manager-protocol -> library/protocol/device-manager/. Its consumers
(pci-bus, usb-xhci-bus, the acpi discovery service, device-manager, device-list,
crash-test, and the not-yet-built intel-integrated display sub-driver) import the
module directly instead of through runtime.device_manager_protocol, which is
deleted. runtime.device_manager (the hello client) already imported the module
by name and is unchanged.
zig build + test green; device-manager, driver-restart, device-list, pci-scan pass.
Start the protocol tree: wire protocols move to library/protocol/<name>/,
entry file <name>-protocol.zig, module name unchanged. block and usb-transfer
are pure moves — every consumer already imports them by module name, so only
the b.addModule paths change and the empty system/services/block/ is removed.
block-protocol -> library/protocol/block/block-protocol.zig
usb-transfer-protocol -> library/protocol/usb-transfer/usb-transfer-protocol.zig
zig build + test green.
library/runtime/log.zig wires std.log to the tagged ring: the root shim
installs std_options for every binary (programs may override), logFn
formats one line per record and emits it with its level via
debug_write — the payload no longer carries the process's name; the
kernel stamps identity structurally and the serial renderer prints the
'<binary path>: ' prefix, so the transcript keeps its shape.
Migrate every writeLine/logLine/log helper family (init, vfs, fat,
device-manager, acpi, pci-bus, ps2-bus x3, usb-hid x2, usb-storage,
usb-xhci-bus, virtio-gpu — 104 call sites) to std.log.info, dropping
the hand-written prefixes. A leveled record is a complete line by
contract (raw emissions may still build lines from pieces). Test
fixtures and the display/input services keep raw writes for now — their
ring records are attributed by the kernel regardless.
Harness regexes follow the renamed prefixes (usb-hid-keyboard,
discovery) and path-named restart lines.
The EFI loader now ENUMERATES /system rather than opening files by name,
so the on-disk name must be exact, not merely case-insensitively
findable. make-fat-image.py stored 8.3-fitting lowercase names as bare
uppercase short entries ('init' -> INIT), garbling the ramdisk paths;
now any name whose case the 8.3 entry cannot reproduce gets a long-name
chain carrying the exact name.
device-manager's driver table stored names in 24 bytes — silently
truncating '/system/drivers/usb-xhci-bus' and failing the spawn; widen
to 64 (= abi.maximum_process_name). The usb-report harness regex learns
that restart lines name drivers by path.
Retire the build-time ramdisk packer and the packed initial-ramdisk.img.
make-fat-image.py now lays every user binary out at its FHS path on the
boot volume (system/services, system/drivers, system/tests), and the EFI
loader walks \system at boot, packing what it finds into an in-RAM v2
initial_ramdisk whose entry names are full FHS paths. init rides the
table like every other binary: its dedicated handoff fields are gone and
the kernel spawns PID 1 via the same lookup as everyone else
(process.spawnBundled).
system_spawn resolves names by exact path first, then unique basename,
and normalizes argv[0] to the stored path — so task names (and, next,
the tagged log ring's attribution) are honest binary paths everywhere.
initrd v2 rejects the old magic so a stale image fails loudly.
Groundwork for per-process logging (/var/log/<boot-stamp>/<binary-path>.log)
and the kernel-VFS /system mount.
The compositor now survives the virtio-gpu driver dying and re-attaches when device-manager
restarts it — the last piece of display v2.
Three parts:
- The driver hellos the device manager (role: bus). It never did, so the manager — which
spawns it supervised and expects a hello — was stopping it at the 3s hello deadline every
run (the gate markers just printed first). Now it is properly supervised: not stopped for
silence, and restarted on death.
- A kernel IPC fix so a call to a dead service errors instead of hanging. An endpoint records
its owner; when that task dies, its registered endpoints are marked dead (and any parked
senders woken with -EPEER), so ipc_call returns -EPEER rather than blocking on a reply that
will never come. Without this the compositor's first present after the driver died blocked
forever. General robustness — any client of any service benefits.
- The compositor re-attaches. Its .scanout calls now fail cleanly (caught), freezing the last
frame; when the restarted driver re-announces, attach_scanout detects the backend is already
virtio and logs "scanout re-attached", mapping the fresh shared surface and re-looking-up
.scanout. (The previous shm mapping leaks — no shm_unmap syscall yet — but its frames are the
dead driver's, reclaimed on exit.)
- device-manager gains a "test-scanout-restart" mode (like test-usb-restart) that kills the
virtio-gpu driver once after it hellos; the displayReattachTest kernel scenario drives it.
Gate: python3 test/qemu_test.py display-reattach — "scanout upgraded to virtio-gpu" then
"scanout re-attached", no CPU exception, passing 3/3. host tests, ipc/ipc-call/ipc-cap,
supervision, shm, display-service, display-demo, virtio-gpu, display-native, and
display-modeset all pass; default zig build clean. v2 (V1-V6) complete.
The first native scanout backend's driver half. The device manager matches the
virtio-gpu PCI function by its Display/Other class triple and spawns the driver,
which claims the device, confirms vendor 0x1AF4/device 0x1050 from config space,
enables memory-space + bus mastering, and walks the virtio vendor capabilities to
find the common-config and notify structures in its BAR.
From there it is the standard modern-virtio bring-up: reset, negotiate VERSION_1,
stand up the control virtqueue in coherent DMA, then drive the GPU end to end —
RESOURCE_CREATE_2D, ATTACH_BACKING (a coherent DMA buffer; V4 swaps in the shm-
shared surface), SET_SCANOUT, paint a known pattern, TRANSFER_TO_HOST_2D,
RESOURCE_FLUSH, and wait for the device's used-ring ack. Reading the backing back
proves it is CPU-visible RAM; the ack proves the device consumed the frame —
together the automated stand-in for "it's on screen", no screenshot.
- system/drivers/virtio-gpu/: the driver, plus virtio-gpu-protocol.zig (control
commands) and virtio-pci.zig (the 1.0 PCI transport + split-virtqueue), both with
host-tested struct sizes.
- device-manager matches the display/other class triple to "virtio-gpu"; the driver
self-confirms the vendor/device id, since the class alone cannot distinguish it.
- ServiceId.scanout (11): the driver registers it so the compositor finds it in V4.
Gate: python3 test/qemu_test.py virtio-gpu (QEMU -device virtio-gpu-pci) — the
device-manager stack discovers the function, the driver brings up a 640x480 scanout
and flushes a test pattern: "virtio-gpu: scanout 640x480 online" + "flush acked,
pixel check ok". host tests, display-service, shm, and device-list still pass.
Note: the pre-existing pci-scan case triple-faults on main (verified at 88ad432,
before this change); it is unrelated and tracked separately.
Flesh out the xHCI host-controller driver into a full transfer engine and build
the three USB class drivers on top, all verified end to end under QEMU.
- xHCI engine (usb-xhci-library.zig): controller reset, command/event rings with
cycle-bit bookkeeping (gated on a No-Op-command proof), device slots, Address
Device, control transfers, full chapter-9 enumeration, Configure Endpoint, and
interrupt/bulk transfers. Each interface is device_registered with its
(class,subclass,protocol) identity, unique per (port,interface).
- Bus<->class transfer protocol (usb-transfer-protocol.zig + runtime.usb): open /
control / interrupt-subscribe (async report pump on a poll timer) / bulk-by-
physical-address, so sector data never crosses the 256-byte IPC limit.
- USB HID keyboard + mouse (usb-hid/): decode boot-protocol reports and publish
to the input service. A USB usage is already the input protocol's keycode.
- USB mass storage (usb-storage/): Bulk-Only Transport + transparent SCSI,
serving a block device under the new .block service id (block-protocol).
- device-manager matches USB interfaces to class drivers (usbDriverForIdentity).
- usb-abi / usb-ids made importable modules; add HID and mass-storage class
requests, packTriple, and a usb_device DeviceClass.
- Fix test/qemu_test.py on macOS: the QMP unix-socket path was built from the
deep worktree path and exceeded the 104-byte sun_path limit, so QEMU exited
before booting. It now lives under a short temp path.
Tests: usb-report, usb-hid, usb-storage pass under python3 test/qemu_test.py;
host units (usb-abi, usb-ids, hid-report, bulk-only-transport, scsi) green.
Time is a kernel concern in danos: the kernel owns the scheduling timer and
already exposes monotonic time via the clock/sleep/timer_bind syscalls, so a
userspace time service would be a redundant, slower path. This adds the generic
runtime.time module over those syscalls, retires the two demonstration drivers,
reorganizes the milestone docs, and makes the monotonic clock correct on Intel,
AMD, and inside any VM.
runtime.time (library/runtime/time.zig)
- Instant/Duration interface: now, sleep, spin, after, monotonicNanos, available
- a thin layer over system.clock/sleep/timerOnce; unit-tested arithmetic
Remove the demo drivers hpet and bus (a teaching example belongs in the docs,
not shipped in the tree)
- system/drivers/ now holds only real drivers: pci-bus, ps2-bus, usb-xhci-bus
- device-manager end-to-end test repointed to pci-bus (asserts on kernel state:
the process table and the device tree, not a racy serial marker)
- device_register containment moved to a new in-kernel `containment` test
- the driver-model worked example moved inline into docs/drivers.md
Reorganize milestone docs into topic docs
- m17-m18 / m19-m20 / m21 plans dissolved into process-lifecycle, device-manager,
discovery, and acpi docs; new docs/power.md and docs/timers.md; ~20 citations
repointed; plan docs deleted
TSC reliability (apic.zig, smp.zig, cpu.zig, kernel.zig)
- check the invariant-TSC bit (CPUID 0x80000007 EDX[8]) on Intel and AMD
- cross-core "warp" check at SMP bring-up, pairwise BSP<->AP as each core comes up
- fall back to the HPET clocksource when the TSC is not invariant (a bare VM) or
not synchronized (a warp), switched continuously so time never jumps
- boot log reports the outcome; new tsc-sync test exercises the TSC + warp path
Verified: zig build; zig build test; 60/60 QEMU cases (incl. new containment and
tsc-sync).
Two related changes to make device identities legible in the boot log and in
the code that matches on them.
Logging: the pci-bus driver decodes each function's class/subclass/prog-IF
triple to human names (via the existing pci-class module), and the acpi
service appends each _HID's human name (via acpi-ids) to its report line. So
"class 0x01 (Mass Storage Controller) subclass 0x06 (Serial ATA Controller)
progif 0x01 (AHCI 1.0)" reads straight off the log when writing a driver.
Naming: a new coding standard ("Named values, not magic numbers") says a value
with meaning gets a name, prefer an enum for value sets. Applied:
- pci-class is refactored from u8-switch tables into a BaseClass enum plus
per-class SubClass/ProgIf enums with name() methods (the usb-ids shape). The
public className/subclassName/progIfName(u8...) API is unchanged, so the
hardware-byte decoders (pci-bus, the kernel dump) are untouched; output is
byte-identical.
- the device-manager builds the xHCI class triple from named parts instead of
a bare 0x0C0330.
- the acpi service's _CRS walk names its resource-descriptor tags as
SmallResourceType/LargeResourceType enums, and the _HID integer decode uses
the AML module's existing *_opcode constants (now re-exported from aml.zig)
rather than bare 0x0A/0xFF/... literals.
The kernel no longer folds AML Device objects into the device tree — the
ring-3 acpi service is the sole builder of _HID device nodes. The kernel
keeps building the namespace only for the \_S5 sleep type, and still
seeds the static tables (MADT, HPET, MCFG, FADT) and the acpi-tables node.
The device manager matches ps2-bus from the service's _HID reports
(PNP0303 / PNP0F13, singleton-deduped) instead of boot-snapshot nodes;
its dead boot-snapshot ps2 arm is gone. The service registers every
device before reporting any, so a driver the manager spawns on the first
report already sees the full set — no keyboard-before-mouse race. The
acpi-ps2 scenario proves the whole chain: report -> spawn -> ps2-bus
finds the controller and attaches its keyboard, entirely in ring 3. The
ioport test moved to the acpi-tables I/O window, since the kernel-built
PS/2 node it used to scan for no longer exists. The retired
device-building functions in acpi.zig are dead but retained (a botched
mechanical deletion is worse mid-migration than a follow-up sweep, which
is flagged as a task). Suite 58/58.
AML method evaluation now runs in userspace touching real hardware: the
service builds an interpreter with a ring-3 Hal (port I/O routed through
its claimed acpi-tables node; a scratch page backs SystemMemory maps so a
stray OperationRegion degrades to zeros instead of faulting a process
that cannot map arbitrary physical memory). It walks the namespace and,
for each present _HID device that is not a PCI root, evaluates _CRS,
registers it under acpi-tables, and reports it with its EISA-decoded hid.
Containment for this needed the broker's irq check to become range-based
— an interrupt line is still indivisible, but a parent may own a range,
so the acpi-tables node's broad irq window contains its children's legacy
lines (a length-1 range is exactly the old equality, so single-irq
parents are unaffected). ChildAdded gained a hid field for firmware
string identity. Matching those reports to drivers stays off until M20.3,
so ps2-bus still comes up via the kernel path — no regression. The
acpi-report scenario proves the PS/2 keyboard (io 0x60/0x64 + IRQ) and
mouse (IRQ) are reported with their resources. Suite 57/57.
The AML module becomes a build module compiled into both the kernel (for
the \_S5 sleep state it still needs) and the new acpi service — one
source, two builds, no fork. The kernel publishes a single acpi-tables
node: the DSDT/SSDT blobs as memory resources, a broad io_port grant (the
honest trust boundary — firmware AML names whatever ports it chose, known
only after parsing), and the SCI for the M21 event track. The acpi
service claims the node, maps each blob through the ordinary mmio grant
(which preserves the sub-page offset onto the bytecode), and runs the
same parser the kernel does. It self-verifies its namespace Device count
against the kernel's — 34 = 34 — deterministically via an argv the
acpi-parse test passes, so no racing the shared serial buffer. Parse-only
touches no hardware; OperationRegion evaluation waits for _CRS/_STA in
M20.2. The manager spawns 'discovery' (the neutral ramdisk name) at
startup. Suite 56/56.
enumeratePci, addBars, pciConfigurationPtr, and the PciHeader struct are
deleted; the kernel seeds only the host bridge, and the ring-3 pci-bus
driver's reports are the sole source of PCI function nodes. The manager
matches PCI drivers from reported identity, deduped by registered device
id so a bus restart never double-spawns.
The flip did its job by exposing a latent SMP race: ring-3
device_register made the broker table concurrent for the first time, and
mmio_map read it lock-free — under load a torn resource length mapped
hpet's window wrong (its user fault) and underflowed r.len-1 into a
kernel integer-overflow panic. Fixed: the broker read in mmio_map (and
claim) runs under the big kernel lock, the arithmetic rejects
zero-length and wrapping windows cleanly, and pci-bus no longer registers
unimplemented size-0 BARs. driver-restart hammered 6x, suite 55/55.
Each function is registered under the bridge with the config-space slice
and BARs sized by the same all-ones probe the kernel uses — byte-for-byte
equal descriptors, so the idempotent register returns the kernel's
existing node ids during coexistence instead of duplicating the tree.
The bridge gained the 16-bit io_port aperture that functions' I/O BARs
need to pass containment. Reports carry the registered device_id, and
the pci-scan scenario drills a forced restart: kill the enumerator after
its reports, watch the respawn re-scan, and assert the broker's PCI node
count never grew. The usb-restart test trigger is pinned to the xHCI
reporter (pci-bus racing it to two reports used to steal the kill).
Harness hardening: failing cases preserve their serial logs; the heavy
scenarios run at 150s.
The manager matches the pci_host_bridge node and spawns pci-bus with the
bridge id as its assignment — hello, supervision, restart, all the M18
contract for free. The driver claims the bridge, maps the ECAM window
(resource 0) through the ordinary mmio grant, and repeats the kernel's
brute-force bus/device/function walk from user space. The pci-scan
scenario builds its expected marker from the kernel's own function count,
so the two enumerations must agree exactly — the equivalence that
licenses retiring the kernel walk in M19.3.
The host bridge now carries MMIO apertures derived from the boot memory
map's gaps below 4 GiB (largest three, sort-merged; a single after-the-
last-region hole dies on OVMF's flash at the top) plus one aperture above
the described space — so a user-space device_register of PCI functions
with BAR resources can pass containment. The discovery test asserts
every PCI memory resource lies inside a bridge window and names any
escapee. device_register is idempotent on exact (parent, class, identity,
resources) match — a restarted registering bus cannot duplicate its
children; proven directly against the broker in the bus test.
ChildAdded gains device_id so a report can carry the registered kernel
id a matched driver needs as its assignment.
Applications ask the device manager for the tree (enumerate: a header
plus ChildEntry records) and subscribe to published add/remove events by
handing their endpoint over as the call's capability — the input-service
pattern; events are the same ChildAdded/ChildRemoved structs the bus
drivers send, one encoding in both directions. device-list is the first
client: it prints the tree, subscribes, and narrates the events through
a driver restart. The protocol's message maximum is capped at the
kernel's IPC MESSAGE_MAXIMUM (256 bytes, ten entries per reply; paging
joins the protocol when a tree outgrows one message). The startUserTask
debug print is gone: it wrote to serial unserialized against user-space
lines and sheared concurrent log markers in half — the root cause of the
scenario flakes.
child_added/child_removed join the device-manager protocol. The driver
maps its register BAR (resource 0 is the ECAM config space; the walk
starts at 1), reads CAPLENGTH and HCSPARAMS1, and reads one PORTSC per
port: the connect bit and speed class come straight from hardware, no
rings needed to see the devices. The manager mirrors reported children
keyed by (parent, port), remembers which instance reported each, and
prunes a dead reporter's children before deciding the restart — the
children describe protocol state that died with the process. The
usb-report scenario drives the whole loop: two QEMU devices reported,
reporter killed, children pruned, driver respawned with backoff, and the
new instance re-claims, re-scans, and re-reports.
The manager is now a harness service on the well-known .device_manager
endpoint. Every driver spawns supervised; drivers with an assignment must
hello (device-manager-protocol, versioned) within a deadline enforced by
a timer sweep. Exit reasons drive the restart decision: clean exits stay
down, faults restart with 300/600/1200ms backoff, and three fast deaths
mark a driver failed instead of respawning forever. usb-xhci-bus is the
first conforming driver; the crash-test fixture claims a device, hellos,
and faults on purpose — each respawn re-proving claim release on death
through the manager's own path. maximum_tasks grows 16 -> 32: the
initial-ramdisk sweep (15 binaries at once) was intermittently
overflowing the static pool.
Turn acpi-ids.zig's flat name table into a HardwareId enum modeled on
ps2-library's Port: one entry() switch holds the registry (variant ->
_HID string + human-readable name), with hid(), description(), and
fromHid() methods. The free description(hid) lookup the kernel's
device-tree dump uses survives, implemented over the enum, and a new
test round-trips every variant through fromHid.
Callers now name the device instead of quoting its id:
- ps2-library's DeviceType.hid() and ps2-bus's descriptor lookups use
HardwareId.ps2_keyboard / .ps2_mouse.
- device-manager's driverFor parses the HID once with fromHid and
switches on named values.
- acpi.zig's isPciRootNode carried the same ids twice, as strings and
as packed-EISA integers (0x030AD041/0x080AD041); both branches now
decode to the string form and answer through one isPciRootHid helper
using .pci_bus / .pci_express_root_bridge.
- build.zig threads the acpi-ids module (previously kernel-only) into
every user binary, like xkeyboard-config.
process_enumerate snapshots the task table (the device_enumerate shape, so
ps is a user program); system_spawn returns the child id, records the caller
as supervisor, and takes an exit endpoint; process_kill is allowed only for
the supervisor. Every death — exit, fault, or kill — posts a child-exit badge
to that endpoint (the IRQ-as-IPC pattern as SIGCHLD). A target caught off-CPU
is reaped in place; a running one is condemned and finished at its next
system call or tick, guarded so teardown never lands mid-kernel-operation.
Tested by process-list, process-kill, and supervision (a ring-3 supervisor
exercising the whole surface); design notes in docs/process-management.md.
Add a `system_spawn(name)` system call: the kernel loads a binary bundled in the
initial-ramdisk, by name, as a fresh ring-3 process. It's the mechanism a user-space
supervisor needs — discovery and policy stay in user space, the kernel only spawns.
The kernel already holds the initial-ramdisk image from the boot handoff; it now
stashes it (process.setInitialRamdisk) so the handler can resolve names, bounds-checks
the name into the user half like debug_write, and returns -1 for an unknown name or a
load failure. Ungated for now (any process may spawn any bundled binary); a spawn
capability belongs here once the model grows one.
The device manager stops logging "would spawn it" and calls runtime.system.spawn on
its matched driver. On QEMU it discovers the HPET, matches `hpet`, and spawns it — and
the driver comes all the way up (claims the timer, maps its MMIO, binds and services
its IRQ, prints "hpet: ok"). The device-manager test now keys on that final marker:
since only the manager is spawned, `hpet: ok` appearing proves the whole
discover -> match -> system_spawn -> driver-up chain end to end.
Transitional: the kernel still auto-spawns the whole initial-ramdisk at boot, so a
real boot briefly double-spawns hpet (the second claim fails harmlessly). Increment 3
removes that redundancy so the manager is the sole owner of driver spawning. Suite
36/36 plus host tests.
The device manager is the ring-3 process that turns the device tree into a running
system — the udev-analog. It is mechanism-vs-policy done right: the kernel
enumerates the hardware and enforces the claim capability; this decides which
driver serves which device, using no special privilege (the same device_enumerate
any process could call).
This first increment does the discovery + matching half: system/services/
device-manager enumerates /system/devices, matches each device to a driver by
class (a small static policy table), and logs the decision — finding the HPET
(a timer) and deciding `hpet` serves it. It does not spawn yet: spawning needs a
`system_spawn` system call (the kernel spawns every initial-ramdisk binary in a
loop today), which is the next increment. New `device-manager` test; suite 36/36
plus host tests.
Daniel Samson
danielandClaude Opus 4.8