Inline V4 review (the boundary-review workflow stalled): the poll ran a due
fat-restart before the presence check and returned, so a fat death followed
by a device removal would respawn fat against the now-dead channel and churn
until the crash cap before the removal was noticed. Reorder: check the
specific device's presence first (unmount if gone), and only fire a due
restart once the device is confirmed present. Neutral: fat-mount,
volume-removal, amd-iommu-usb-storage green.
Noted V4 limitations (not fixed here, edge cases outside the user unplug
case): a usb-storage DRIVER crash (device stays, driver restarts with a new
endpoint) leaves fat holding a dead channel — the device is still present so
removal is not detected; fat would need to observe its channel death and
exit. Deferred with the medium_changed subscription and multi-volume.
The volume manager, per-sender range confinement + gate, medium_changed
event, per-volume spawning + supervision, ownership-gated fs_unmount, and
the removal half of the lifecycle are built. Left honestly pending: the
volumes.csv/filesystems.csv maps, the fuller identity ladder, multi-volume,
the VM consuming medium_changed (removal uses device-presence polling), and
remount-on-replug end-to-end (bench-pending — QEMU can't re-present the
boot-controller device). Full suite 127/127.
The full suite caught a V4 regression: under AMD-Vi the device-manager tree
carries more than one mass-storage-identity entry (a phantom no driver is
bound to, which answers a consumer hello with NO channel). V4 split presence
from acquisition and picked the FIRST identity match blindly, so it kept
helloing the phantom (device 27) and never reached the real storage (device
31). V3's inline loop had skipped no-channel entries with `orelse continue`;
the split lost that.
Restore it: openAnyStorage tries each matching entry and takes the first whose
channel opens, recording its device id. Removal detection then watches THAT
specific device id leave the tree (isDevicePresent), not "any mass-storage" —
so a phantom that never leaves cannot mask a real removal. Both are bare
enumerates; the hello only happens while bringing a volume up.
Green: amd-iommu-usb-storage, fat-mount, volume-removal.
The volume manager stops probing-once and polls storage presence for the life
of the boot: findStorageDevice enumerates the device-manager tree (presence
only, no consumer-hello, so it is cheap and leaks nothing). The volume is now
a field that goes null and back — the whole lifecycle:
- storage present + no volume -> open the channel, probe, confine + spawn the
filesystem (openStorage is the one consumer-hello, on the insertion edge);
- storage gone + have volume -> kill the filesystem (its mounts retire via
the kernel dead-backend sweep), close the dead channel, clear the volume;
- fat crash -> the same supervised backoff/cap as before,
folded into the poll (one timer).
This also subsumes the V3-review leak fix (no per-poll consumer-hello) and the
no-volume retry (a present-but-unreadable device keeps polling).
The user's case — pull the boot stick, plug it back — is a DEVICE unplug (the
stick IS the device), so the mass-storage child leaves the device-manager tree
and the poll catches it. volume-removal asserts the unmount and discriminates:
against the V3 probe-once volume manager the removal is never noticed (0/1).
The re-mount on replug is the VM's bringUpVolume firing when the device
returns — correct and in place, but not QEMU-testable here: device_add of
usb-storage to the boot xHCI controller is not re-presented to the guest (no
port-connect on any port), a harness quirk, not a VM issue. On real hardware
the bus's per-tick port poll catches a reconnect (H1 proves reconnect on a
second controller); bench-verify the full round trip.
The mount-failure diagnostic wrote through a fixed [96]u8 bufPrint buffer,
which the bounds gate flags as an undeclared ceiling. Three ring appends
instead — no buffer, no fixed length to justify. This fix was made during
the V2a bounds work but never git-added, so the committed harness still
carried the flagged buffer; committing it now cleans the tip.
Five confirmed defects from the boundary review:
1. (security) The VM never checked a partition fit inside the device, so a
crafted MBR could hand the driver a range whose base+lba wraps past a u32
— panicking usb-storage in a loop, and at multi-volume overlapping a
neighbour. This is the exact invariant the clamp's overflow-safety rests
on. partition.firstVolume now skips any entry that runs past the device
(host-tested), establishing the invariant where the untrusted bytes are
first read.
2. (leak) The probe re-acquired a fresh block channel on every 500 ms retry,
leaking a handle each time on a medium-absent device. The channel is now
acquired once and kept.
3. (wedge) A failed spawn or defineRange stranded the volume with no retry;
both now arm a backoff restart.
4. (loop) fat respawn had no exit-reason gate, no backoff, no crash-loop cap
— a faulting filesystem respawned in a zero-delay loop, and a clean exit
was resurrected. Supervision now mirrors the device manager: a clean exit
is not restarted, a fault backs off, three fast deaths give up.
5. (removable) A device that parsed to no volume was terminal; it now keeps
polling so an inserted medium is picked up — the removal-lifecycle trigger.
Known limitation (noted, not fixed here): if the VM itself crashes and init
restarts it, the orphaned fat keeps serving vfs while the new VM spawns a
second fat whose bind is refused — the same "manager restart re-learns the
world" gap the device manager also defers. The old fat keeps storage working.
Neutral: partition unit tests + fat-mount, volume-probe, block-range, logger
all green.
The load-bearing step. The FAT service stops acquiring its own volume: the
volume manager spawns it (per volume), defines its partition range on the
storage driver BEFORE it runs, and answers its startup hello with the
range-confined block channel over a new volume-manager protocol. fat never
finds its storage by name and never sees the whole device — establishment
by lineage, one layer up from the driver tree.
- New library/protocol/volume-manager: one verb, hello(volume-id) -> the
block channel as the reply capability (the P0 reply-cap path).
- The volume manager becomes the confinement CONTROLLER: it defines the first
range on usb-storage, so no other party can confine a filesystem. It
supervises the filesystems it spawns and respawns one on death (the reap-
and-rebuild the device manager proved, one layer up).
- fat: drops acquireVolume(device-manager); hellos the volume manager for its
channel; reads its volume id from argv[1]. main takes process.Init now.
- init.csv no longer spawns fat (the volume manager does); protocol.csv
rewires fat to be supervised by the volume manager (bind vfs, open
volume-manager) and drops fat open device-manager.
- The block-range fixture boots registry + device-manager only (not the full
tree), so the volume manager is absent and the fixture stays the sole
confinement definer — otherwise the volume manager would take the
controller first and refuse it.
Verified end to end (VM probes -> spawns fat -> confines it -> hands over the
channel -> fat mounts) and neutral: 18/18 across the fat family, logging,
shutdown, both IOMMU variants, usb restart, vfs, conformance, confinement.
The storage layer gains its policy home (storage-architecture.md): a new
system/services/volume-manager, spawned by init, that acquires the mass-
storage block channel through the device manager (the same lineage a
filesystem uses), reads block 0, and parses the first volume out of it. The
partition-table walk that lived in the FAT engine moves here, above the
driver where it belongs (partition.zig, host-tested: MBR entry, bare-FAT,
no-signature). Identity is the MBR disk signature + partition index — the
weak rung of the ladder; GPT GUID and FAT serial refine identityOf without
changing shape.
This increment is discovery + probe + log only, additive: the FAT service
still acquires its own volume, so nothing changes for it. Confining each
filesystem to its partition and spawning one per volume (the flip) lands
next, keeping fat working throughout.
Grants + wiring: init.csv spawns it after the device manager; protocol.csv
grants bind volume-manager + open device-manager. Verified: volume-probe
asserts the parse (bare-FAT volume at lba 0), neutral 10/10 across storage,
restart, display, logging, confinement — the volume manager now runs in
every boot and disturbs nothing.
Two defects the V2 boundary review confirmed:
1. The per-badge range table was never reclaimed. usb-storage receives exit
notifications (via the subscriber watch), but onNotification handled only
the timer and dropped child-exits, so a dead filesystem left its range
slot used forever. The medium-removal lifecycle churns filesystems, so
after maximum_ranges confine/die cycles define_range would return ENOSPC
and no volume could be confined again until reboot. onNotification now
frees the dead badge's slot, mirroring fat's open-node sweep.
2. define_range checked only that the CALLER was unconfined, never that it
owned the target badge — so any unconfined opener could install a range
for another live client and silently redirect its I/O. Confinement now has
a single controller: the first unconfined party to define a range (the
volume manager, which confines every filesystem before handing it a
channel). Only the controller may thereafter; the slot releases on its
death so a restarted manager re-takes it. This is the mechanism half; V3
adds the grant half (only the volume manager gets an unconfined channel).
Neutral: block-range (the fixture is the sole definer -> controller),
fat-mount, usb-report all green. End-to-end exercise of both lands in V3/V4
(the VM+filesystem relationship and the remount churn).
The naive bound `lba + count > r.count` wraps for an lba near u64 max: the
sum overflows to a small value, sails under the check, and `base + lba`
wraps to an absolute block OUTSIDE the range. Calibrated, it is a real
confinement escape — a process confined to [1,3) reads absolute block 0
(the boot sector) with lba = maxInt(u64), since base + lba wraps to 0.
The bound is rewritten as two subtractions that cannot overflow: lba within
the range, and count within what remains. block-range gains a wrap-refused
assertion calibrated to be exploitable against the naive form — it FAILS
against the old bound (reads block 0) and passes against the fix (verified
by reverting the clamp). Caught pre-emptively before the V2 boundary review.
The block protocol gains a pushed medium_changed event (present + a monotonic
change counter; presence only, never content). usb-storage becomes a
Subscribers provider and runs a slow TEST UNIT READY poll (1 s): success is
present, failure absent, and a transition bumps the counter and publishes.
This is the second removal trigger — the DEVICE stays while the MEDIUM leaves
(card readers, ATAPI trays) — which channel death cannot see
(storage-architecture.md, two triggers one lifecycle).
The subscriber is the volume manager (V3); until it exists the publish is a
no-op fan-out, so this commit is behaviour-neutral, and its end-to-end test
(eject -> medium_changed -> unmount/remount) lands in V4 with the real
consumer rather than a throwaway subscriber fixture (recorded sequencing).
Sense-key inspection to tell medium-absent from other transport errors is a
noted refinement; a clean eject reads correctly as not-ready.
Neutral: 12/12 across the block-serving surface, restart, confinement,
conformance, and logging.
A process acquires a block channel the way a filesystem does (consumer-hello
the device manager), confines ITSELF to blocks [1,3), then proves the clamp
and the gate: volume-relative LBA 0 maps inside the range and reads; a read
reaching past the range is refused; geometry reports the confined size; and
a confined caller can no longer call define_range (no widening, no escape).
It gates on argv so the ramdisk sweep leaves it silent in other boots, and
coexists with fat (ranges are per-badge).
Discrimination (verified by reverting usb-storage to pre-clamp f1bdce2~1):
the unconfined read still succeeds but define_range returns ENOSYS, so the
fixture cannot arm confinement and the case fails — exactly the property
the clamp adds. With the clamp: block-range 1/1.
The block protocol gains define_range (appended, numbers hold): confine the
process named by `badge` to blocks [base, base+count). usb-storage keeps a
per-badge range table and, in read/write, translates volume-relative LBAs
(base added) and refuses any transfer past the volume end. geometry returns
the confined size, so a filesystem mounts against what it may actually touch.
The security seam (decision 4, settled): the clamp lives at the PROVIDER, so
a channel carries exactly the authority it grants — handing a filesystem the
whole disk plus a base offset would let it reach the neighbouring partition.
The gate: a confined caller may NOT call define_range, so a filesystem cannot
widen its own range or confine anyone; only an unconfined party (the volume
manager, whole-device) may. The volume manager defines a filesystem's range
before handing it the channel, so the ordering holds by construction.
Default (no range for a badge) is the whole device — behaviour-neutral for a
single-volume boot and what the volume manager itself uses to probe
partitions. The range table is declared through bounds.md as a runaway
detector (ours, refuse at limit), not a real-partition cap. Neutral:
fat-mount, usb-storage, iommu-usb-storage green. The discrimination fixture
(a confined process reads past its range and is refused) follows next.
V1 boundary review (4 reviewers converged): the extraction routed mount-
FAILURE logs through std.log where old fat used logging.write. That matters
precisely when the failed mount IS /system/logs — a std.log record would
then have nowhere to land. Restore the direct kernel-ring write for the
failure branch (generic prefix, since the harness is filesystem-agnostic
now); success stays std.log.info as before. Unexercised error path; fat-mount
still green. Everything substantive in the extraction verified neutral.
fat was one binary doing four jobs; the three that are not FAT-specific move
to library/kernel/file-system-harness, a Server(comptime Engine) generic over
the engine type: the badge-scoped open-node table, the nine vfs handlers, the
not-mounted politeness, the exit sweep, mount registration, and durable-on-
close. A filesystem is now an engine plus a main that hands the harness a
mounted volume; a second engine reuses the harness wholesale.
Placement note: the plan said library/file-system, but the harness is a
specialization of `service` (its sibling) and needs nothing from the device
domain, so it lives beside service in library/kernel and stays block-free —
durability rides a caller closure (Volume.flush), no backwards kernel->device
dependency, no new-domain scaffolding. The engine type is inferred from
resolve()'s return, so engine.zig is untouched (its Node stays module-scope).
fat keeps only its FAT-specific bring-up (acquireVolume, DMA, engine.mount,
the attach/detach round trip) and the three mount prefixes as data. Behavior-
neutral: 13/13 across the fat/vfs/logger/IOMMU surface, nothing observable
changed. This lands first so every later phase touches the harness once.
Per-sender range confinement at the provider, one serving endpoint: the
badge-scoped provider pattern the xHCI bus already uses (the per-client
device-token table), applied to blocks. Endpoint-per-volume would buy the
same enforced property only by inventing a multi-endpoint harness; it stays
available as a future refactor, same wire contract. The plan's flag-for-veto
is gone: nothing in the track waits on a choice.
fs_unmount was gated by nothing but the /protocol carve-out: any process
could unmount any prefix — latent with one mount owner, an obvious
cross-tenant hole once volumes multiply. Each backend mount now records the
mounting task, and the syscall layer enforces two rules that keep the
restart story intact: only the owner unmounts (a dead owner's mount is
swept lazily by resolution — strangers gain nothing by racing that), and a
mount may be REPLACED only by its live owner or after its owner died (the
respawned-filesystem path; displacement of a live mount would be worse than
unmounting it). Kernel-installed mounts are never displaceable.
The vfs-test park role is the discrimination: with the volume provably
mounted it attempts the foreign unmount, requires the refusal AND the
subtree still resolving, and withholds its "parked" marker otherwise —
against the ungated kernel the unmount was ALLOWED and vfs-client-death
fails; with the gate, green. (Its verification handle closes immediately:
the kernel test string-matches "released 1 handle(s)".)
The sharpest instance of the return story, folded into the architecture: the
boot volume is a recorded content identity (the volume carrying
/system/configuration and /system/logs), the system runs on without it (the
ramdisk is the OS; only log persistence pauses), the kernel ring is the
buffer during absence — bounded, so a wrapped ring is a data loss window
that gets MARKED in the file on resume, never spliced silently — and on
return at any port the same identity remounts the same prefixes and the
logger appends into the same boot-stamp tree. The logger requirement is
named: failed flushes retry on the patient cadence, never abandoned after
the first not_found. The dirty-honesty rule stands; the boot volume gets no
exemption.
Decision 8 in the rationale, mirrored into the architecture's volume-manager
section: the mount map keys on CONTENT identity, never port or arrival order
— Linux's /dev/sda1-era fstab broke on every port move until UUID= replaced
it, and danos skips that era. The identity ladder the prober reads off the
medium: GPT partition GUID, filesystem UUID, FAT serial+label, MBR
signature+index, anonymous. Consequences are mechanical: port moves change
nothing (USB, hub, SATA bay, or transport swaps), replug remounts at the
same path, the boot volume is a recorded identity findable anywhere, and
cloned duplicates are a loud policy case instead of silent shadowing.
/volumes/<name> stands as the hierarchy's home for attached media; minting
identifiers (formatting, entropy) stays deliberately out of scope.
storage-stack-discussion.md was misplaced at the docs root — that level is
for track plans; this is the file-system domain's design record. Moved to
file-system-development/storage-design-rationale.md, renamed to say what it
is, cross-references updated.
The NVMe/SATA assessment folded into the storage discussion: what transfers
untouched (the block contract and everything above it — one driver binary
plus one devices.csv row per transport), NVMe as the shorter stack whose
namespaces are the reserved multi-volume case, AHCI as an open shape choice
(leaning per-port processes, the matrix-proven granularity), and the three
pressure points named honestly: multi-volume is reserved-not-implemented,
synchronous call/reply bottlenecks NVMe until the shm-ring data plane, and
media lifecycle is not device lifecycle.
That last one becomes decision 7 and enters the architecture doc: the
removal path has TWO TRIGGERS, ONE LIFECYCLE — channel death (device leaves)
and a planned pushed medium_changed event (medium leaves, device stays: card
readers and trays, USB ones today), translated by the storage driver from
its transport's native signal, presence never content, consumed by the
volume manager into the same kill-retire-remount path. Without it a swapped
card would be served with the previous card's filesystem state.
The kernel was always symmetric (dma_bind 51 / dma_unbind 52); the two
protocols that forward an attachment up the stack were one-way, so a live
client could grant a device reach into its buffer but never revoke it while
alive — exactly the one-way lifecycle the storage architecture's enforcement
section forbids. Death stays the mechanical backstop; detach is the living
process's path.
Both verbs are appended, so every existing number holds. The shape mirrors
attach precisely: the same region capability rides the cap slot again — the
kernel matches the region, so no layer retains anything between the calls
(the bus never kept the handle; now it never needs to).
fat's bring-up does attach -> detach -> attach, exercising both verbs
through the whole chain (fat -> storage -> bus -> kernel) on every boot: a
broken detach fails every fat case instead of lying dormant until the first
buffer replacement. Honest scope: the round trip proves the plumbing; unbind
semantics are the kernel iommu tests' (map/unmap/translationOf); the full
composition (detach then DMA faults) is a future iommu-fault extension.
The enforcement section of the storage architecture: the three levers the
device lifecycle already proved, mapped onto volumes — a filesystem can only
be GIVEN its volume (no establishment grants, channel at spawn), the volume
manager supervises with teeth (deadline, kill-on-removal, crash-loop cap),
and the shared harness makes every engine inherit the state machine by
construction (engines never see channels). Kernel backstops: ownership-gated
fs_unmount + the lazy dead-endpoint sweep. Checkable via a lifecycle
conformance drill parameterized over filesystems — supporting a filesystem
MEANS passing it.
The settled shape from the storage-stack discussion, written as the
reference: the data path (vfs -> filesystem service -> block -> driver) as
the application/service/protocol/driver model applied twice; the three kinds
of boundary (protocol between processes, library inside them, control-plane
beside them); the volume manager as the policy home (planned — the FAT
service squats on its duties today, marked as such); adding a filesystem as
engine + shared harness + one configuration row; and the per-layer
responsibility table for removable media — one removal path, kill/retire/
respawn, dirty data lost and SAID to be lost. Indexed from docs/README.md.
The hot-plug matrix's last two cells. H4: unplug from hub port 1.1, replug
on 1.2 — per-port identity means the old child is removed and reaped while
the new port binds a fresh driver; nothing ties a driver to the old port.
H5: three unplug/replug cycles on one port — three reaps, a bind after the
last, proving slots, the bus open table, and the manager's driver entries
are all reusable across generations.
hub -> hub -> keyboard, one device_del of the outer hub: the H2 recursion
runs at depth two (the inner hub is torn down as a child, its keyboard
first), the keyboard's driver is reaped, and re-adding all three rebinds.
The hot-plug matrix's predicted bug, found on first contact: tearDownPort
never recursed into a departing hub's children — only tearDownHubDevice
(a hub leaving one level down) did. Yank a populated hub from a root port
and the downstream slots stayed live against vanished hardware, their class
drivers were never reaped, and the replugged hub found its port still
occupied, so nothing ever re-enumerated: the subtree was gone for the boot.
tearDownPort now recurses children-first, exactly like tearDownHubDevice.
The usb-hub-yank case is the discrimination: one device_del removes a hub
carrying a keyboard AND a mouse, both drivers must be reaped, and the
re-added hub must rebind both — it failed against the unfixed bus and
passes with the recursion.
The tearDownPort path, exercised for the first time with a real removal and
return: QEMU raises no root-port change events, so the bus's 250 ms
reconcile tick notices the PORTSC change alone — and it does. Reap, re-add,
rebind, second generation binds on the same port. Discrimination: against
the pre-reap manager (4a2df58~1) the case fails at the dedupe wall.
The hot-unplug path (onChildRemoved, a report from a live bus) cleared the
child but left the bound class driver: a process blocked on reports that
will never come, whose stale entry made the matcher's dedupe refuse the
respawn when the device was plugged back in — the same wall the restart
zombie hit, one path over. Unplug now reaps exactly like reporter death.
The usb-hub-unplug case grows the replug: device_del the hub keyboard, then
device_add it back (qmp_sequence); the ordered tail — child removed, reaping,
delegated, ok — can only be satisfied by the second generation, since every
boot keyboard's ok precedes the unplug. Discrimination: without the reap the
replug never rebinds and the case times out (verified by stash run). Hub
family, restart drill, and the two-controller proof all green (8/8).
P4 of docs/establishment-planes-plan.md. The new usb-two-controllers case is
the Ryzen mouse bug pinned in the suite: a second xHCI controller with its
own keyboard while the boot controller keeps the default one — both must
come up, on different device ids, which requires each class driver to reach
ITS OWN controller. Discrimination: at 72807c2 (name-based establishment)
the case fails — one keyboard is unreachable, exactly the bench failure —
verified against a checkout of that merge; with lineage routing it passes.
The existing second-controller cases could not prove this: the boot bus
always carries a keyboard, so their expects were satisfiable by it.
Docs updated with the code: device-manager.md (hello moves the channels,
paged enumerate, delegation as built, reap-and-rebuild in the restart
sequence), device-authority.md (the fourth as-built decision: driver-layer
channels ride the hello; hello is no longer only the liveness handshake).
Full suite: 118/119, the one failure being iommu-fault's fixed 200 ms
fault window under end-of-run host load — 3/3 green standalone, deflake
flagged separately. usb-two-controllers passed inside the full run.
P3 of docs/establishment-planes-plan.md — the restart-zombie fix. A class
driver cannot observe its provider's death: an HID driver blocks on
interrupt reports that will simply never come, and storage answers its
callers with refusals forever. Worse, the dead generation's still-used
entries made the matcher's dedupe refuse the respawn when the restarted bus
re-reported — the subtree was a permanent zombie, which is the exact
opposite of the restart-a-driver-live goal the driver model exists for.
pruneChildrenOf now reaps: each pruned child's bound driver is killed and
its entry cleared (the exit notification finds no entry, so the death is
never double-counted; its device returns by the loan rule; its stored
endpoint handle is closed). The re-report then spawns a fresh generation
whose hellos fetch the successor's channel.
The usb-report drill now asserts the subtree WORKS after the restart: the
respawned storage opens its device on the NEW bus instance and reads block
0. Discrimination: against the pre-reap manager the drill fails — no reap
line, no post-restart respawn (the survivors were zombies), verified by a
stash run. Note the scenario boots no input service, so the HID drivers of
BOTH generations exit after their input lookup times out — storage is the
functional proof.
P2 of docs/establishment-planes-plan.md. usb-storage serves nameless — one
process per stick cannot share an exclusive bind, and a second stick used to
die silently on -EBUSY before ever helloing. Its one hello now moves both
directions at once: the block-serving endpoint up, its controller channel
down. fat finds its volume through the manager — a new `.consumer` role asks
for the channel of the driver BOUND TO a device (distinct from the device's
reporter), found by enumerating the tree for the mass-storage identity.
fat stays single-volume; the boot-volume-by-content choice is M21.
The conversion immediately caught a live truncation of exactly the audit's
shape: ChildEntry grew to 32 bytes, one enumerate reply holds ~7, and a real
tree carries a dozen ACPI nodes before the first USB child — the storage
entry silently never fit (the protocol comment already said "paging joins
the protocol if a tree ever outgrows one packet"). enumerate is now paged:
Header.target is the start cursor, a short page is the end; device-list's
page-0 read is unchanged.
Grant rows move with the code: the block bind and fat's block open die, fat
gains open device-manager. Gate: 18 cases green including the registry
trio, device-list, and both IOMMU storage variants.
P1 of docs/establishment-planes-plan.md. The bus no longer binds
/protocol/usb-transfer — the bind race made whichever instance came second
unreachable, which on a real three-controller Ryzen meant a mouse no class
driver could reach ("could not open device 50"). Each instance hands its
serving endpoint up in the hello that already delegates its controller, and
class drivers receive their OWN controller's channel from helloForChannel —
routed by the manager's lineage, retried while a provider is mid-restart,
on one manager handle so retries never spend handle-table slots.
usb.open(bus, id) now takes the channel it used to look up; the name rows
leave protocol.csv with the code (bind 67, opens 119/120/122); the
conformance fixture's prose stops claiming the bus binds; and the stale
input-client import leaves the bus with the channel one.
Gate: 19 QEMU cases green (usb family, hubs, both IOMMU variants, fat chain,
boot-from-USB, orderly shutdown, conformance).
P0 of docs/establishment-planes-plan.md; no behavior changes yet, nothing
sets the new flag or sends a hello capability.
- service.run gains a reply-capability out-slot (replyWithCapability), the
registry idiom init already uses, lifted into the harness; null stays the
untouched common path. Subscribers gains claimArrival() so a provider
handler can keep a turn capability through the same flag the reserved
subscribe uses.
- Hello wire struct: the padding byte becomes wants_channel — old callers
wire-compatibly say 0, and the manager nominates a reply capability ONLY
when asked, because a capability sent to a caller that never reads one is
a leaked slot in that caller's table.
- driver.helloExchange: one handshake can hand a serving endpoint up and
receive the device's provider channel down (usb-storage will need both at
once). No channel in the reply is retryable, never a verdict.
- The manager stores each instance's serving endpoint on its Driver entry,
routes consumer hellos by lineage (child -> reporter -> endpoint), replaces
on re-hello, and closes the stale handle on death - the 32-slot table is
the bound that makes forgetting this boot-fatal.
The namespace holds protocols, never instances; what multiplies is provider
processes, and their establishment routes through the device manager, which
owns the topology. communication.md gets the model; the plan converts the
usb-transfer and block seams in one flag-day, closes the restart-zombie hole
the scoping found, and pins the three-controller mouse bug as a QEMU case.
Two command encodings checked against the specification:
- COMPLETION_WAIT set bit 1 (I, interrupt) instead of bit 0 (S, store), so the
IOMMU was never asked to write the sentinel, the poll always exhausted its
spins, and completeAndWait returned without any guarantee the preceding
invalidation had executed — no invalidation barrier has ever existed, on QEMU
or on silicon. The in-code claim that "QEMU's amd-iommu does not implement
the store form" was a misdiagnosis of this bug: with S set, QEMU stores the
sentinel fine, and the amd-iommu cases now run without the warn line. On real
hardware, which fetches commands asynchronously, the missing barrier was an
IOTLB use-after-free window: unmap returned before the invalidation was
confirmed and the caller freed the frames.
- INVALIDATE_IOMMU_PAGES "invalidate everything" used address bits 51:12
all-ones; the architected encoding is bits 62:12 all-ones (the spec's literal
0x7FFF_FFFF_FFFF_F000). Real silicon is free to misread the non-architected
form as a bounded range.
Three holes from the real-AMD audit, one shared root: the delegation flag-day
added paths that touch the broker table and the IOMMU records without the big
kernel lock, and a loan-return rule whose re-delegation skipped confinement.
- device_enumerate walked the table with no lock. The table stopped being a
static array in the bounds track — reserve() regrows it through realloc on
every boot — so an unlocked reader can be mid-copy out of a slice that
device_register on another core has already freed and reused, or pair a fresh
count with a stale slice. Each chunk is now snapshotted under the lock; the
copy to the user stays outside it.
- system_spawn's give ran entirely unlocked — the comment claiming "the lock
has not been dropped" was false (spawnProcessSupervised takes and releases it
internally). The ownership pre-check now only spares creating a doomed child;
the give itself re-checks, confines and moves in one lock hold, and a give
that fails after the spawn kills the child rather than leaving it running
without the hardware it was spawned for.
- A re-delegated device after a driver death was never re-confined. Death tears
the domain down before the loan returns to the lender, so the next give found
no active record, reassign no-op'd, and the respawned driver ran the device
with a V=0 device-table entry and no domain — silently unconfined, the exact
fail-open the fail-closed claim was built to remove. Both give paths now share
one body (giveDeviceLocked): check first, confine afresh when no record is
active — refusing with ECONFINE like the claim — and move last, when nothing
can fail.
The iommu test drives the death-and-respawn sequence directly; with the old
reassign-only behaviour its two confinement checks fail, with this change the
suite is 118/118.
An AMD Ryzen booted to a working compositor with no USB and no storage. The
cause was maximum_children_per_parent = 16 — a number nobody had justified,
in a kernel where 235 compile-time ceilings turned out to exist, 139 of them
on quantities the machine or a file decides rather than us, and 171 silent
when reached.
Both invented ceilings are deleted rather than raised. The device table
grows, because no specification bounds how many devices a machine has, and a
runaway is bounded per-registrar so it costs only the driver that caused it.
Device authority became real rather than advisory. Matching was always
policy, but claiming was first-come, so any process could take any device
nobody happened to be holding. Now every driver receives its hardware in the
spawn that creates it — it never runs without it — a grant is a loan that
returns to its lender when the borrower dies, and nothing firmware-discovered
is left unheld except the framebuffer, which the compositor owns.
Along the way: the AMD boot fix itself (STAR's SYSRET base carries the RPL,
which Intel forgives and AMD does not); one errno space with every refusal
naming the rule that refused it; the IOMMU confinement moving with its
device, and refusing rather than silently succeeding at the edge of its
table; and PCI apertures derived without copying the firmware memory map
into a fixed array, which had let a large map turn occupied RAM into a
window a driver could map.
A gate now stands behind the audit: zig build bounds refuses a new ceiling
that will not say what it counts, who decides its size, what happens when it
is reached, and how anyone finds out. The 269 that predate the rule are
allowlisted and that list can only shrink.
Suite 114 -> 118, green at every step.
Docs: fixed-bounds-audit.md, os-development/bounds.md, bounds-track-plan.md,
os-development/device-authority.md.
The name list and its predicate existed so drivers could move to delegation
one at a time with the suite green throughout. Every driver is delegated
now, so the manager simply hands over whatever device a driver was assigned.
Deleting it caught a real consequence: crash-test finally got delegated too,
and it was still claiming its device — so it got AlreadyClaimed because it
already held it, exited, and the restart drill had nothing to restart. Its
own comment named what the case was really checking: "the respawn only
reaches this line because the kernel released the previous instance's claim
at death". That property still holds, by a different mechanism — the device
reverts to the manager on death and is handed to the replacement, which is
the same guarantee without the race it used to rely on.
All four delegation paths verified: the xHCI controller, the PCI bridge, the
PS/2 two-node singleton, and virtio-gpu's restart re-attach.
Run 3 complete. Suite 118/118.
A device nobody holds can be claimed by anyone, so the manager now takes
every firmware-discovered device that carries mappable resources, whether or
not a driver wants it. The real gap was the HPET: an MMIO window, an IRQ, no
user-space driver, and there for the taking. Held by the manager it is
inert; unheld it was a way into physical memory.
Two deliberate exclusions. The loader's framebuffer, which the compositor
claims and which the manager must not take because it starts first. And
anything with no resources, which grants nothing worth holding.
Scope is the boot snapshot. A device reported later and matched to no driver
stays claimable — pci-cap-test and iommu-fault-test both reach an unmatched
NIC that way, so narrowing it is a separate change with those fixtures in
scope. Recorded in the plan rather than left implied.
The attacker fixture gains the assertion deferred since D2: after the system
settles, nothing with resources may be taken.
That assertion defeated itself twice before it worked, and both failures are
worth remembering. First it swept at 0.029 while the manager did not bind
its protocol until 0.047, so it reported a hole that closed a millisecond
later. The retry loop that "fixed" that was worse: the first pass TAKES the
device, so the second finds it unavailable because this process now holds
it, and concludes all is well — it passed with the manager's claiming
removed entirely. It now settles once and sweeps once, and fails when the
claiming is removed.
Suite 118/118.
The rule as planned: a device that was given to someone may be handed on,
never taken. Implemented, and honest about what it is worth.
Writing the test showed the plan had the wrong step doing the work. A
delegated device is HELD, so an attempt to take it is refused as
AlreadyClaimed before the giver is ever consulted; and once a borrower's
death returns the device to its lender — or clears both when the lender is
gone — there is no state where a device is unheld and still on loan. The
window a stranger could have used stops existing at E2. This check is
unreachable.
It stays anyway: one comparison, failing closed, guarding any future path
that frees a device without clearing its giver, which is exactly the hole
this run closed. The comment says it is unreachable rather than implying a
protection it does not provide.
The attacker fixture does not gain the assertion that was deferred to this
step, and its header records why: there is no refusal for it to observe, and
on a bare boot with no device manager nothing is delegated at all, so the
assertion had nothing to bite on. It failed loudly on its first run rather
than passing quietly, which is the only reason this was noticed.
It also leaves the loader's framebuffer alone without naming it: nobody
delegates the framebuffer, so it has no giver, so the display service claims
it exactly as before.
Suite 118/118.
When a driver dies, a device it was *given* now goes back to whoever lent
it, rather than to nobody. The device manager gets its hardware back the
instant a driver dies and hands it to the replacement, with no window in
between.
That window was real: the kernel released the claim to no one and the
manager re-claimed first-come, so every driver restart reopened the hole
this run is closing. It also becomes load-bearing at the next step — once
claim refuses a device that has a giver, releasing to nobody would strand a
dead driver's hardware permanently, because nobody could ever take it again.
A dead lender is no lender: the claim and the giver clear together, so a
device is never owed to a ghost. A device nobody lent is released outright,
exactly as before.
The broker cannot see the task table, so liveness arrives through the same
hook idiom the scheduler already uses. Null means assume dead, so a kernel
built without the hook frees claims rather than handing them to a ghost.
A stale binary nearly passed as proof for the third time this session: the
first discrimination patch left `alive` unused, the build failed with three
errors, and the old binary reported every assertion passing. Checking the
build before reading results is what caught it.
Suite 118/118.
Daniel Samson