/etc/init.csv and /etc/devices.csv become /system/configuration/*.csv (the
repo's etc/ moves to system/configuration/, mirroring the runtime tree),
/var/log becomes /system/logs, and /mnt/usb becomes /volumes/usb. The
kernel VFS gains a carve-out so FAT may serve exactly /system/configuration
and /system/logs beneath the initrd-backed /system while /system and /test
themselves stay unshadowable; FAT's single /var mount splits into those two
rewritten mounts. The kvfs readdir check learns /system's third child and
the ramdisk spawn sweep skips the configuration tree.
Suite 106/106.
The module-to-domain table in build-support duplicated what each
domain's build.zig already states with its addModule exports. userBinary
now resolves each named import by searching the packages the binary
declared in its own build.zig.zon (b.available_deps), which also makes
the zon the literal include path: an import can only be satisfied by a
domain the binary claims, and naming a module whose domain is missing
fails the build graph with the domain to declare. build-support is down
to the recipe alone. All build variants green; manifest unchanged.
display-client and input-client (files and module names), so a service,
its wire contract, and its client never share a name: `display` the
service, `display-protocol` the contract, `display-client` a program's
view of it. The nine consumers' imports and their packages' declared
lists follow (regenerated from the source scan); build-support's
module_homes table carries the new names.
Every binary's build.zig now names precisely the modules its source
imports (derived by scanning each artifact's sources, transitively
through same-directory files), and its zon carries only the domains
those come from — kernel stays implicit (the root shim + link script
live there). build-support's userBinary resolves each name through one
module-to-domain table (module_homes); Domains/domains()/defaultImports
and the raw recipe entry point are deleted. An undeclared @import is a
compile error (verified: injecting @import("xkeyboard-config") into
logger fails with 'no module named ... available within module
program'), and e.g. xkeyboard-config now appears in exactly two
manifests — the two keyboard drivers. Availability never bloated the
emitted binaries (Zig compiles only what a program imports); this makes
the declared interfaces honest. Production and -Dtest-case manifests
byte-identical; all build variants and standalone package builds
green.
init, fat, display, display-demo, device-manager, input, logger, and
the two discovery fillers (acpi, fdt — each exporting an artifact named
"discovery"; the root -Ddiscovery picks which ships) convert to binary
packages on the pci-bus template. init's serial heartbeat flag rides
the dependency options (the root forwards its -Dserial). fat's and
display's unit tests move into their packages and the root aggregate
delegates to them. Boot-image file list unchanged.
build-support gains the domains-based userBinary: the default import set
(the library/kernel concern modules, driver/service clients, mmio,
acpi-ids, xkeyboard-config) is assembled from the domain packages, and
the root shim + user link script come from the kernel package directory.
system/drivers/pci-bus is the first binary package: a ~15-line
declarative build.zig naming only its extras (device-manager-protocol,
pci-class); the root build consumes the artifact for the boot image and
the driver also builds standalone from its own directory. Boot-image
file list unchanged.
Replaces L2's interim DMA pool (every buffer reachable by every claimed
device) with true per-grant confinement: a device reaches only buffers
whose capability was delegated to its driver.
Kernel:
- DmaRegionObject (handle kind 2): a delegation token naming a dma_alloc'd
region, passable across processes on the IPC cap slot like an endpoint
or shared-memory object. Frames stay owned by the allocating address
space (freed on dma_free/teardown as before); the token carries a `dead`
flag so a stale downstream handle can no longer bind a freed region.
- dma_alloc gains the dma_shareable flag: it returns a capability handle
in r8 and every region is tracked in a registry. A task's own regions
auto-bind into the devices it claims (its rings just work); foreign
buffers are bound explicitly.
- dma_bind / dma_unbind / handle_close syscalls (51-53). dma_bind maps a
held region (or shared-memory) capability into a claimed device's domain;
it is idempotent. handle_close reclaims a table slot (raised 16 -> 32).
- dma_free and task death unmap a region from every domain and invalidate
BEFORE its frames return to the allocator — the stale-IOTLB use-after-
free window, closed structurally.
Protocols (flag-day): block gains attach, usb-transfer gains dma_attach —
each carries a region capability on the cap slot. fat allocates its bounce
buffer shareable and attaches it; usb-storage allocates its transport
buffers shareable, attaches them to the controller, and forwards fat's
capability downstream; usb-xhci-bus binds and closes; virtio-gpu binds its
shared scanout surface. The physical addresses on the wire are unchanged
(identity IOVA), so no register-programming code moved.
Cross-process DMA (fat -> usb-storage -> xHC) now flows only through
delegated capabilities. iommu-usb-storage / iommu-usb-hid / iommu-fault
all green under per-grant enforcement; 104/104 overall (fail-open paths
unchanged).
library/device/pci is now the complete generic floor a leaf PCI driver
needs, instead of just what virtio-gpu used:
- pci-class: capability IDs, MSI/MSI-X/power-management/PCI-Express
register layouts, extended-capability header decode (host-tested),
per-bit command constants, remaining header offsets.
- pci.Function: header accessors, disableBusMaster + interrupt-disable
helpers, findCapability, programMsi/disableMsi, MsiX vector-table
struct, ensurePowerStateD0, functionLevelReset (BAR save/restore),
extended-capability iterator.
Proven by the new pci-caps QEMU case: a pci-cap-test fixture claims an
extra e1000e (PM+MSI+PCIe+MSI-X, no danos driver) and readback-verifies
every surface, including the first driver-side use of msi_bind.
usb-xhci-bus converts from 8 ms event-ring polling to message-signalled
interrupts: plain MSI where offered (real Intel xHC), MSI-X entry 0
otherwise (qemu-xhci has no MSI capability), byte-identical polling as
fallback. The timer survives as a 250 ms port-reconcile/lost-edge tick —
real-hardware USB2 hub debounce still needs it. MSI setup runs BEFORE
controller bring-up: QEMU's xhci only registers the MSI-X vector as used
when IMAN.IE is written while MSI-X is already enabled; interrupts are
silently dropped otherwise (real hardware does not care about the order).
101/101 QEMU cases green; real-hardware smoke passed (mouse works,
boot 2026-07-23T174805Z, plain-MSI branch, vector 33).
Each bus's discovery line now prints the device's would-be /etc/devices.csv row
(bus, base, class, prog_if, vendor, device, subsystem / hid) in uppercase hex,
followed by the human-readable names — so a row for a new driver reads straight
off the boot log, for pci, usb, and acpi alike.
- pci-bus: logFunction moved to registerAndReport (where vendor/device/subsystem
are read from config space) and reformatted to columns + names; subsystem
prints '*' when the function has none. Read unclaimed via the bridge ECAM, so
no per-function claim is needed.
- usb-xhci-bus, acpi: the same column framing on their existing discovery lines.
- qemu_test.py: the acpi-ps2 and acpi-report regexes updated to the new ACPI
format (both verified passing in QEMU).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Replace the three hardcoded switch tables (pciDriverForIdentity, hidDriverFor,
usbDriverForIdentity) with an authoritative, human-readable device registry the
manager reads at boot. Matching is most-specific-wins across
base/subclass/prog_if/vendor/device/subsystem/hid, so a precise vendor:device
rule and a generic class rule coexist; an unmatched device is logged, never
guessed. This resolves docs' "matching stays code until the third bus".
- ABI: child_added and DeviceDescriptor gain vendor/device/subsystem; child_added
gains a bus discriminator (BusKind) so PCI and USB class triples match against
the right namespace.
- pci-bus reads vendor/device (config 0x00) and subsystem (0x2C, type-0) and
reports them.
- library/device/registry: freestanding CSV parser + matchDriver() with
specificity scoring; 5 unit tests wired into `zig build test`.
- etc/devices.csv bundled into the initrd; the kernel serves /etc directly, so
the manager reads it before any filesystem service is up (fat starts later).
- virtio-gpu: drop the now-redundant post-spawn 1AF4:1050 re-confirm, since the
registry binds this driver by exact identity.
- Remove the orphaned system/drivers/display driver (unreferenced by build or
registry).
- docs: new devices-csv.md; device-manager.md "matching stays code" resolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Every user binary and the two device-logic library modules (pci, usb) now
`@import` the concern modules directly instead of aliasing through `runtime`:
runtime.ipc/process/time/service/input/block/display -> @import("<module>")
runtime.device / runtime.device_manager -> @import("driver")
runtime.fs -> @import("file-system")
runtime.Thread -> @import("thread").Thread
runtime.system.{write,writeRecord,klog*} -> logging.*
runtime.system.{sleep,timerOnce,wallClock,clock} -> time.*
runtime.system.{spawn*,kill,exit,yield,processes,...}-> process.*
runtime.system.{mmap,munmap,PROT_*} -> memory.*
runtime.dma.* / runtime.shared_memory.* / runtime.allocator -> memory.*
Each consumer keeps its own alias name (e.g. `const device = @import("driver")`),
so call sites are unchanged and there are no collisions with local `driver`
variables. build.zig now injects the concern modules into every user binary via
`default_imports`; pci/usb module import lists were updated to match.
The `runtime` and `system` shims remain for one more step (root.zig still uses
runtime); they are deleted in C5. Nothing but root.zig imports `runtime` now.
Verified: zig build, zig build test, and 17 QEMU cases (smoke, device-manager,
logger, fat-mount, fat-mutations, usb-storage, usb-hid, display-native,
virtio-gpu, input, thread-spawn, thread-mutex, process-kill, shared-memory,
driver-restart, acpi-ps2, pci-scan).
mmio is device-driver code, so it joins the other domains under
library/device/mmio/ (module name "mmio" unchanged — a pure relocation, only
the build paths move). And its abbreviated function names are spelled out per
docs/coding-standards.md:
read -> readRegister mb -> memoryBarrier
write -> writeRegister rmb -> readMemoryBarrier
wmb -> writeMemoryBarrier
All call sites updated (virtio-gpu, usb-xhci-library, pci.Function); the two
display-driver placeholders import mmio but use nothing, so they're untouched.
Docs (driver-model graph, README layout, drivers.md, the FHS note) follow the
new path and names.
zig build + test green; virtio-gpu, display-native, display-reattach, usb-hid,
usb-hub, usb-storage, pci-scan pass.
Finish the direct-import convention: a protocol is now aliased to its module
name in snake_case in every file that imports one — consumers and the runtime
client wrappers alike. This kills the last of the alias variety (the generic
`protocol`, plus `power` and `transfer`), so `device_manager_protocol.Hello`
means the same thing everywhere and grepping a module name finds all its uses.
Pure rename (identifier uses only; prose references left untouched via a
guarded pattern). 14 files, 299/299 lines.
zig build + test green; 16 QEMU cases pass (smoke, device-manager,
driver-restart, device-list, pci-scan, usb-hid, usb-storage, display-service,
display-native, display-reattach, input, acpi-ps2, vfs, fat-mount,
power-button, orderly-shutdown).
display-protocol -> library/protocol/display/, scanout-protocol ->
library/protocol/scanout/. Consumers import both directly, killing the cryptic
dp/sp aliases in virtio-gpu (now display_protocol / scanout_protocol) and the
runtime.display_protocol / runtime.scanout_protocol re-exports. runtime.display
(the client) still imports display-protocol by name; scanout has no runtime
client, so its runtime module import is dropped too.
zig build + test green; display-service, virtio-gpu, display-native,
display-reattach pass.
device-manager-protocol -> library/protocol/device-manager/. Its consumers
(pci-bus, usb-xhci-bus, the acpi discovery service, device-manager, device-list,
crash-test, and the not-yet-built intel-integrated display sub-driver) import the
module directly instead of through runtime.device_manager_protocol, which is
deleted. runtime.device_manager (the hello client) already imported the module
by name and is unchanged.
zig build + test green; device-manager, driver-restart, device-list, pci-scan pass.
input-protocol -> library/protocol/input/input-protocol.zig. Its five
consumers (usb-hid keyboard/mouse, ps2 keyboard/mouse, the input service) now
import the module directly and alias it input_protocol, replacing the
runtime.input_protocol re-export (deleted) and the inconsistent local `protocol`
aliases. runtime.input (the client) still imports the module by name.
zig build + test green; input, acpi-ps2, usb-hid pass.
Start the protocol tree: wire protocols move to library/protocol/<name>/,
entry file <name>-protocol.zig, module name unchanged. block and usb-transfer
are pure moves — every consumer already imports them by module name, so only
the b.addModule paths change and the empty system/services/block/ is removed.
block-protocol -> library/protocol/block/block-protocol.zig
usb-transfer-protocol -> library/protocol/usb-transfer/usb-transfer-protocol.zig
zig build + test green.
The USB class-driver transfer client was library/runtime/usb.zig, re-exported
as runtime.usb — which compiled the USB client and usb-transfer-protocol into
every user binary (init, fat, the compositor…), none of which speak USB. It is
bus-family logic, not core runtime.
Move it to its domain home, library/device/usb/usb.zig (module "usb"), alongside
the usb-abi and usb-ids data modules; fix its internal imports to go through the
runtime module; and re-export usb.abi / usb.ids so a class driver reaches the
whole USB domain through one import. runtime.usb and runtime's
usb-transfer-protocol import are removed; the three class drivers
(usb-hid keyboard/mouse, usb-storage) import module "usb" directly.
zig build + test green; usb-hid, usb-storage pass.
New pci logic module (library/device/pci/pci.zig): a device driver's view of
the one PCI function it has claimed — Function.map (config space = resource 0),
vendorId/deviceId/command/status, enableMemoryAndBusMaster, mapBar (BAR decode
+ resource correlation + mmio_map, cached), and a capabilities() iterator. The
generic PCI mechanics every leaf PCI driver used to re-derive inline.
The config-space layout it needs — header offsets, the command MEM|bus-master
bits, the status capabilities-list bit, the capability-pointer mask, and the
BAR bit fields — is named in the pci-class data module (a "Configuration-space
layout" section), so the bus enumerator can share the same constants later.
virtio-gpu is the first consumer: its inline mapBar + walkCapabilities + config
header reads are gone, replaced by pci.Function; only the virtio-specific
cfg_type dispatch (and the virtio common-config cfgRead/cfgWrite, which are NOT
PCI config space) stay in the driver. The generic display driver will use the
same module. pci-bus's enumerator (arbitrary-function probing) is untouched.
zig build + test green; virtio-gpu, display-native, display-reattach, pci-scan pass.
Every supervised driver owes the device manager a hello at startup, and
the retry-lookup-call-check for it had been copied into five drivers:
usb.helloManager (misfiled in the USB client) plus hand-rolled twins in
display, virtio-gpu, pci-bus, and usb-xhci-bus.
Extract it once as a runtime client, runtime.device_manager.hello(role,
device_id) ?Handle — returns the manager endpoint (bus drivers keep it to
report children through), null when there is no manager or it refused the
handshake, and logs the outcome itself so each call site is one line.
Also correct two roles while collapsing their calls: virtio-gpu and the
display driver each claim one PCI function and report no children, so they
are Role.device, not Role.bus. The manager ignores role today, so this is
cosmetic, but it matches the protocol's own definition (bus = reports
children via child_added).
Behavior-preserving otherwise: pci-bus and usb-xhci-bus move their hello
logging from raw serial writes to std.log, which the kernel renders with
the same "<path>: " prefix, so driver-restart still matches
"usb-xhci-bus: hello acknowledged". zig build clean; 8 QEMU cases pass
(driver-restart, pci-scan, usb-hid, usb-storage, virtio-gpu,
display-reattach, device-list, device-manager).
The compound-hub investigation is resolved (Keychron keyboard + ROG
mouse enumerate through the Genesys USB2 companion hub and type on real
hardware). Turn the debugging spam back down for main: the full 22-port
PORTSC dump runs only when a scan finds NOTHING (a 'why is this empty'
aid), not every boot; a downstream hub port logs only when a device
actually appears or leaves, not for every empty seed-sweep port. The
readable device names, class names, and port-change events stay.
Real-hardware root-cause, from the now-readable log: the tick-poll fix
DID catch the user's full-speed devices (USB2 root ports 3, 9, 11
'device appeared — Full-speed'), but every one failed 'Address Device
completion code 4' = USB Transaction Error. Cause: danos addressed the
device immediately after the port reset, but USB 2.0 (spec 7.1.7.5)
requires a reset-recovery interval (TRSTRCY, 10 ms) before a device
answers SET_ADDRESS. QEMU tolerates the omission; real full-speed
devices do not.
setupDevice now waits 10 ms after a port reset before addressing, and
addressDeviceCommand retries up to 3 times on a transaction error,
re-resetting a root-port device between attempts (xHCI 4.6.5 recovery).
QEMU USB cases still green. Real-hardware confirmation pending — this is
the specific fix for the user's keyboard/mouse failing to address.
Make the USB diagnostic logs scannable by eye:
- usb-ids gains speedName + linkStateName (USB3 PORTSC link states: U0,
RxDetect, Polling, ...), host-tested with usb-ids.
- The PORTSC dump decodes the register instead of printing hex flags:
PORTSC[3] 0x00021203: connected, enabled, link=U0, power=on, SuperSpeed
PORTSC[5] 0x00020ee1: connected, disabled, link=Polling, power=on, High-speed
(the raw word stays for reference). A USB2 device reads connected+
disabled+Polling until reset — so the user's next log shows at a glance
whether the companion port ever reaches that state.
- The library reads STRING descriptors (readString, UTF-16LE -> ASCII,
English langid), and the device line now names the maker + product:
port 5 device: Hub "Genesys Logic USB3.0 Hub" (0x05e3:0x0626), 1 interface(s)
instead of a bare vendor/product id pair.
Full USB case set green (hub regexes follow the new 'device: <class>
"<maker> <product>"' format). Branch diagnostics for the SuperSpeed
compound-hub investigation.
The logs read 'class 9/0/0' where they could say 'Hub'. usb-ids gains
className (a device/interface class byte -> 'Hub', 'HID', 'Mass Storage',
...) and interfaceName (a HID boot interface -> 'HID boot keyboard' /
'HID boot mouse'), both host-tested with usb-ids. The bus driver's
enumerate + register lines now read, e.g.:
port 21 device: Hub vendor 0x05e3 product 0x0626, 1 interface(s)
port 21 interface 0: Hub (9/0/0) registered as device 47
port 5 interface 0: HID boot keyboard (3/1/1) registered as device 32
so a real-hardware log is scannable by eye. Full USB case set green
(hub regexes follow the new 'device: <name>' format).
Real-hardware root-cause: the user's keyboard works in the firmware boot
menu but dies under danos. The port dump shows why — danos scans the
root ports ~3 ms after the controller reset, but a USB2 connection needs
~100 ms to debounce, so the boot scan sees the USB2 companion hub's port
empty (all USB2 ports ccs=0), while the SuperSpeed devices (hub, storage)
train instantly and DO show up. danos then relied on a Port Status
Change EVENT to catch the late USB2 connection, which does not fire
reliably on this hardware.
The tick now polls every root port and reconciles on the empty->connected
EDGE (previous-state tracked per port, seeded from the boot scan so
already-up ports never re-fire), bringing up a device that appears after
the scan without depending on the event. Edge-triggered so a port that
fails to enumerate is not retried every 8 ms. Branch-only until the
keyboard is confirmed on real hardware; the diagnostic dumps stay for
the next boot.
The USB2 companion hub carrying the user's full-speed keyboard never
appears — only the SuperSpeed hub (port 19) and SS storage (port 21)
connect. To find where the USB2 root ports are and whether the companion
is presenting on one, walk the xECP Supported Protocol capabilities
(logging each USB 2.0 / 3.0 root-port range) and dump every port's raw
PORTSC unconditionally (ccs/ped/pls/pp/speed). QEMU confirms the dump
(USB2 ports 5-8, USB3 ports 1-4). Branch-only diagnostic.
Branch-only debugging for the real SuperSpeed compound hub: the spin fix
(27f87cb) stopped the hang — the SS hub's 4 SuperSpeed ports now enumerate
(all empty, correct: the full-speed keyboard isn't a SuperSpeed device) —
but the USB2 COMPANION hub, where the keyboard actually lives, never
appears. Only root ports 19 (SS hub) and 21 (SS storage) connect.
Dump every root port's raw PORTSC at scan (connect/enable/link-state/
speed) and log every root port-change event, so the next boot shows
whether the companion is connected on a USB2 port we misread, connects
late as a port-change event, or is simply absent. Diagnostic logging —
to be removed once the companion path works.
Real-hardware finding: a SuperSpeed hub (the user's Genesys, 4 SS
downstream ports) HUNG the bus driver at boot — it stopped logging with
no fault, and the hub's port power dropped. Cause: a SuperSpeed hub has
change bits a USB2 hub lacks (link-state, BH-reset), and the B4 servicing
— written and tested against QEMU's USB2 hub — never cleared them, so
the hub's status-change endpoint re-reported the same port forever and
the driver spun servicing it, never reaching the USB2 companion hub
where the full-speed keyboard actually lives.
hubPortStatusAck now clears the SuperSpeed-only change features too
(gated on hub speed; a USB2 hub is unaffected), and the tick services a
bounded batch of hub changes (32) before yielding — so even if a hub's
change bits misbehave, the driver cannot spin the machine. A per-port
status log line surfaces exactly what a hub reports, so the next real
boot shows the SS hub's port states and whether the USB2 companion
enumerates. QEMU usb-hub cases still green (the SS clears are gated off
for QEMU's USB2 hub).
There was no way to confirm a keyboard actually registers keys on real
hardware (enumeration binding the driver only proves the device came
up). The keyboard driver now echoes each decoded printable character
(and newline) to its log: type a known phrase and read it back from
usb-hid-keyboard.log, or watch it appear live on screen in a -Ddiagnose
boot (where the kernel console is a log sink). It exercises the whole
path — HID report -> diff -> layout -> character — not just enumeration,
and works identically for a keyboard behind a hub.
A usb-key-echo QEMU case injects a phrase via QMP send-key and asserts
it echoes (stable across repeated runs). Full suite 92/92.
Completes hub support (docs/usb-hub.md). On the bus tick, a downstream
hub-port change now dispatches: a connect enumerates the new device
(B4b), a disconnect tears the old one down — recursively, since a hub
that leaves takes its whole subtree with it (children first), reporting
each interface ChildRemoved and Disable-Slotting the device. Route
strings compose across tiers, so a device two hubs deep enumerates with
a two-tier route.
QEMU's hub — unlike root-port hot-plug — DOES raise downstream
status-change events, so both paths are harness-tested: usb-hub (a
keyboard behind a hub binds usb-hid-keyboard), usb-hub-nested (a
keyboard two hubs deep), usb-hub-unplug (device_del behind the hub tears
it down). Full suite 91/91.
Real-hardware validation of the user's SuperSpeed Genesys hub with
full-speed devices on its USB2 companion is flagged for the user (QEMU's
USB2 hub does not model the compound USB3 hub).
The core of hub support (docs/usb-hub.md): a device on a hub's
downstream port now reaches its class driver exactly like one on a root
port. The hub's interrupt status-change endpoint is armed with an
in-process subscription — completions set the hub's pending-port mask
instead of queuing a class-driver report — and setupHub seeds every
downstream port pending so a STATIC topology (a device present at
power-on) enumerates without waiting on the initial interrupt edge.
On the bus tick, each pending hub port is serviced: GET_STATUS + clear
the change bits, and on a connect reset the port, read the speed, then
enable a slot and Address Device with the composed route string
((parent_route<<4)|port), the inherited root port, and the parent hub's
slot/port as the TRANSACTION TRANSLATOR — so the controller routes a
full/low-speed device's split transactions through the hub's TT. The
device then enumerates and registers its interfaces through the normal
path (a compact topology-unique port key keeps the id tag within its
8-byte cap), recursing setupHub if it is itself a hub.
Verified in QEMU (a keyboard behind a USB2 hub on a second controller):
'hub slot 1 port 1 device vendor 0x0627 ... usb-hid-keyboard: ok
(device 35)'. Full suite 89/89. The user's SuperSpeed Genesys hub +
full-speed keyboard/mouse on its USB2 companion is the real-hardware
target, flagged separately.
First slice of hub support (docs/usb-hub.md), handled IN the xhci-bus
because a device behind a hub is reached by the CONTROLLER via a route
string in its slot context — topology only exists inside the driver
that owns the controller. When the scan enumerates a class-9 device it
now calls setupHub: read the hub descriptor for the downstream port
count and TT arrangement, tell the controller the slot is a hub
(Configure Endpoint with the slot add-flag sets the Hub bit, Number of
Ports, and MTT/TT-Think-Time per xHCI 4.6.6), SET_HUB_DEPTH for a
SuperSpeed hub so it can compose route strings, and SET_FEATURE
PORT_POWER every downstream port.
The Device gains topology fields (route string, root port, parent hub
slot/port for the TT) and buildAddressInputContext now fills them, so
the Address Device path is ready for downstream devices — those are
enumerated in B4b. A root-port device gets route 0 and its own port as
the chain root, unchanged behavior.
Verified in QEMU with a hub on a second controller and a device behind
it (a usb-hub QEMU case, since the boot controller's auto-assigned
devices collide on the low ports): 'hub slot 1: 8 downstream ports
powered (USB2 single-TT)'. Full suite 89/89. The user's SuperSpeed
Genesys hub is the real-hardware target, flagged separately (QEMU's
USB2 hub doesn't model the compound USB3 hub).
Regression from the B3 MPS0 work, caught on real hardware: a SuperSpeed
hub (and the SuperSpeed boot stick) failed to enumerate. bMaxPacketSize0
(device descriptor byte 7) is a LITERAL size for USB 2.0 and below
(8/16/32/64) but an EXPONENT for SuperSpeed (9 = 2^9 = 512). The refresh
read the exponent 9 as a size and issued Evaluate Context to set EP0 to
9 bytes, corrupting the control endpoint so every following transfer
failed. SuperSpeed's EP0 is fixed at 512 and needs no correction, so
the refresh is now skipped for speed >= 4; only full/low/high speed,
where the field is a literal that can differ from the speed default,
still run it. QEMU tolerated the wrong MPS0 — real silicon does not
(the class of bug the harness can't reach, flagged real-HW-pending).
This likely also explains intermittent no-storage/no-logs on a normal
boot: the boot stick is SuperSpeed and hit the same corruption. Full
suite green (orderly-shutdown's lone failure was its known QMP-timing
flake — three clean re-runs).
B3 — the runtime lifecycle a hot-pluggable bus needs, plus two init
fixes that runtime device arrival depends on:
- pump() now handles PORT STATUS CHANGE events (silently dropped
before): it queues the port, and the bus driver brings the port up
(a device arrived) or tears it down (a device left) on its tick —
reporting each interface ChildRemoved to the device manager, which
prunes the node, notifies watchers, and lets the class driver's world
end honestly, then Disable Slot frees the controller-side state.
- ALL root-hub ports are powered at init, not just those with a
boot-time device: an unpowered port (PP=0) cannot signal a later
connect, so a hot-plug would never be seen.
- the interrupter is enabled (IMAN.IE + USBCMD.INTE) while the ring
stays polled — some controllers only WRITE runtime events to the
ring when the interrupter is enabled.
Real-hardware validation is flagged for the user: QEMU's qemu-xhci does
not raise a runtime port-change event to a polling driver on device_add,
so the end-to-end hot-plug path can't be exercised in the harness (the
port-change handling itself IS proven — a late boot device's PSCE is
caught and acked). The harness gained qmp_sequence (multi-step QMP
injection with arguments) for when a drivable case exists. Full suite
88/88; the working USB path (enumeration, HID, storage) is unregressed
by the port-power and interrupter changes.
Two real-hardware correctness fixes from the M20 list, both invisible
to QEMU's forgiving controller:
refreshMaxPacketSize0 existed only as a comment. The EP0 context kept
the SPEED-DEFAULT max packet size (full-speed: 8) even when the device
declares 16/32/64 — real controllers fault the very first full
descriptor read on the mismatch, which is the standing suspect for
full-speed mice dying on the user's PC. Enumeration now probes the
device descriptor's first 8 bytes (deliverable at any legal MPS0),
and issues Evaluate Context to correct EP0 before any longer transfer,
naming the failure and codes if the controller refuses.
And a USB2 port's PORTSC speed field is only meaningful once the port
reset ENABLES the port: the speed (and the MPS0 default derived from
it) is now sampled after the reset instead of trusting the connect-time
read.
B2 — the 1-in-3 boot-time READ CAPACITY failure, root-caused: the xHCI
library's awaitTransfer claimed ANY unclaimed transfer event as its own
completion. An interrupt-endpoint event whose TRB pointer no longer
matched the armed subscription (an error or stale completion from the
keyboard/mouse polling concurrently with storage bring-up) fell through
and was misread as the bulk transfer's completion — desynchronizing the
mass-storage bulk protocol in controller state that SURVIVED driver
restarts, so every retry failed too. Awaited transfers now match the
event's slot id and endpoint DCI; foreign events are dropped and named.
Twelve consecutive runs of the previously-flaky cases pass; the full
suite is green with none of its old intermittents.
B1 — and when storage does fail transiently, the system now heals
instead of giving up forever: a nonzero exit maps to ExitReason.aborted
(a deliberate FAILURE exit — supervisors restart those with backoff,
unlike a clean .exited), usb-storage exits nonzero when a PRESENT
device fails bring-up, and the fat service no longer blocks its harness
polling for a block device and then dies — it serves immediately
(requests fail politely), retries on a 500 ms timer, and mounts
whenever storage appears, including after a driver restart.
Real-PC diagnose boot (photo + OCR): the boot stick connects at
SuperSpeed on port 21 and 'device setup failed' lands in the SAME
millisecond — an instant failure, not a timeout. setupDevice reset
every port unconditionally; that is required to enable USB2 ports, but
a SuperSpeed port that trained its link is ALREADY enabled (xHCI
advances USB3 ports to Enabled, no reset — spec 4.3), and driving a hot
reset into the live link drops PED mid-reset on real silicon. QEMU
tolerates the spurious reset, which is why the harness never saw it.
An enabled speed>=4 port now skips the reset (a not-yet-enabled SS link
still gets one). Every setup step names its failure — port reset with
the PORTSC value, Enable Slot, device-slot exhaustion, Address Device
with its completion code — so the on-screen transcript of the next
failure identifies the exact xHCI command instead of one blanket line.
block.open's give-up window drops 60 s -> 30 s (the slowest observed
healthy chain completed at ~24 s); a machine whose stick failed setup
should not sit a further minute pretending otherwise.
Per docs/coding-standards.md (no Unix-abbreviation exception): syscalls
shared_memory_create/map/physical, kernel SharedMemoryObject + handlers,
runtime.shared_memory (library/runtime/shared-memory.zig), the
shared-memory-server/-client test services, the shared-memory QEMU case,
and docs incl. vdso.md's danos_shared_memory_*. 87/87 QEMU tests pass.
library/runtime/log.zig wires std.log to the tagged ring: the root shim
installs std_options for every binary (programs may override), logFn
formats one line per record and emits it with its level via
debug_write — the payload no longer carries the process's name; the
kernel stamps identity structurally and the serial renderer prints the
'<binary path>: ' prefix, so the transcript keeps its shape.
Migrate every writeLine/logLine/log helper family (init, vfs, fat,
device-manager, acpi, pci-bus, ps2-bus x3, usb-hid x2, usb-storage,
usb-xhci-bus, virtio-gpu — 104 call sites) to std.log.info, dropping
the hand-written prefixes. A leveled record is a complete line by
contract (raw emissions may still build lines from pieces). Test
fixtures and the display/input services keep raw writes for now — their
ring records are attributed by the kernel regardless.
Harness regexes follow the renamed prefixes (usb-hid-keyboard,
discovery) and path-named restart lines.
Both EDID moments the system has are now captured and carried to the
compositor's frame clock:
- EFI: the loader derives refresh from the preferred detailed timing
(pixel clock / total pixels) while GOP is still alive — the only moment
it is readable — and hands it through the boot handoff into the
display0 node's DisplayInfo (new refresh_hz field, 0 = unknown).
- GPU: the virtio-gpu driver derives the same figure from its own EDID
read and carries it in the attach_scanout announce (request.y).
updateFrameClock() re-derives the interval from the active backend's
info at bring-up and again on every backend change — the boot
framebuffer's clock dies with the GOP floor at upgrade, replaced by the
GPU's rate. Unknown rate defaults to 60 Hz; the result is clamped to
[30, 120] Hz so a mis-parsed EDID can neither starve nor flood the
compositor. Rate only, never phase: without vblank, presents still
free-run (docs/display-v2.md, 'Fenced is not vsync').
Observed in QEMU: OVMF exposes no EDID for the VGA adapter, so the GOP
floor logs 'frame clock 62 Hz (default)' (real firmware does expose it);
the virtio-gpu EDID advertises 75 Hz and the upgrade logs 'frame clock
76 Hz (panel EDID)'. The display kernel test asserts refresh_hz rides
the seeded node; all 7 display QEMU cases pass.
The virtio-gpu present fence completes when the device has consumed the
frame: real completion feedback, and tear-freedom by snapshot semantics.
It is not a vblank — base virtio-gpu 2D has no display-refresh event at
all (Linux fakes one with a timer), so nothing paces presents to the
monitor. The code and docs claimed vsync anyway; now they don't.
- backend.hasVsync -> hasFencedPresent, with an honest doc comment
- marker 'display: vsync present ok' -> 'display: fenced present ok'
(display-modeset test expectation updated, passes)
- display-v2.md gains a 'Fenced is not vsync' note; the vsync claims in
both v2 docs are corrected
- true vsync arrives with a native driver's vblank IRQ, or approximated
by a compositor frame clock
The compositor now survives the virtio-gpu driver dying and re-attaches when device-manager
restarts it — the last piece of display v2.
Three parts:
- The driver hellos the device manager (role: bus). It never did, so the manager — which
spawns it supervised and expects a hello — was stopping it at the 3s hello deadline every
run (the gate markers just printed first). Now it is properly supervised: not stopped for
silence, and restarted on death.
- A kernel IPC fix so a call to a dead service errors instead of hanging. An endpoint records
its owner; when that task dies, its registered endpoints are marked dead (and any parked
senders woken with -EPEER), so ipc_call returns -EPEER rather than blocking on a reply that
will never come. Without this the compositor's first present after the driver died blocked
forever. General robustness — any client of any service benefits.
- The compositor re-attaches. Its .scanout calls now fail cleanly (caught), freezing the last
frame; when the restarted driver re-announces, attach_scanout detects the backend is already
virtio and logs "scanout re-attached", mapping the fresh shared surface and re-looking-up
.scanout. (The previous shm mapping leaks — no shm_unmap syscall yet — but its frames are the
dead driver's, reclaimed on exit.)
- device-manager gains a "test-scanout-restart" mode (like test-usb-restart) that kills the
virtio-gpu driver once after it hellos; the displayReattachTest kernel scenario drives it.
Gate: python3 test/qemu_test.py display-reattach — "scanout upgraded to virtio-gpu" then
"scanout re-attached", no CPU exception, passing 3/3. host tests, ipc/ipc-call/ipc-cap,
supervision, shm, display-service, display-demo, virtio-gpu, display-native, and
display-modeset all pass; default zig build clean. v2 (V1-V6) complete.
The native backend can now change resolution and presents tear-free.
Mode-setting without churn. The driver sizes its scanout resource + shared surface to the
largest mode it offers and treats a mode change as re-pointing the scanout rectangle within
that surface — so the resource, its backing, and the shared mapping never change, and the
surface's row stride (the max width) is fixed while the active width/height move. The
compositor is handed that stride in the announce and composes at it; a smaller mode just
paints the top-left rectangle. This sidesteps the surface re-share a true resolution change
would otherwise need (the service harness can't reply with a capability).
- scanout-protocol gains get_modes + set_mode; the driver offers {640x480, 800x600} and
re-points set_scanout on set_mode.
- backend.VirtioGpu carries the surface stride, exposes modes()/setMode(), and reports
canModeSet = hasVsync = true.
- runtime.display gains modes()/setMode() (display-protocol get_modes/set_mode, forwarded to
the backend) — the client-facing API.
- EDID: the driver negotiates VIRTIO_GPU_F_EDID when the device offers it, reads the monitor's
EDID, and logs its preferred mode (parsed from the first detailed timing descriptor).
- vsync: every resource_flush is fenced (VIRTIO_GPU_FLAG_FENCE); the device signals the fence
when the frame is on screen, which the used-ring ack the synchronous present already waits
on gates — so a completed present is a tear-free one.
After the native upgrade the compositor runs a one-shot mode-set self-check: query the modes,
switch to a different one, re-composite, and confirm the backend reports the new geometry —
the gate's markers.
Gate: python3 test/qemu_test.py display-modeset (reuses the display-native boot) — "display:
mode set to 800x600, verified" + "display: vsync present ok", passing 3/3. host tests,
display-service, display-demo, shm, virtio-gpu, and display-native still pass.
The compositor now boots on the GOP framebuffer and upgrades to the virtio-gpu driver
the moment it announces itself — the pluggable-scanout payoff.
The shared surface. The scanout resource is an shm region the driver creates
(shm_physical, a new syscall, hands it the guest-physical for attach_backing) and passes
to the compositor as a capability. The compositor maps it and composites straight into
it: on x86 DMA is cache-coherent, so the cacheable shared pages the CPU paints are exactly
what the device transfers-and-flushes — no copy, no explicit flush.
The handshake. After bring-up the driver looks up .display and sends attach_scanout with
the geometry + the surface capability. The compositor maps the surface, looks up the
driver's .scanout endpoint itself (the driver registered it — no need to pass it), switches
to backend.VirtioGpu, and re-composites the current frame. present() over the native
backend is a present request on .scanout -> transfer-to-host + resource flush. The first
native present is deferred to a one-shot timer: presenting inline from the announce handler
would deadlock, since the driver is still blocked on our reply and not yet serving .scanout.
After it lands, the compositor reads a pixel back from the shared surface to confirm the
frame reached the device's backing.
- shm_physical (syscall 36) + runtime.shm.physical.
- scanout-protocol (the compositor->driver present channel), separate from the
client-facing display protocol; the display protocol gains attach_scanout.
- backend.VirtioGpu joins backend.Gop in the tagged union; select() still boots GOP.
- the virtio-gpu driver's scanout backing is now shm (was DMA); it announces + serves
.scanout present requests (transfer-to-host + flush of the shared surface).
Also fixes a latent framebuffer-geometry corruption the display service hit only when it
enumerated the device tree alongside a busy device-manager: Gop.init now captures the
geometry into a small value the instant device_enumerate returns (rather than re-reading
the 328-byte descriptor across the later claim/mmio_map syscalls) and retries on a zero
geometry. The underlying device-table clobber is a separate kernel bug, tracked apart.
Gate: python3 test/qemu_test.py display-native (QEMU -device virtio-gpu-pci) — "display:
scanout upgraded to virtio-gpu" + "display: native present verified" + "display-demo: ok",
passing 3/3. host tests, display-service, display-demo, shm, and virtio-gpu still pass.
The first native scanout backend's driver half. The device manager matches the
virtio-gpu PCI function by its Display/Other class triple and spawns the driver,
which claims the device, confirms vendor 0x1AF4/device 0x1050 from config space,
enables memory-space + bus mastering, and walks the virtio vendor capabilities to
find the common-config and notify structures in its BAR.
From there it is the standard modern-virtio bring-up: reset, negotiate VERSION_1,
stand up the control virtqueue in coherent DMA, then drive the GPU end to end —
RESOURCE_CREATE_2D, ATTACH_BACKING (a coherent DMA buffer; V4 swaps in the shm-
shared surface), SET_SCANOUT, paint a known pattern, TRANSFER_TO_HOST_2D,
RESOURCE_FLUSH, and wait for the device's used-ring ack. Reading the backing back
proves it is CPU-visible RAM; the ack proves the device consumed the frame —
together the automated stand-in for "it's on screen", no screenshot.
- system/drivers/virtio-gpu/: the driver, plus virtio-gpu-protocol.zig (control
commands) and virtio-pci.zig (the 1.0 PCI transport + split-virtqueue), both with
host-tested struct sizes.
- device-manager matches the display/other class triple to "virtio-gpu"; the driver
self-confirms the vendor/device id, since the class alone cannot distinguish it.
- ServiceId.scanout (11): the driver registers it so the compositor finds it in V4.
Gate: python3 test/qemu_test.py virtio-gpu (QEMU -device virtio-gpu-pci) — the
device-manager stack discovers the function, the driver brings up a 640x480 scanout
and flushes a test pattern: "virtio-gpu: scanout 640x480 online" + "flush acked,
pixel check ok". host tests, display-service, shm, and device-list still pass.
Note: the pre-existing pci-scan case triple-faults on main (verified at 88ad432,
before this change); it is unrelated and tracked separately.
init writes /mnt/usb/DANOS.LOG at shutdown and then enters S5 — but the write sat in the USB flash controller's write cache and was lost when power was cut, because nothing issued SCSI SYNCHRONIZE CACHE. Invisible in QEMU (its backing file commits immediately); real on hardware. The boot-time flush survived only because the machine kept running afterward and the cache drained on its own.
- block protocol gains a flush op; usb-storage serves it with SYNCHRONIZE CACHE (10); runtime.block gains Device.flush()
- the FAT server tracks whether blocks were written and, on a file close, commits the device cache (durable-on-close — the right default for removable media, and it makes init's existing shutdown close() persist the log before S5, no init/VFS change needed)
- verified the SYNCHRONIZE CACHE actually reaches the driver on each dirty close; usb-storage/fat-mount/mutations/rename/mtime/log-flush all green
Daniel Samson
Daniel Samson