Real-hardware finding: a SuperSpeed hub (the user's Genesys, 4 SS
downstream ports) HUNG the bus driver at boot — it stopped logging with
no fault, and the hub's port power dropped. Cause: a SuperSpeed hub has
change bits a USB2 hub lacks (link-state, BH-reset), and the B4 servicing
— written and tested against QEMU's USB2 hub — never cleared them, so
the hub's status-change endpoint re-reported the same port forever and
the driver spun servicing it, never reaching the USB2 companion hub
where the full-speed keyboard actually lives.
hubPortStatusAck now clears the SuperSpeed-only change features too
(gated on hub speed; a USB2 hub is unaffected), and the tick services a
bounded batch of hub changes (32) before yielding — so even if a hub's
change bits misbehave, the driver cannot spin the machine. A per-port
status log line surfaces exactly what a hub reports, so the next real
boot shows the SS hub's port states and whether the USB2 companion
enumerates. QEMU usb-hub cases still green (the SS clears are gated off
for QEMU's USB2 hub).
There was no way to confirm a keyboard actually registers keys on real
hardware (enumeration binding the driver only proves the device came
up). The keyboard driver now echoes each decoded printable character
(and newline) to its log: type a known phrase and read it back from
usb-hid-keyboard.log, or watch it appear live on screen in a -Ddiagnose
boot (where the kernel console is a log sink). It exercises the whole
path — HID report -> diff -> layout -> character — not just enumeration,
and works identically for a keyboard behind a hub.
A usb-key-echo QEMU case injects a phrase via QMP send-key and asserts
it echoes (stable across repeated runs). Full suite 92/92.
Completes hub support (docs/usb-hub.md). On the bus tick, a downstream
hub-port change now dispatches: a connect enumerates the new device
(B4b), a disconnect tears the old one down — recursively, since a hub
that leaves takes its whole subtree with it (children first), reporting
each interface ChildRemoved and Disable-Slotting the device. Route
strings compose across tiers, so a device two hubs deep enumerates with
a two-tier route.
QEMU's hub — unlike root-port hot-plug — DOES raise downstream
status-change events, so both paths are harness-tested: usb-hub (a
keyboard behind a hub binds usb-hid-keyboard), usb-hub-nested (a
keyboard two hubs deep), usb-hub-unplug (device_del behind the hub tears
it down). Full suite 91/91.
Real-hardware validation of the user's SuperSpeed Genesys hub with
full-speed devices on its USB2 companion is flagged for the user (QEMU's
USB2 hub does not model the compound USB3 hub).
The core of hub support (docs/usb-hub.md): a device on a hub's
downstream port now reaches its class driver exactly like one on a root
port. The hub's interrupt status-change endpoint is armed with an
in-process subscription — completions set the hub's pending-port mask
instead of queuing a class-driver report — and setupHub seeds every
downstream port pending so a STATIC topology (a device present at
power-on) enumerates without waiting on the initial interrupt edge.
On the bus tick, each pending hub port is serviced: GET_STATUS + clear
the change bits, and on a connect reset the port, read the speed, then
enable a slot and Address Device with the composed route string
((parent_route<<4)|port), the inherited root port, and the parent hub's
slot/port as the TRANSACTION TRANSLATOR — so the controller routes a
full/low-speed device's split transactions through the hub's TT. The
device then enumerates and registers its interfaces through the normal
path (a compact topology-unique port key keeps the id tag within its
8-byte cap), recursing setupHub if it is itself a hub.
Verified in QEMU (a keyboard behind a USB2 hub on a second controller):
'hub slot 1 port 1 device vendor 0x0627 ... usb-hid-keyboard: ok
(device 35)'. Full suite 89/89. The user's SuperSpeed Genesys hub +
full-speed keyboard/mouse on its USB2 companion is the real-hardware
target, flagged separately.
First slice of hub support (docs/usb-hub.md), handled IN the xhci-bus
because a device behind a hub is reached by the CONTROLLER via a route
string in its slot context — topology only exists inside the driver
that owns the controller. When the scan enumerates a class-9 device it
now calls setupHub: read the hub descriptor for the downstream port
count and TT arrangement, tell the controller the slot is a hub
(Configure Endpoint with the slot add-flag sets the Hub bit, Number of
Ports, and MTT/TT-Think-Time per xHCI 4.6.6), SET_HUB_DEPTH for a
SuperSpeed hub so it can compose route strings, and SET_FEATURE
PORT_POWER every downstream port.
The Device gains topology fields (route string, root port, parent hub
slot/port for the TT) and buildAddressInputContext now fills them, so
the Address Device path is ready for downstream devices — those are
enumerated in B4b. A root-port device gets route 0 and its own port as
the chain root, unchanged behavior.
Verified in QEMU with a hub on a second controller and a device behind
it (a usb-hub QEMU case, since the boot controller's auto-assigned
devices collide on the low ports): 'hub slot 1: 8 downstream ports
powered (USB2 single-TT)'. Full suite 89/89. The user's SuperSpeed
Genesys hub is the real-hardware target, flagged separately (QEMU's
USB2 hub doesn't model the compound USB3 hub).
Regression from the B3 MPS0 work, caught on real hardware: a SuperSpeed
hub (and the SuperSpeed boot stick) failed to enumerate. bMaxPacketSize0
(device descriptor byte 7) is a LITERAL size for USB 2.0 and below
(8/16/32/64) but an EXPONENT for SuperSpeed (9 = 2^9 = 512). The refresh
read the exponent 9 as a size and issued Evaluate Context to set EP0 to
9 bytes, corrupting the control endpoint so every following transfer
failed. SuperSpeed's EP0 is fixed at 512 and needs no correction, so
the refresh is now skipped for speed >= 4; only full/low/high speed,
where the field is a literal that can differ from the speed default,
still run it. QEMU tolerated the wrong MPS0 — real silicon does not
(the class of bug the harness can't reach, flagged real-HW-pending).
This likely also explains intermittent no-storage/no-logs on a normal
boot: the boot stick is SuperSpeed and hit the same corruption. Full
suite green (orderly-shutdown's lone failure was its known QMP-timing
flake — three clean re-runs).
B3 — the runtime lifecycle a hot-pluggable bus needs, plus two init
fixes that runtime device arrival depends on:
- pump() now handles PORT STATUS CHANGE events (silently dropped
before): it queues the port, and the bus driver brings the port up
(a device arrived) or tears it down (a device left) on its tick —
reporting each interface ChildRemoved to the device manager, which
prunes the node, notifies watchers, and lets the class driver's world
end honestly, then Disable Slot frees the controller-side state.
- ALL root-hub ports are powered at init, not just those with a
boot-time device: an unpowered port (PP=0) cannot signal a later
connect, so a hot-plug would never be seen.
- the interrupter is enabled (IMAN.IE + USBCMD.INTE) while the ring
stays polled — some controllers only WRITE runtime events to the
ring when the interrupter is enabled.
Real-hardware validation is flagged for the user: QEMU's qemu-xhci does
not raise a runtime port-change event to a polling driver on device_add,
so the end-to-end hot-plug path can't be exercised in the harness (the
port-change handling itself IS proven — a late boot device's PSCE is
caught and acked). The harness gained qmp_sequence (multi-step QMP
injection with arguments) for when a drivable case exists. Full suite
88/88; the working USB path (enumeration, HID, storage) is unregressed
by the port-power and interrupter changes.
Two real-hardware correctness fixes from the M20 list, both invisible
to QEMU's forgiving controller:
refreshMaxPacketSize0 existed only as a comment. The EP0 context kept
the SPEED-DEFAULT max packet size (full-speed: 8) even when the device
declares 16/32/64 — real controllers fault the very first full
descriptor read on the mismatch, which is the standing suspect for
full-speed mice dying on the user's PC. Enumeration now probes the
device descriptor's first 8 bytes (deliverable at any legal MPS0),
and issues Evaluate Context to correct EP0 before any longer transfer,
naming the failure and codes if the controller refuses.
And a USB2 port's PORTSC speed field is only meaningful once the port
reset ENABLES the port: the speed (and the MPS0 default derived from
it) is now sampled after the reset instead of trusting the connect-time
read.
B2 — the 1-in-3 boot-time READ CAPACITY failure, root-caused: the xHCI
library's awaitTransfer claimed ANY unclaimed transfer event as its own
completion. An interrupt-endpoint event whose TRB pointer no longer
matched the armed subscription (an error or stale completion from the
keyboard/mouse polling concurrently with storage bring-up) fell through
and was misread as the bulk transfer's completion — desynchronizing the
mass-storage bulk protocol in controller state that SURVIVED driver
restarts, so every retry failed too. Awaited transfers now match the
event's slot id and endpoint DCI; foreign events are dropped and named.
Twelve consecutive runs of the previously-flaky cases pass; the full
suite is green with none of its old intermittents.
B1 — and when storage does fail transiently, the system now heals
instead of giving up forever: a nonzero exit maps to ExitReason.aborted
(a deliberate FAILURE exit — supervisors restart those with backoff,
unlike a clean .exited), usb-storage exits nonzero when a PRESENT
device fails bring-up, and the fat service no longer blocks its harness
polling for a block device and then dies — it serves immediately
(requests fail politely), retries on a 500 ms timer, and mounts
whenever storage appears, including after a driver restart.
Real-PC diagnose boot (photo + OCR): the boot stick connects at
SuperSpeed on port 21 and 'device setup failed' lands in the SAME
millisecond — an instant failure, not a timeout. setupDevice reset
every port unconditionally; that is required to enable USB2 ports, but
a SuperSpeed port that trained its link is ALREADY enabled (xHCI
advances USB3 ports to Enabled, no reset — spec 4.3), and driving a hot
reset into the live link drops PED mid-reset on real silicon. QEMU
tolerates the spurious reset, which is why the harness never saw it.
An enabled speed>=4 port now skips the reset (a not-yet-enabled SS link
still gets one). Every setup step names its failure — port reset with
the PORTSC value, Enable Slot, device-slot exhaustion, Address Device
with its completion code — so the on-screen transcript of the next
failure identifies the exact xHCI command instead of one blanket line.
block.open's give-up window drops 60 s -> 30 s (the slowest observed
healthy chain completed at ~24 s); a machine whose stick failed setup
should not sit a further minute pretending otherwise.
Per docs/coding-standards.md (no Unix-abbreviation exception): syscalls
shared_memory_create/map/physical, kernel SharedMemoryObject + handlers,
runtime.shared_memory (library/runtime/shared-memory.zig), the
shared-memory-server/-client test services, the shared-memory QEMU case,
and docs incl. vdso.md's danos_shared_memory_*. 87/87 QEMU tests pass.
library/runtime/log.zig wires std.log to the tagged ring: the root shim
installs std_options for every binary (programs may override), logFn
formats one line per record and emits it with its level via
debug_write — the payload no longer carries the process's name; the
kernel stamps identity structurally and the serial renderer prints the
'<binary path>: ' prefix, so the transcript keeps its shape.
Migrate every writeLine/logLine/log helper family (init, vfs, fat,
device-manager, acpi, pci-bus, ps2-bus x3, usb-hid x2, usb-storage,
usb-xhci-bus, virtio-gpu — 104 call sites) to std.log.info, dropping
the hand-written prefixes. A leveled record is a complete line by
contract (raw emissions may still build lines from pieces). Test
fixtures and the display/input services keep raw writes for now — their
ring records are attributed by the kernel regardless.
Harness regexes follow the renamed prefixes (usb-hid-keyboard,
discovery) and path-named restart lines.
Both EDID moments the system has are now captured and carried to the
compositor's frame clock:
- EFI: the loader derives refresh from the preferred detailed timing
(pixel clock / total pixels) while GOP is still alive — the only moment
it is readable — and hands it through the boot handoff into the
display0 node's DisplayInfo (new refresh_hz field, 0 = unknown).
- GPU: the virtio-gpu driver derives the same figure from its own EDID
read and carries it in the attach_scanout announce (request.y).
updateFrameClock() re-derives the interval from the active backend's
info at bring-up and again on every backend change — the boot
framebuffer's clock dies with the GOP floor at upgrade, replaced by the
GPU's rate. Unknown rate defaults to 60 Hz; the result is clamped to
[30, 120] Hz so a mis-parsed EDID can neither starve nor flood the
compositor. Rate only, never phase: without vblank, presents still
free-run (docs/display-v2.md, 'Fenced is not vsync').
Observed in QEMU: OVMF exposes no EDID for the VGA adapter, so the GOP
floor logs 'frame clock 62 Hz (default)' (real firmware does expose it);
the virtio-gpu EDID advertises 75 Hz and the upgrade logs 'frame clock
76 Hz (panel EDID)'. The display kernel test asserts refresh_hz rides
the seeded node; all 7 display QEMU cases pass.
The virtio-gpu present fence completes when the device has consumed the
frame: real completion feedback, and tear-freedom by snapshot semantics.
It is not a vblank — base virtio-gpu 2D has no display-refresh event at
all (Linux fakes one with a timer), so nothing paces presents to the
monitor. The code and docs claimed vsync anyway; now they don't.
- backend.hasVsync -> hasFencedPresent, with an honest doc comment
- marker 'display: vsync present ok' -> 'display: fenced present ok'
(display-modeset test expectation updated, passes)
- display-v2.md gains a 'Fenced is not vsync' note; the vsync claims in
both v2 docs are corrected
- true vsync arrives with a native driver's vblank IRQ, or approximated
by a compositor frame clock
The compositor now survives the virtio-gpu driver dying and re-attaches when device-manager
restarts it — the last piece of display v2.
Three parts:
- The driver hellos the device manager (role: bus). It never did, so the manager — which
spawns it supervised and expects a hello — was stopping it at the 3s hello deadline every
run (the gate markers just printed first). Now it is properly supervised: not stopped for
silence, and restarted on death.
- A kernel IPC fix so a call to a dead service errors instead of hanging. An endpoint records
its owner; when that task dies, its registered endpoints are marked dead (and any parked
senders woken with -EPEER), so ipc_call returns -EPEER rather than blocking on a reply that
will never come. Without this the compositor's first present after the driver died blocked
forever. General robustness — any client of any service benefits.
- The compositor re-attaches. Its .scanout calls now fail cleanly (caught), freezing the last
frame; when the restarted driver re-announces, attach_scanout detects the backend is already
virtio and logs "scanout re-attached", mapping the fresh shared surface and re-looking-up
.scanout. (The previous shm mapping leaks — no shm_unmap syscall yet — but its frames are the
dead driver's, reclaimed on exit.)
- device-manager gains a "test-scanout-restart" mode (like test-usb-restart) that kills the
virtio-gpu driver once after it hellos; the displayReattachTest kernel scenario drives it.
Gate: python3 test/qemu_test.py display-reattach — "scanout upgraded to virtio-gpu" then
"scanout re-attached", no CPU exception, passing 3/3. host tests, ipc/ipc-call/ipc-cap,
supervision, shm, display-service, display-demo, virtio-gpu, display-native, and
display-modeset all pass; default zig build clean. v2 (V1-V6) complete.
The native backend can now change resolution and presents tear-free.
Mode-setting without churn. The driver sizes its scanout resource + shared surface to the
largest mode it offers and treats a mode change as re-pointing the scanout rectangle within
that surface — so the resource, its backing, and the shared mapping never change, and the
surface's row stride (the max width) is fixed while the active width/height move. The
compositor is handed that stride in the announce and composes at it; a smaller mode just
paints the top-left rectangle. This sidesteps the surface re-share a true resolution change
would otherwise need (the service harness can't reply with a capability).
- scanout-protocol gains get_modes + set_mode; the driver offers {640x480, 800x600} and
re-points set_scanout on set_mode.
- backend.VirtioGpu carries the surface stride, exposes modes()/setMode(), and reports
canModeSet = hasVsync = true.
- runtime.display gains modes()/setMode() (display-protocol get_modes/set_mode, forwarded to
the backend) — the client-facing API.
- EDID: the driver negotiates VIRTIO_GPU_F_EDID when the device offers it, reads the monitor's
EDID, and logs its preferred mode (parsed from the first detailed timing descriptor).
- vsync: every resource_flush is fenced (VIRTIO_GPU_FLAG_FENCE); the device signals the fence
when the frame is on screen, which the used-ring ack the synchronous present already waits
on gates — so a completed present is a tear-free one.
After the native upgrade the compositor runs a one-shot mode-set self-check: query the modes,
switch to a different one, re-composite, and confirm the backend reports the new geometry —
the gate's markers.
Gate: python3 test/qemu_test.py display-modeset (reuses the display-native boot) — "display:
mode set to 800x600, verified" + "display: vsync present ok", passing 3/3. host tests,
display-service, display-demo, shm, virtio-gpu, and display-native still pass.
The compositor now boots on the GOP framebuffer and upgrades to the virtio-gpu driver
the moment it announces itself — the pluggable-scanout payoff.
The shared surface. The scanout resource is an shm region the driver creates
(shm_physical, a new syscall, hands it the guest-physical for attach_backing) and passes
to the compositor as a capability. The compositor maps it and composites straight into
it: on x86 DMA is cache-coherent, so the cacheable shared pages the CPU paints are exactly
what the device transfers-and-flushes — no copy, no explicit flush.
The handshake. After bring-up the driver looks up .display and sends attach_scanout with
the geometry + the surface capability. The compositor maps the surface, looks up the
driver's .scanout endpoint itself (the driver registered it — no need to pass it), switches
to backend.VirtioGpu, and re-composites the current frame. present() over the native
backend is a present request on .scanout -> transfer-to-host + resource flush. The first
native present is deferred to a one-shot timer: presenting inline from the announce handler
would deadlock, since the driver is still blocked on our reply and not yet serving .scanout.
After it lands, the compositor reads a pixel back from the shared surface to confirm the
frame reached the device's backing.
- shm_physical (syscall 36) + runtime.shm.physical.
- scanout-protocol (the compositor->driver present channel), separate from the
client-facing display protocol; the display protocol gains attach_scanout.
- backend.VirtioGpu joins backend.Gop in the tagged union; select() still boots GOP.
- the virtio-gpu driver's scanout backing is now shm (was DMA); it announces + serves
.scanout present requests (transfer-to-host + flush of the shared surface).
Also fixes a latent framebuffer-geometry corruption the display service hit only when it
enumerated the device tree alongside a busy device-manager: Gop.init now captures the
geometry into a small value the instant device_enumerate returns (rather than re-reading
the 328-byte descriptor across the later claim/mmio_map syscalls) and retries on a zero
geometry. The underlying device-table clobber is a separate kernel bug, tracked apart.
Gate: python3 test/qemu_test.py display-native (QEMU -device virtio-gpu-pci) — "display:
scanout upgraded to virtio-gpu" + "display: native present verified" + "display-demo: ok",
passing 3/3. host tests, display-service, display-demo, shm, and virtio-gpu still pass.
The first native scanout backend's driver half. The device manager matches the
virtio-gpu PCI function by its Display/Other class triple and spawns the driver,
which claims the device, confirms vendor 0x1AF4/device 0x1050 from config space,
enables memory-space + bus mastering, and walks the virtio vendor capabilities to
find the common-config and notify structures in its BAR.
From there it is the standard modern-virtio bring-up: reset, negotiate VERSION_1,
stand up the control virtqueue in coherent DMA, then drive the GPU end to end —
RESOURCE_CREATE_2D, ATTACH_BACKING (a coherent DMA buffer; V4 swaps in the shm-
shared surface), SET_SCANOUT, paint a known pattern, TRANSFER_TO_HOST_2D,
RESOURCE_FLUSH, and wait for the device's used-ring ack. Reading the backing back
proves it is CPU-visible RAM; the ack proves the device consumed the frame —
together the automated stand-in for "it's on screen", no screenshot.
- system/drivers/virtio-gpu/: the driver, plus virtio-gpu-protocol.zig (control
commands) and virtio-pci.zig (the 1.0 PCI transport + split-virtqueue), both with
host-tested struct sizes.
- device-manager matches the display/other class triple to "virtio-gpu"; the driver
self-confirms the vendor/device id, since the class alone cannot distinguish it.
- ServiceId.scanout (11): the driver registers it so the compositor finds it in V4.
Gate: python3 test/qemu_test.py virtio-gpu (QEMU -device virtio-gpu-pci) — the
device-manager stack discovers the function, the driver brings up a 640x480 scanout
and flushes a test pattern: "virtio-gpu: scanout 640x480 online" + "flush acked,
pixel check ok". host tests, display-service, shm, and device-list still pass.
Note: the pre-existing pci-scan case triple-faults on main (verified at 88ad432,
before this change); it is unrelated and tracked separately.
init writes /mnt/usb/DANOS.LOG at shutdown and then enters S5 — but the write sat in the USB flash controller's write cache and was lost when power was cut, because nothing issued SCSI SYNCHRONIZE CACHE. Invisible in QEMU (its backing file commits immediately); real on hardware. The boot-time flush survived only because the machine kept running afterward and the cache drained on its own.
- block protocol gains a flush op; usb-storage serves it with SYNCHRONIZE CACHE (10); runtime.block gains Device.flush()
- the FAT server tracks whether blocks were written and, on a file close, commits the device cache (durable-on-close — the right default for removable media, and it makes init's existing shutdown close() persist the log before S5, no init/VFS change needed)
- verified the SYNCHRONIZE CACHE actually reaches the driver on each dirty close; usb-storage/fat-mount/mutations/rename/mtime/log-flush all green
Flesh out the xHCI host-controller driver into a full transfer engine and build
the three USB class drivers on top, all verified end to end under QEMU.
- xHCI engine (usb-xhci-library.zig): controller reset, command/event rings with
cycle-bit bookkeeping (gated on a No-Op-command proof), device slots, Address
Device, control transfers, full chapter-9 enumeration, Configure Endpoint, and
interrupt/bulk transfers. Each interface is device_registered with its
(class,subclass,protocol) identity, unique per (port,interface).
- Bus<->class transfer protocol (usb-transfer-protocol.zig + runtime.usb): open /
control / interrupt-subscribe (async report pump on a poll timer) / bulk-by-
physical-address, so sector data never crosses the 256-byte IPC limit.
- USB HID keyboard + mouse (usb-hid/): decode boot-protocol reports and publish
to the input service. A USB usage is already the input protocol's keycode.
- USB mass storage (usb-storage/): Bulk-Only Transport + transparent SCSI,
serving a block device under the new .block service id (block-protocol).
- device-manager matches USB interfaces to class drivers (usbDriverForIdentity).
- usb-abi / usb-ids made importable modules; add HID and mass-storage class
requests, packTriple, and a usb_device DeviceClass.
- Fix test/qemu_test.py on macOS: the QMP unix-socket path was built from the
deep worktree path and exceeded the 104-byte sun_path limit, so QEMU exited
before booting. It now lives under a short temp path.
Tests: usb-report, usb-hid, usb-storage pass under python3 test/qemu_test.py;
host units (usb-abi, usb-ids, hid-report, bulk-only-transport, scsi) green.
Time is a kernel concern in danos: the kernel owns the scheduling timer and
already exposes monotonic time via the clock/sleep/timer_bind syscalls, so a
userspace time service would be a redundant, slower path. This adds the generic
runtime.time module over those syscalls, retires the two demonstration drivers,
reorganizes the milestone docs, and makes the monotonic clock correct on Intel,
AMD, and inside any VM.
runtime.time (library/runtime/time.zig)
- Instant/Duration interface: now, sleep, spin, after, monotonicNanos, available
- a thin layer over system.clock/sleep/timerOnce; unit-tested arithmetic
Remove the demo drivers hpet and bus (a teaching example belongs in the docs,
not shipped in the tree)
- system/drivers/ now holds only real drivers: pci-bus, ps2-bus, usb-xhci-bus
- device-manager end-to-end test repointed to pci-bus (asserts on kernel state:
the process table and the device tree, not a racy serial marker)
- device_register containment moved to a new in-kernel `containment` test
- the driver-model worked example moved inline into docs/drivers.md
Reorganize milestone docs into topic docs
- m17-m18 / m19-m20 / m21 plans dissolved into process-lifecycle, device-manager,
discovery, and acpi docs; new docs/power.md and docs/timers.md; ~20 citations
repointed; plan docs deleted
TSC reliability (apic.zig, smp.zig, cpu.zig, kernel.zig)
- check the invariant-TSC bit (CPUID 0x80000007 EDX[8]) on Intel and AMD
- cross-core "warp" check at SMP bring-up, pairwise BSP<->AP as each core comes up
- fall back to the HPET clocksource when the TSC is not invariant (a bare VM) or
not synchronized (a warp), switched continuously so time never jumps
- boot log reports the outcome; new tsc-sync test exercises the TSC + warp path
Verified: zig build; zig build test; 60/60 QEMU cases (incl. new containment and
tsc-sync).
The debug-message refactor prefixed each service/driver line with its
source path (system/drivers/hpet:, ...) for readability, but two things
left main red: a 'drviers' typo in hpet.zig and pci-bus.zig, and five
kernel tests (init, hpet, pci-scan, device-manager, vfs-client-death)
that starts-with-matched the old short markers, which no longer sit at
the front of the prefixed lines.
Fix the typo, and convert the fragile starts-with matchers to substring
matching via a bufferHas helper — 'hpet: ok' now matches inside
'system/drivers/hpet: ok' regardless of prefix. Future-proof against
further prefix changes and harmless for the tests that already passed.
Suite 58/58.
Two related changes to make device identities legible in the boot log and in
the code that matches on them.
Logging: the pci-bus driver decodes each function's class/subclass/prog-IF
triple to human names (via the existing pci-class module), and the acpi
service appends each _HID's human name (via acpi-ids) to its report line. So
"class 0x01 (Mass Storage Controller) subclass 0x06 (Serial ATA Controller)
progif 0x01 (AHCI 1.0)" reads straight off the log when writing a driver.
Naming: a new coding standard ("Named values, not magic numbers") says a value
with meaning gets a name, prefer an enum for value sets. Applied:
- pci-class is refactored from u8-switch tables into a BaseClass enum plus
per-class SubClass/ProgIf enums with name() methods (the usb-ids shape). The
public className/subclassName/progIfName(u8...) API is unchanged, so the
hardware-byte decoders (pci-bus, the kernel dump) are untouched; output is
byte-identical.
- the device-manager builds the xHCI class triple from named parts instead of
a bare 0x0C0330.
- the acpi service's _CRS walk names its resource-descriptor tags as
SmallResourceType/LargeResourceType enums, and the _HID integer decode uses
the AML module's existing *_opcode constants (now re-exported from aml.zig)
rather than bare 0x0A/0xFF/... literals.
The root-hub scan logged the raw PORTSC port-speed class ("speed class 3").
Decode it to a human name — Low/Full/High/SuperSpeed/SuperSpeedPlus with the
USB generation and line rate — so the boot log says what enumerated on each
port, the USB analog of the pci-bus class line. This is the link speed only;
the device class/subclass/protocol needs descriptor reads (the USB track).
enumeratePci, addBars, pciConfigurationPtr, and the PciHeader struct are
deleted; the kernel seeds only the host bridge, and the ring-3 pci-bus
driver's reports are the sole source of PCI function nodes. The manager
matches PCI drivers from reported identity, deduped by registered device
id so a bus restart never double-spawns.
The flip did its job by exposing a latent SMP race: ring-3
device_register made the broker table concurrent for the first time, and
mmio_map read it lock-free — under load a torn resource length mapped
hpet's window wrong (its user fault) and underflowed r.len-1 into a
kernel integer-overflow panic. Fixed: the broker read in mmio_map (and
claim) runs under the big kernel lock, the arithmetic rejects
zero-length and wrapping windows cleanly, and pci-bus no longer registers
unimplemented size-0 BARs. driver-restart hammered 6x, suite 55/55.
Each function is registered under the bridge with the config-space slice
and BARs sized by the same all-ones probe the kernel uses — byte-for-byte
equal descriptors, so the idempotent register returns the kernel's
existing node ids during coexistence instead of duplicating the tree.
The bridge gained the 16-bit io_port aperture that functions' I/O BARs
need to pass containment. Reports carry the registered device_id, and
the pci-scan scenario drills a forced restart: kill the enumerator after
its reports, watch the respawn re-scan, and assert the broker's PCI node
count never grew. The usb-restart test trigger is pinned to the xHCI
reporter (pci-bus racing it to two reports used to steal the kill).
Harness hardening: failing cases preserve their serial logs; the heavy
scenarios run at 150s.
The manager matches the pci_host_bridge node and spawns pci-bus with the
bridge id as its assignment — hello, supervision, restart, all the M18
contract for free. The driver claims the bridge, maps the ECAM window
(resource 0) through the ordinary mmio grant, and repeats the kernel's
brute-force bus/device/function walk from user space. The pci-scan
scenario builds its expected marker from the kernel's own function count,
so the two enumerations must agree exactly — the equivalence that
licenses retiring the kernel walk in M19.3.
Applications ask the device manager for the tree (enumerate: a header
plus ChildEntry records) and subscribe to published add/remove events by
handing their endpoint over as the call's capability — the input-service
pattern; events are the same ChildAdded/ChildRemoved structs the bus
drivers send, one encoding in both directions. device-list is the first
client: it prints the tree, subscribes, and narrates the events through
a driver restart. The protocol's message maximum is capped at the
kernel's IPC MESSAGE_MAXIMUM (256 bytes, ten entries per reply; paging
joins the protocol when a tree outgrows one message). The startUserTask
debug print is gone: it wrote to serial unserialized against user-space
lines and sheared concurrent log markers in half — the root cause of the
scenario flakes.
child_added/child_removed join the device-manager protocol. The driver
maps its register BAR (resource 0 is the ECAM config space; the walk
starts at 1), reads CAPLENGTH and HCSPARAMS1, and reads one PORTSC per
port: the connect bit and speed class come straight from hardware, no
rings needed to see the devices. The manager mirrors reported children
keyed by (parent, port), remembers which instance reported each, and
prunes a dead reporter's children before deciding the restart — the
children describe protocol state that died with the process. The
usb-report scenario drives the whole loop: two QEMU devices reported,
reporter killed, children pruned, driver respawned with backoff, and the
new instance re-claims, re-scans, and re-reports.
The manager is now a harness service on the well-known .device_manager
endpoint. Every driver spawns supervised; drivers with an assignment must
hello (device-manager-protocol, versioned) within a deadline enforced by
a timer sweep. Exit reasons drive the restart decision: clean exits stay
down, faults restart with 300/600/1200ms backoff, and three fast deaths
mark a driver failed instead of respawning forever. usb-xhci-bus is the
first conforming driver; the crash-test fixture claims a device, hellos,
and faults on purpose — each respawn re-proving claim release on death
through the manager's own path. maximum_tasks grows 16 -> 32: the
initial-ramdisk sweep (15 binaries at once) was intermittently
overflowing the static pool.
Turn acpi-ids.zig's flat name table into a HardwareId enum modeled on
ps2-library's Port: one entry() switch holds the registry (variant ->
_HID string + human-readable name), with hid(), description(), and
fromHid() methods. The free description(hid) lookup the kernel's
device-tree dump uses survives, implemented over the enum, and a new
test round-trips every variant through fromHid.
Callers now name the device instead of quoting its id:
- ps2-library's DeviceType.hid() and ps2-bus's descriptor lookups use
HardwareId.ps2_keyboard / .ps2_mouse.
- device-manager's driverFor parses the HID once with fromHid and
switches on named values.
- acpi.zig's isPciRootNode carried the same ids twice, as strings and
as packed-EISA integers (0x030AD041/0x080AD041); both branches now
decode to the string form and answer through one isPciRootHid helper
using .pci_bus / .pci_express_root_bridge.
- build.zig threads the acpi-ids module (previously kernel-only) into
every user binary, like xkeyboard-config.
Add an AttachStatus enum to ps2-library.zig for AttachReply.status,
distinguishing the three failure causes handleAttach previously
collapsed into a bare -1: invalid_request (message too short),
missing_endpoint (no capability passed), and no_such_device (no port
identified the requested device type). The keyboard and mouse drivers
check against AttachStatus.ok rather than a literal 0.
Replace the mouse driver's synthetic stream with the real path, the
way the keyboard was wired:
- The auxiliary port's IRQ12 is enumerated on the mouse's own ACPI
node (PNP0F13), and the kernel only lets a device's claimer bind or
ack its IRQs — so ps2-bus now claims that node alongside the
controller whenever port 2 carries a device, binds IRQ12 to its one
endpoint, and re-arms whichever line the notification's badge names.
The forwarding loop already routed auxiliary bytes by status bit 5.
- mouse-packet.zig (new, pure, host-tested): three-byte stream-mode
packet assembly — bit-3 sync with resynchronization, ACK/BAT bytes
dropped at packet start, nine-bit two's-complement movement,
overflow packets discarded, and PS/2 positive-Y-up converted to the
screen convention (positive down).
- mouse.zig mirrors the keyboard driver: no hardware claim, attaches
to the bus as its mouse, and publishes button_down/button_up per
changed button plus motion events with the pressed-button mask.
Verified end to end in QEMU via monitor mouse_move/mouse_button:
motion round-trips in screen coordinates, buttons transition with the
right mask, and keyboard events keep flowing alongside. The follow-up
is the IntelliMouse magic-knock for a scroll wheel (four-byte packets)
and scroll events.
Replace the keyboard driver's synthetic stream with the real path:
- ps2-bus binds IRQ1 (interrupt bits set only after the bind), drains
port 0x60 on each interrupt, and forwards every byte to the attached
child driver over async ipc_send, routed by the status register's
auxiliary-output bit. Children attach via the new well-known ps2_bus
service, handing over their endpoint as a capability.
- scancode.zig (new, pure, host-tested): scancode set 2 -> USB HID
usage decoding (F0/E0/E1 prefix state machine) plus keyboard state —
pressed-key bitmap, typematic-repeat classification, modifier and
caps-lock tracking.
- keyboard.zig decodes the forwarded stream and publishes real
key_down/key_press/key_up events, filling key_press characters via
xkeyboard-config (layout from argv[2], default us) and synthesizing
ASCII control characters for Enter/Tab/Backspace/Escape.
- protocol.zig names the full HID usage set in Keycode; build.zig
threads the xkeyboard-config module into user binaries.
Verified end to end in QEMU via monitor sendkey: shift, caps lock,
and control-character synthesis all decode correctly. The mouse
driver still publishes its synthetic stream; attaching it to the
bus the same way is the follow-up.
Extend the input service beyond the keyboard so mouse and joystick/gamepad
drivers can broadcast too, with per-device publish and subscribe methods.
- protocol: KeyEvent joins MouseEvent (motion/buttons/scroll) and
JoystickEvent (axes/buttons), all carried in a common InputEvent envelope
tagged with a DeviceKind. A subscribe request carries a device_mask, so a
subscriber names the classes it wants and the service routes each event only
to interested subscribers (a mouse-only listener never wakes for keystrokes).
- runtime: per-device publish methods (publishKeyboardEvent/publishMouseEvent/
publishJoystickEvent) and subscribe helpers (subscribeKeyboard/Mouse/Joystick,
each typed, plus subscribe(mask)/subscribeAll returning the tagged envelope).
- service: subscriber table gains a device_mask; broadcast routes by the
event's device class.
- mouse driver now publishes (synthetic) mouse events like the keyboard driver;
input-source cycles all three classes; input-test subscribes to all and only
emits its "ok" marker once it has received one of each class — so the passing
test proves per-device routing, not just delivery. Real HID decoding stays a
follow-up.
No kernel changes: ipc_send is generic and the 36-byte InputEvent fits its
64-byte payload. Full QEMU suite 48/48; serial log confirms keyboard, mouse,
and joystick all reach one subscription.
Programs can now subscribe to keyboard events (key_down/key_up/key_press)
and drivers can broadcast them, through a new user-space input service.
The delivery model is forced by danos IPC: a synchronous rendezvous holds
one pending reply, so a server cannot park N subscribers blocked in a
"wait for next event" call — delivery must be push. But a synchronous push
has no timeout and the kernel never wakes a sender parked on a dead peer's
endpoint, so one dying subscriber would hang all input. So this lands the
roadmap's planned asynchronous buffered send and builds the service on it:
- ipc_send (syscall 26): non-blocking post to an endpoint's bounded payload
ring, delivered through reply_wait as a buffered message (notify_message_bit).
A full ring drops the oldest. It can never hang on a dead/slow peer.
- input-protocol + runtime.input helpers (subscribe/next, connectSource/
publish) — the first real consumer of M13 capability passing: a subscriber
hands the service its own endpoint as a capability.
- input service (fan-out via ipc_send, dead-subscriber pruning), a synthetic
input-source, and input-test; the ps2-bus keyboard driver publishes to it.
Real IRQ1 scancode decoding (which must live in the bus, the PNP0303 owner)
is a documented follow-up; the source is synthetic for now.
- build/init wiring, an `input` QEMU case, and docs/input.md.
Full QEMU suite 48/48, including the new input case and every IPC/endpoint
regression (ipc, ipc-call, ipc-cap, vfs, hpet, bus, irqfree).
Daniel Samson
Daniel Samson