library/device/pci is now the complete generic floor a leaf PCI driver
needs, instead of just what virtio-gpu used:
- pci-class: capability IDs, MSI/MSI-X/power-management/PCI-Express
register layouts, extended-capability header decode (host-tested),
per-bit command constants, remaining header offsets.
- pci.Function: header accessors, disableBusMaster + interrupt-disable
helpers, findCapability, programMsi/disableMsi, MsiX vector-table
struct, ensurePowerStateD0, functionLevelReset (BAR save/restore),
extended-capability iterator.
Proven by the new pci-caps QEMU case: a pci-cap-test fixture claims an
extra e1000e (PM+MSI+PCIe+MSI-X, no danos driver) and readback-verifies
every surface, including the first driver-side use of msi_bind.
usb-xhci-bus converts from 8 ms event-ring polling to message-signalled
interrupts: plain MSI where offered (real Intel xHC), MSI-X entry 0
otherwise (qemu-xhci has no MSI capability), byte-identical polling as
fallback. The timer survives as a 250 ms port-reconcile/lost-edge tick —
real-hardware USB2 hub debounce still needs it. MSI setup runs BEFORE
controller bring-up: QEMU's xhci only registers the MSI-X vector as used
when IMAN.IE is written while MSI-X is already enabled; interrupts are
silently dropped otherwise (real hardware does not care about the order).
101/101 QEMU cases green; real-hardware smoke passed (mouse works,
boot 2026-07-23T174805Z, plain-MSI branch, vector 33).
A plain `zig build` was baking all 11 /test/system/services fixtures into the
production image. Split the bundle: the production set always, the userspace test
fixtures only when -Dtest-case is set — which the QEMU harness passes for every
scenario, exactly when the fixtures are needed on the boot volume. So `zig build`
with no options now ships a clean image (no test binaries in the manifest,
capsule, or FAT tree) and assumes neither -Dtest-case nor -Ddiagnose (the
diagnose display-omission was already gated by the init.csv selection).
Verified: a plain build has zero /test entries in the manifest; -Dtest-case=smoke
includes all fixtures; the args/vfs/device-list QEMU scenarios still pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Each bus's discovery line now prints the device's would-be /etc/devices.csv row
(bus, base, class, prog_if, vendor, device, subsystem / hid) in uppercase hex,
followed by the human-readable names — so a row for a new driver reads straight
off the boot log, for pci, usb, and acpi alike.
- pci-bus: logFunction moved to registerAndReport (where vendor/device/subsystem
are read from config space) and reformatted to columns + names; subsystem
prints '*' when the function has none. Read unclaimed via the bridge ECAM, so
no per-function claim is needed.
- usb-xhci-bus, acpi: the same column framing on their existing discovery lines.
- qemu_test.py: the acpi-ps2 and acpi-report regexes updated to the new ACPI
format (both verified passing in QEMU).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Replace init's hardcoded boot_services array with an authoritative,
human-readable service list read at boot, mirroring /etc/devices.csv. Each row
is a service binary path followed by its argv; startup order is file order,
shutdown the reverse. There is no hardcoded fallback — a missing file starts
nothing (the no-ramdisk isolation behavior).
- library/csv: shared CSV helpers (comment strip, field iteration) with unit
tests; device-registry is refactored onto them so both /etc/*.csv files parse
through one place.
- init reads /etc/init.csv into fixed-max static tables (the same pattern as the
device registry) and passes each row's argv straight to spawnSupervised. This
also makes boot-time modes (e.g. device-manager test-usb-restart) expressible
as data rather than hardcoded.
- Diagnose mode (-Ddiagnose omits the display stack) becomes build-time file
selection between etc/init.csv and etc/init-diagnose.csv, so init carries no
comptime service logic; the diagnose build option is dropped from init.
- build.zig: csv module wired; /etc/init.csv bundled into the initrd; the
device-registry tests move to a dedicated block since they now import csv.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
Replace the three hardcoded switch tables (pciDriverForIdentity, hidDriverFor,
usbDriverForIdentity) with an authoritative, human-readable device registry the
manager reads at boot. Matching is most-specific-wins across
base/subclass/prog_if/vendor/device/subsystem/hid, so a precise vendor:device
rule and a generic class rule coexist; an unmatched device is logged, never
guessed. This resolves docs' "matching stays code until the third bus".
- ABI: child_added and DeviceDescriptor gain vendor/device/subsystem; child_added
gains a bus discriminator (BusKind) so PCI and USB class triples match against
the right namespace.
- pci-bus reads vendor/device (config 0x00) and subsystem (0x2C, type-0) and
reports them.
- library/device/registry: freestanding CSV parser + matchDriver() with
specificity scoring; 5 unit tests wired into `zig build test`.
- etc/devices.csv bundled into the initrd; the kernel serves /etc directly, so
the manager reads it before any filesystem service is up (fat starts later).
- virtio-gpu: drop the now-redundant post-spawn 1AF4:1050 re-confirm, since the
registry binds this driver by exact identity.
- Remove the orphaned system/drivers/display driver (unreferenced by build or
registry).
- docs: new devices-csv.md; device-manager.md "matching stays code" resolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJqSiLLchDUUCoXn5jsiwd
ReleaseSafe kept full DWARF/symtab because nothing set strip on the
kernel module, bloating the flashable image to 2.4MB (~1.4MB of which
was debug info). Strip for every optimize mode except Debug, so
release builds shrink to ~900KB while debug builds keep symbols.
The 11 QEMU-suite fixtures lived mixed into system/services/ with their
binaries bundled at /system/tests/<name>. Now the repo path is the boot
path, like every real service: test/system/services/<name>. fat-test
moves out of the fat server's directory into its own; display-demo
stays a boot service.
- kernel VFS: setInitialRamdisk derives one read-only initrd mount per
top-level tree named by the ramdisk entry paths (/system, /test),
registers ancestors generically with self-parented roots, and refuses
backend shadowing of any initrd tree
- EFI loader: the fallback walk also enumerates \test (optional — a
volume without fixtures still boots); manifest and capsule unchanged
- path literals: vfs-test self-open + create probe, process-test
process_enumerate matches, the args-echo argv[0] expectation; the
kvfs case now covers the /test root end to end
- docs: DFHS /test rows, tree diagrams, loader prose, and the location
convention gain the third home; fixed the input-source link
100/100 QEMU cases pass.
An adversarial review of the reorg surfaced two doc inaccuracies:
- timers.md: a `time` code example still did `@import("runtime").time`, which
no longer compiles — the runtime aggregator was deleted. Now `@import("time")`.
(Reorg regression: the sweep rewrote `runtime.x` member access but not the
`@import("runtime")` form inside a fenced code block.)
- sysv.md: `callconv(system.kernel_abi)` named a non-existent `system` module;
`kernel_abi` lives in the boot-handoff contract (system/boot-handoff.zig),
imported as `boot_handoff`. Now `boot_handoff.kernel_abi`. (Pre-existing bug —
"system" was the file-path prefix mistaken for a module namespace — found and
fixed opportunistically.)
Rewrote the repository-layout and driver-model docs to describe the new tree —
library/kernel (the kernel32-style system library: syscall surface split by
concern), library/device (mmio/model/pci/usb/acpi/driver/block), library/client
(display, input service clients), library/protocol (wire contracts) — and swept
the reference docs off the retired runtime shim:
runtime.system.* -> logging.* / time.* / process.* / memory.*
runtime.dma.* / runtime.shared_memory.* / runtime.allocator -> memory.*
runtime.ipc/process/time/service/Thread/block/display/input -> the module name
runtime.device / runtime.device_manager -> driver
library/runtime/<file>.zig -> its new home (kernel/ client/ device/)
docs/README.md (repository layout + source map), docs/driver-model.md (the module
graph + import lists), and the concern/reference docs (ipc, threading, timers,
logging, power, process-lifecycle, device-manager, display, vdso, sysv, drivers,
input, vfs-protocol, coding-standards, ...) now reflect the split. "runtime" that
remains is the userspace-library *concept*, which is still accurate.
The historical plan docs (display-plan, display-v2-plan, threading-plan) and the
zig-self-hosting design note are left as point-in-time snapshots.
With every consumer migrated (C2), nothing imports `runtime` or `system` any
longer. Removed both shim files and their build.zig module definitions, dropped
the `runtime` entry from `default_imports`, and rewrote the per-binary root shim
(library/kernel/root.zig) to import `start` (_start + panic) and `logging`
(std_options) directly instead of through `runtime`.
addUserBinary/addThreadedUserBinary/addUserBinaryImpl lose their `runtime_module`
parameter; the root module now pulls the two modules it needs out of
`default_imports` via a small findImport helper.
The runtime dumping ground is gone: the userspace library is library/kernel
(concern modules), library/device (driver/pci/usb/block/model/mmio/acpi data),
library/client (display, input service clients), and library/protocol (wire
contracts). Consumers @import concern modules by name.
Verified: zig build, zig build test, and QEMU (smoke, thread-spawn, thread-mutex,
logger, display-native, device-manager, usb-storage, input).
Every user binary and the two device-logic library modules (pci, usb) now
`@import` the concern modules directly instead of aliasing through `runtime`:
runtime.ipc/process/time/service/input/block/display -> @import("<module>")
runtime.device / runtime.device_manager -> @import("driver")
runtime.fs -> @import("file-system")
runtime.Thread -> @import("thread").Thread
runtime.system.{write,writeRecord,klog*} -> logging.*
runtime.system.{sleep,timerOnce,wallClock,clock} -> time.*
runtime.system.{spawn*,kill,exit,yield,processes,...}-> process.*
runtime.system.{mmap,munmap,PROT_*} -> memory.*
runtime.dma.* / runtime.shared_memory.* / runtime.allocator -> memory.*
Each consumer keeps its own alias name (e.g. `const device = @import("driver")`),
so call sites are unchanged and there are no collisions with local `driver`
variables. build.zig now injects the concern modules into every user binary via
`default_imports`; pci/usb module import lists were updated to match.
The `runtime` and `system` shims remain for one more step (root.zig still uses
runtime); they are deleted in C5. Nothing but root.zig imports `runtime` now.
Verified: zig build, zig build test, and 17 QEMU cases (smoke, device-manager,
logger, fat-mount, fat-mutations, usb-storage, usb-hid, display-native,
virtio-gpu, input, thread-spawn, thread-mutex, process-kill, shared-memory,
driver-restart, acpi-ps2, pci-scan).
The one giant `runtime` module (with a `system.zig` that was itself a dumping
ground of unrelated syscalls) is split into directly-importable, flat concern
modules under library/kernel/:
system-call ipc memory process thread time logging file-system service start
(+ the device/service clients: device, device-manager, block, display, input)
system.zig is dissolved — its functions moved to their concern home (mmap ->
memory, spawn/kill/exit -> process, sleep/clock -> time, write/klog -> logging,
fs* -> file-system). `memory` merges heap+dma+shared-memory behind one flat API
(memory.allocator/dmaAlloc/sharedCreate/mmap), keeping heap's state and malloc
export single. The memory<->thread dependency cycle (heap needs Thread.Mutex,
thread needs mmap) is broken by having thread allocate its own stack via the raw
mmap syscall, so the module graph is a DAG.
This is the atomic step: all 42 internal cross-imports flip from relative to
module imports at once. `runtime.zig` and `system.zig` become thin re-export
SHIMS so the ~38 consumers keep compiling on `runtime.*` untouched; they migrate
to direct imports in C2, after which the shims are deleted (C5).
zig build + zig build test green; 14 QEMU cases pass (smoke, process,
process-kill, thread-spawn/join, logger, vfs, fat-mount, display-native,
usb-storage, virtio-gpu, device-manager, input, power-button).
First step of splitting the runtime dumping ground. Pure directory rename
(git mv library/runtime library/kernel) + the six build.zig path references
repointed. The module is still named "runtime" for now; the next commits split
it into concern modules (ipc, memory, process, time, logging, file-system, ...),
dissolve system.zig, and delete the runtime aggregator.
The library/kernel name follows the kernel32 model: it is the userspace library
that wraps the private kernel ABI, distinct from system/kernel/ (the kernel).
zig build green.
Follow-up cleanup of the just-moved kernel device code:
- power.zig -> acpi.zig. Its reboot() is built entirely on the FADT reset
register (acpi.power_information) plus the legacy 0xCF9/8042 fallbacks — it is
ACPI reboot, so it becomes acpi.reboot (the "P" in ACPI). platform.reboot
still delegates; soft-off/S5 stays the ring-3 acpi service's job as before.
- device-tree.zig -> fdt.zig. It is a discovery *backend* (the ARM/FDT parser,
a sibling of acpi.zig), not part of the model — renaming it to its actual
subject kills the confusing device-tree / device-model DeviceTree name clash
and makes acpi.zig + fdt.zig read as the two parallel backends.
- Flatten: device-model.zig, acpi.zig, fdt.zig, platform.zig move out of the
system/kernel/devices/ subdir up into system/kernel/, joining devices-broker.zig
(already flat). The kernel dir is a flat pile by convention (only architecture/
is a subdir), so the subdir — and its poor "devices" name — is gone.
Pure restructure; "platform" module name unchanged, all cross-file deps are
relative siblings that moved together. zig build + test green; smoke, discovery,
acpi-parse, acpi-ps2, acpi-report, power-button, orderly-shutdown, reboot pass.
With the shared device data (device-abi, pci-class, usb-abi, usb-ids, acpi-ids,
aml) now in library/device/, what remained in system/devices/ was purely
kernel-internal: the firmware-discovery machinery and the rich pointer-based
device model (platform, device-model, acpi, device-tree, power), reached only
through the "platform" module by three kernel files. It belongs with the kernel.
Move it to system/kernel/devices/. A pure relocation: the "platform" module
name is unchanged and every cross-dir dependency is a module import, so only the
one b.path and some comments move. The top-level split is now clean —
system/kernel/ is the kernel, library/device/ the shared device libraries,
system/{drivers,services} the userspace. The runtime /system/devices concept
(the virtual device tree) is unaffected; system/kernel/devices/ is its
implementation.
Also fixes two comment refs that still pointed device-abi at its pre-Wave-1a
home (system/devices/) — it lives at library/device/model/ now.
zig build + test green; smoke, discovery, acpi-parse, acpi-ps2, acpi-report pass.
mmio is device-driver code, so it joins the other domains under
library/device/mmio/ (module name "mmio" unchanged — a pure relocation, only
the build paths move). And its abbreviated function names are spelled out per
docs/coding-standards.md:
read -> readRegister mb -> memoryBarrier
write -> writeRegister rmb -> readMemoryBarrier
wmb -> writeMemoryBarrier
All call sites updated (virtio-gpu, usb-xhci-library, pci.Function); the two
display-driver placeholders import mmio but use nothing, so they're untouched.
Docs (driver-model graph, README layout, drivers.md, the FHS note) follow the
new path and names.
zig build + test green; virtio-gpu, display-native, display-reattach, usb-hid,
usb-hub, usb-storage, pci-scan pass.
Finish the direct-import convention: a protocol is now aliased to its module
name in snake_case in every file that imports one — consumers and the runtime
client wrappers alike. This kills the last of the alias variety (the generic
`protocol`, plus `power` and `transfer`), so `device_manager_protocol.Hello`
means the same thing everywhere and grepping a module name finds all its uses.
Pure rename (identifier uses only; prose references left untouched via a
guarded pattern). 14 files, 299/299 lines.
zig build + test green; 16 QEMU cases pass (smoke, device-manager,
driver-restart, device-list, pci-scan, usb-hid, usb-storage, display-service,
display-native, display-reattach, input, acpi-ps2, vfs, fat-mount,
power-button, orderly-shutdown).
Update the docs to match the reorganized tree:
- driver-model.md: the "Families" graph now shows library/device/<domain>/
(data + logic split) and library/protocol/ instead of the old bus/ + proto/
sketch; add the microkernel note (the kernel's one library/ import is the
device-abi data module) and the rule that a driver's hardware command set is
not a service-seam protocol.
- README.md "Repository layout": device-abi and the protocols move out of
system/ into library/device/ and library/protocol/; rewrite the "public
interface as a module" paragraph around the protocol tree and the device
domains; fix the source-map paths.
- Sweep every remaining stale path reference across docs/ to the moved files.
- Fix vfs-protocol.zig's own header (the standalone VFS server retired; the fat
server is the backend today).
zig build + test green.
power-protocol -> library/protocol/power/power-protocol.zig. Its two consumers
(init and the acpi discovery service) import it directly; the
runtime.power_protocol re-export and the runtime module import are dropped (no
runtime client speaks it). The now-empty system/services/power/ is removed.
This completes library/protocol/: every driver<->service wire contract lives
there and is imported by module name, no protocol is re-exported through
runtime, and the alias chaos (dp/sp, mixed protocol/<x>_protocol) is resolved.
virtio-gpu-protocol stays a driver-private relative import (a hardware command
set, not a driver<->service seam — like virtio-pci.zig beside it).
zig build + test green; power-button, orderly-shutdown pass.
display-protocol -> library/protocol/display/, scanout-protocol ->
library/protocol/scanout/. Consumers import both directly, killing the cryptic
dp/sp aliases in virtio-gpu (now display_protocol / scanout_protocol) and the
runtime.display_protocol / runtime.scanout_protocol re-exports. runtime.display
(the client) still imports display-protocol by name; scanout has no runtime
client, so its runtime module import is dropped too.
zig build + test green; display-service, virtio-gpu, display-native,
display-reattach pass.
device-manager-protocol -> library/protocol/device-manager/. Its consumers
(pci-bus, usb-xhci-bus, the acpi discovery service, device-manager, device-list,
crash-test, and the not-yet-built intel-integrated display sub-driver) import the
module directly instead of through runtime.device_manager_protocol, which is
deleted. runtime.device_manager (the hello client) already imported the module
by name and is unchanged.
zig build + test green; device-manager, driver-restart, device-list, pci-scan pass.
input-protocol -> library/protocol/input/input-protocol.zig. Its five
consumers (usb-hid keyboard/mouse, ps2 keyboard/mouse, the input service) now
import the module directly and alias it input_protocol, replacing the
runtime.input_protocol re-export (deleted) and the inconsistent local `protocol`
aliases. runtime.input (the client) still imports the module by name.
zig build + test green; input, acpi-ps2, usb-hid pass.
vfs-protocol -> library/protocol/vfs/vfs-protocol.zig. fat, its only consumer,
now imports the module directly (const vfs_protocol = @import("vfs-protocol"))
instead of through runtime.vfs_protocol, and the runtime re-export is deleted.
The runtime's own client (runtime.fs) still imports the module by name.
zig build + test green; vfs, fat-mount pass.
Start the protocol tree: wire protocols move to library/protocol/<name>/,
entry file <name>-protocol.zig, module name unchanged. block and usb-transfer
are pure moves — every consumer already imports them by module name, so only
the b.addModule paths change and the empty system/services/block/ is removed.
block-protocol -> library/protocol/block/block-protocol.zig
usb-transfer-protocol -> library/protocol/usb-transfer/usb-transfer-protocol.zig
zig build + test green.
The USB class-driver transfer client was library/runtime/usb.zig, re-exported
as runtime.usb — which compiled the USB client and usb-transfer-protocol into
every user binary (init, fat, the compositor…), none of which speak USB. It is
bus-family logic, not core runtime.
Move it to its domain home, library/device/usb/usb.zig (module "usb"), alongside
the usb-abi and usb-ids data modules; fix its internal imports to go through the
runtime module; and re-export usb.abi / usb.ids so a class driver reaches the
whole USB domain through one import. runtime.usb and runtime's
usb-transfer-protocol import are removed; the three class drivers
(usb-hid keyboard/mouse, usb-storage) import module "usb" directly.
zig build + test green; usb-hid, usb-storage pass.
New pci logic module (library/device/pci/pci.zig): a device driver's view of
the one PCI function it has claimed — Function.map (config space = resource 0),
vendorId/deviceId/command/status, enableMemoryAndBusMaster, mapBar (BAR decode
+ resource correlation + mmio_map, cached), and a capabilities() iterator. The
generic PCI mechanics every leaf PCI driver used to re-derive inline.
The config-space layout it needs — header offsets, the command MEM|bus-master
bits, the status capabilities-list bit, the capability-pointer mask, and the
BAR bit fields — is named in the pci-class data module (a "Configuration-space
layout" section), so the bus enumerator can share the same constants later.
virtio-gpu is the first consumer: its inline mapBar + walkCapabilities + config
header reads are gone, replaced by pci.Function; only the virtio-specific
cfg_type dispatch (and the virtio common-config cfgRead/cfgWrite, which are NOT
PCI config space) stay in the driver. The generic display driver will use the
same module. pci-bus's enumerator (arbitrary-function probing) is untouched.
zig build + test green; virtio-gpu, display-native, display-reattach, pci-scan pass.
The kernel's only dependency on pci-class and acpi-ids was cosmetic: the
boot-time device-tree dump (DeviceTree.dump) decoded class codes and _HID
strings to human names. That is device decoding — a user-space concern in a
microkernel, and the last thing pinning two userspace taxonomies into the
kernel's compile.
dumpNode now prints raw values (the class tag, the raw _HID string, and the
packed PCI class code as hex); a user-space tool that owns the taxonomies
(the device manager already imports them) can pretty-print when wanted. The
pci-class / acpi-ids imports are dropped from device-model.zig and from the
platform module.
The kernel's sole remaining library/device/ import is now device-abi (pure
data — the device-model types the broker marshals across the syscall
boundary), the one intentional kernel->library crossing.
zig build + test green; smoke, discovery, acpi-parse, acpi-ps2 pass.
First step of the device-code reorganization (plan: group device code by
domain, split each domain into a shareable data module + a logic module).
This moves the pure-data modules — the enums, wire types, and taxonomies
that any layer including the kernel can import cheaply — out of
system/devices/ and into their domain home:
device-abi -> library/device/model/device-abi.zig
pci-class -> library/device/pci/pci-class.zig
usb-abi -> library/device/usb/usb-abi.zig
usb-ids -> library/device/usb/usb-ids.zig
acpi-ids -> library/device/acpi/acpi-ids.zig
aml/ -> library/device/acpi/aml/
Module names are unchanged, so this is a pure file move: only the
b.addModule paths and the host-test file list in build.zig change; no
importer is touched. system/devices/ now holds only the kernel-internal
device model (device-model, platform, acpi, device-tree, power).
The device *logic* (the pci Function helper, the usb transfer client) and
the kernel's cosmetic taxonomy dependency are handled in following commits.
zig build + zig build test green.
Every supervised driver owes the device manager a hello at startup, and
the retry-lookup-call-check for it had been copied into five drivers:
usb.helloManager (misfiled in the USB client) plus hand-rolled twins in
display, virtio-gpu, pci-bus, and usb-xhci-bus.
Extract it once as a runtime client, runtime.device_manager.hello(role,
device_id) ?Handle — returns the manager endpoint (bus drivers keep it to
report children through), null when there is no manager or it refused the
handshake, and logs the outcome itself so each call site is one line.
Also correct two roles while collapsing their calls: virtio-gpu and the
display driver each claim one PCI function and report no children, so they
are Role.device, not Role.bus. The manager ignores role today, so this is
cosmetic, but it matches the protocol's own definition (bus = reports
children via child_added).
Behavior-preserving otherwise: pci-bus and usb-xhci-bus move their hello
logging from raw serial writes to std.log, which the kernel renders with
the same "<path>: " prefix, so driver-restart still matches
"usb-xhci-bus: hello acknowledged". zig build clean; 8 QEMU cases pass
(driver-restart, pci-scan, usb-hid, usb-storage, virtio-gpu,
display-reattach, device-list, device-manager).
A fault in any thread, exit from any thread, or process_kill on any member id
now takes down the entire thread group; only a worker's voluntary thread_exit
stays per-thread. Leader id (tgid-style), dying latch on the address space,
group notification at space destruction, shared-memory mapping references,
per-space DMA/shm arena cursors, and the shm/DMA page-table walks brought
under the big kernel lock. Design + red-team + review in
docs/shared-fate-plan.md. QEMU suite: 100/100.
The adversarial review of the branch confirmed the big one: the shm/DMA
page-table walks and their pmm/heap calls ran outside the big kernel lock —
pre-existing, but fatal once the per-space cursors invited sibling threads to
race them (two concurrent creates could orphan a page table: one thread's
region silently unmapped, the frame leaked — a plausible root for the
long-standing intermittent AP ring-3 fault at the shm base). All three paths
now follow the mmap discipline: allocation, object build, record, and handle
under one lock hold with full rollback; the map itself per-page under brief
holds; dma_free's translate/unmap/free per-page likewise.
Contract edges from the same review: thread_spawn into a dying group returns
-ESRCH (was generic -1); process_kill during the condemned window answers
from the latch's stashed supervisor (0 or -EPERM, was -ESRCH once the leader
slot was reaped); exit derives the group reason from its own argument rather
than the racy exit_code global; a worker's thread_exit no longer overwrites a
concurrent group-kill stamp; checkGroupDead now asserts exactly-one
notification via the drained ring. Full suite: 100/100.
Eight new QEMU cases (thread-fault-group, kill-threaded-group,
kill-via-worker-tid, racing-triggers, exit-group, leader-thread-exit,
thread-exit-solo, shm-mapping-ref) driving seven new thread-test modes; a
shared checkGroupDead asserts the contract everywhere: one notification,
badged with the leader, reason on the leader's record, no member listed,
claims released first.
The shm-mapping-ref case flushed out the per-task DMA/shared-memory arena
cursor bug directly (a sibling's regions mapped over the worker's), so both
cursors moved to the AddressSpaceRef like the mmap/MMIO cursors before them
(threading M7 pattern). Runtime gains Thread.tryExitCurrent for the leader
-EPERM refusal path. Docs updated: threading.md's shared-fate gap is closed,
process-management.md and process-lifecycle.md describe the leader re-key,
plan status = implemented. Full suite: 100/100.
system.img was only mentioned in passing (efi.md, system-requirements.md);
its format, builder, fallback chain, and kernel-side life were spread across
pack-system-image.py, build.zig, efi.zig, initial-ramdisk.zig, and vfs.zig.
New dedicated page covers all of it, cross-linked from every prior mention
and added to the docs index.
Each address space that maps a shared-memory region now holds its own
reference, recorded on the AddressSpaceRef and dropped when the space is
destroyed — so 'last reference' means no handles AND no mappings, and a
region's frames can no longer be freed out from under a sibling thread (or
any other live mapper) when the handle-holding task dies. The group-death
notification still posts after every mapping release. (docs/shared-fate-plan.md M3)
All process deaths (exit from any thread, ring-3 fault, process_kill) now kill
the whole thread group via killGroupLocked: latch the AddressSpaceRef as dying
(refusing new members, closing the thread_spawn escape), stamp every member
(leader carries the group reason — the record the supervisor reads), reap
parked members to fixpoint, condemn running ones. The leader's exit
notification and subscriber broadcast move to the group-death moment — the
last address-space reference drop — via scheduler.group_exit_hook, which
re-stamps the leader's exit record first. Leader thread_exit is refused with
-EPERM. kill_pending is atomic; exit_reason and fault_kill_count writes moved
under the big kernel lock. (docs/shared-fate-plan.md M2)
Every task carries its process leader's id (main task: own id; threads:
copied from the spawner; kernel tasks: 0, never followed). process_kill and
process_signal resolve any member id to the leader and authorize against the
leader's supervisor, making both capabilities per-process. ProcessDescriptor
gains the leader field. No fan-out yet (docs/shared-fate-plan.md M1).
Every doc verified claim-by-claim against the code by parallel audit agents,
then fixed and adversarially re-verified. Two waves of staleness corrected:
the originally audited findings (higher-half boot handoff, kernel VFS
takeover, fault isolation + claim release + driver restart, AML/S5 moving to
ring 3, threading's shipped design, USB+FAT landing) and a second pass of
adjacent claims the verifiers caught (smp.md 'not built yet' intro,
system-requirements' PS/2-only and no-storage claims, halting.md's red-panic
and no-IDT text, testing.md's serial mirroring, router-era vfs-protocol
wording, capsule-first boot loading).
threading.md now documents the shared-fate gap explicitly: the design says a
process dies whole, the kernel today kills only the offending thread.
Also fixes three stale code comments (isr.s exceptionHandler, acpi.zig
sleepValue, build.zig boot-volume) — comments only, no behavior change.
Verified every deferred item in the nine docs with a what's-next section and
marked what has since landed (reaper, per-process CR3, higher-half kernel,
kernel heap, contiguous frame alloc, RSDP capture, cap-passing, driver restart
with backoff, PS/2 keyboard) while keeping the genuinely open items. Also
corrects interrupts.md's claim that the keyboard skipped the IO-APIC, and
updates scheduler.zig/heap.zig comments that predated the reaper and the big
kernel lock.
Rename the file to match current naming conventions and update all
cross-references throughout the documentation. Update the content to
reflect the current state of the codebase:
- Module is now called "architecture", not "arch"
- Directory is system/kernel/architecture/, not system/kernel/arch/
- Generic kernel file is kernel.zig, not main.zig
- Updated file listing for x86_64 architecture module (added smp.zig,
per-cpu.zig, ioapic.zig, trampoline.s)
- Updated function list in cpu.zig to include enterUser/userExit and
address-space management
- Clarified that _start lives in architecture-specific isr.s
- Expanded architecture-specific vs generic comparison table with
actual current modules
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Directory scans and FAT-chain walks re-read the same sectors constantly:
resolving many paths under one directory (a logging burst opening dozens
of files under /var/log/<stamp>/) re-read that directory's sectors and the
FAT from the device every time. Add a 16-line write-through cache under the
single-sector blockRead/blockWrite path, keyed by filesystem-relative LBA
with round-robin eviction, so repeated metadata reads come from RAM instead
of a USB round trip each.
Disciplines that keep it safe:
- Write-through: every write reaches the device immediately and refreshes
the cache, so it never holds dirty-only data — the crash-safe
data->FAT->directory write order and the flush-on-close are unchanged.
- Bulk file data (the B5a multi-sector run path) bypasses the cache and a
run write invalidates any overlapping cached sector, so metadata that
shares a range can never go stale.
- Cluster-zeroing writes uncached (write-once bulk that would only evict
live metadata), invalidating any stale copy.
A new engine test proves a repeated resolve does zero device reads and that
a write is coherent both in-cache and against a cold-mounted filesystem
(it really reached the device).
Full QEMU suite green (the one intermittent `logger` miss is the documented
pre-existing AP ring-3 fault: 16/16 logger reruns pass on this change and
every other storage case is green; the fat engine is single-threaded and
the cache is bounded global state, so it cannot itself fault intermittently).
The FAT engine read and wrote one sector per device command — one SCSI
READ(10)/WRITE(10) over USB Bulk-Only Transport per 512 bytes, so every
file read, log write, and cluster fill paid a full USB round trip per
sector. The lower stack (runtime.block, the block protocol, usb-storage's
read10/write10 + count*block_size data stage) already carried a
multi-sector count; only the engine's BlockDevice interface and IpcBlock
were single-sector.
Widen BlockDevice to move a run of `count` contiguous sectors per call
(readBlock/writeBlock kept as count=1 wrappers, so metadata call sites —
FAT sectors, directory entries — are untouched). readFile and writeFile
now coalesce the aligned full-sector middle of a transfer into one command
(capped at the cluster boundary and the 4 KiB bounce = 8 sectors), reading
straight into / writing straight from the caller's buffer with no staging
copy. Full-sector writes skip the read-modify-write entirely, since they
overwrite the whole sector. Partial head/tail sectors keep the per-sector
RMW path.
IpcBlock passes count through to the block driver and sizes its bounce from
engine.max_transfer_sectors (already 4 KiB — no new allocation). A new
spc=8 engine test proves runs coalesce (a 21-sector overwrite drops from
>=21 writes to <=5) and that reads/writes round-trip byte-identical,
including an unaligned offset spanning a cluster boundary.
Full QEMU suite 92/92.
USB hub support, root-caused and validated on the user's real machine:
a Keychron keyboard and ROG mouse now enumerate through a Genesys
compound USB3 hub (its USB2 companion + transaction translators) and
type on danos.
Fixes found by reading logs off the stick, each invisible to QEMU:
- SuperSpeed hubs have change bits a USB2 hub lacks; not clearing them
spun the driver. Clear them (gated on speed) + a per-tick service cap.
- The boot scan runs ~3 ms after the controller reset, too early for a
USB2 connection to debounce; the SuperSpeed devices train instantly
and hid the problem. Poll the root ports on the tick (edge-triggered).
- Full-speed devices failed Address Device with a USB Transaction Error:
USB 2.0 requires a 10 ms reset-recovery interval before SET_ADDRESS.
Wait it out, and retry Address Device on a transaction error.
Plus richer logs: readable PORTSC decode, USB link-state and speed
names, device string descriptors (maker/product), and a readable name
for every class/subclass/protocol enum in usb-ids. Full suite 92/92.
Open follow-up: 'port 11 enumeration failed' — a third device that did
not come up; to be investigated separately.
Daniel Samson
Daniel Samson