The 11 QEMU-suite fixtures lived mixed into system/services/ with their
binaries bundled at /system/tests/<name>. Now the repo path is the boot
path, like every real service: test/system/services/<name>. fat-test
moves out of the fat server's directory into its own; display-demo
stays a boot service.
- kernel VFS: setInitialRamdisk derives one read-only initrd mount per
top-level tree named by the ramdisk entry paths (/system, /test),
registers ancestors generically with self-parented roots, and refuses
backend shadowing of any initrd tree
- EFI loader: the fallback walk also enumerates \test (optional — a
volume without fixtures still boots); manifest and capsule unchanged
- path literals: vfs-test self-open + create probe, process-test
process_enumerate matches, the args-echo argv[0] expectation; the
kvfs case now covers the /test root end to end
- docs: DFHS /test rows, tree diagrams, loader prose, and the location
convention gain the third home; fixed the input-source link
100/100 QEMU cases pass.
An adversarial review of the reorg surfaced two doc inaccuracies:
- timers.md: a `time` code example still did `@import("runtime").time`, which
no longer compiles — the runtime aggregator was deleted. Now `@import("time")`.
(Reorg regression: the sweep rewrote `runtime.x` member access but not the
`@import("runtime")` form inside a fenced code block.)
- sysv.md: `callconv(system.kernel_abi)` named a non-existent `system` module;
`kernel_abi` lives in the boot-handoff contract (system/boot-handoff.zig),
imported as `boot_handoff`. Now `boot_handoff.kernel_abi`. (Pre-existing bug —
"system" was the file-path prefix mistaken for a module namespace — found and
fixed opportunistically.)
Rewrote the repository-layout and driver-model docs to describe the new tree —
library/kernel (the kernel32-style system library: syscall surface split by
concern), library/device (mmio/model/pci/usb/acpi/driver/block), library/client
(display, input service clients), library/protocol (wire contracts) — and swept
the reference docs off the retired runtime shim:
runtime.system.* -> logging.* / time.* / process.* / memory.*
runtime.dma.* / runtime.shared_memory.* / runtime.allocator -> memory.*
runtime.ipc/process/time/service/Thread/block/display/input -> the module name
runtime.device / runtime.device_manager -> driver
library/runtime/<file>.zig -> its new home (kernel/ client/ device/)
docs/README.md (repository layout + source map), docs/driver-model.md (the module
graph + import lists), and the concern/reference docs (ipc, threading, timers,
logging, power, process-lifecycle, device-manager, display, vdso, sysv, drivers,
input, vfs-protocol, coding-standards, ...) now reflect the split. "runtime" that
remains is the userspace-library *concept*, which is still accurate.
The historical plan docs (display-plan, display-v2-plan, threading-plan) and the
zig-self-hosting design note are left as point-in-time snapshots.
Follow-up cleanup of the just-moved kernel device code:
- power.zig -> acpi.zig. Its reboot() is built entirely on the FADT reset
register (acpi.power_information) plus the legacy 0xCF9/8042 fallbacks — it is
ACPI reboot, so it becomes acpi.reboot (the "P" in ACPI). platform.reboot
still delegates; soft-off/S5 stays the ring-3 acpi service's job as before.
- device-tree.zig -> fdt.zig. It is a discovery *backend* (the ARM/FDT parser,
a sibling of acpi.zig), not part of the model — renaming it to its actual
subject kills the confusing device-tree / device-model DeviceTree name clash
and makes acpi.zig + fdt.zig read as the two parallel backends.
- Flatten: device-model.zig, acpi.zig, fdt.zig, platform.zig move out of the
system/kernel/devices/ subdir up into system/kernel/, joining devices-broker.zig
(already flat). The kernel dir is a flat pile by convention (only architecture/
is a subdir), so the subdir — and its poor "devices" name — is gone.
Pure restructure; "platform" module name unchanged, all cross-file deps are
relative siblings that moved together. zig build + test green; smoke, discovery,
acpi-parse, acpi-ps2, acpi-report, power-button, orderly-shutdown, reboot pass.
With the shared device data (device-abi, pci-class, usb-abi, usb-ids, acpi-ids,
aml) now in library/device/, what remained in system/devices/ was purely
kernel-internal: the firmware-discovery machinery and the rich pointer-based
device model (platform, device-model, acpi, device-tree, power), reached only
through the "platform" module by three kernel files. It belongs with the kernel.
Move it to system/kernel/devices/. A pure relocation: the "platform" module
name is unchanged and every cross-dir dependency is a module import, so only the
one b.path and some comments move. The top-level split is now clean —
system/kernel/ is the kernel, library/device/ the shared device libraries,
system/{drivers,services} the userspace. The runtime /system/devices concept
(the virtual device tree) is unaffected; system/kernel/devices/ is its
implementation.
Also fixes two comment refs that still pointed device-abi at its pre-Wave-1a
home (system/devices/) — it lives at library/device/model/ now.
zig build + test green; smoke, discovery, acpi-parse, acpi-ps2, acpi-report pass.
mmio is device-driver code, so it joins the other domains under
library/device/mmio/ (module name "mmio" unchanged — a pure relocation, only
the build paths move). And its abbreviated function names are spelled out per
docs/coding-standards.md:
read -> readRegister mb -> memoryBarrier
write -> writeRegister rmb -> readMemoryBarrier
wmb -> writeMemoryBarrier
All call sites updated (virtio-gpu, usb-xhci-library, pci.Function); the two
display-driver placeholders import mmio but use nothing, so they're untouched.
Docs (driver-model graph, README layout, drivers.md, the FHS note) follow the
new path and names.
zig build + test green; virtio-gpu, display-native, display-reattach, usb-hid,
usb-hub, usb-storage, pci-scan pass.
Update the docs to match the reorganized tree:
- driver-model.md: the "Families" graph now shows library/device/<domain>/
(data + logic split) and library/protocol/ instead of the old bus/ + proto/
sketch; add the microkernel note (the kernel's one library/ import is the
device-abi data module) and the rule that a driver's hardware command set is
not a service-seam protocol.
- README.md "Repository layout": device-abi and the protocols move out of
system/ into library/device/ and library/protocol/; rewrite the "public
interface as a module" paragraph around the protocol tree and the device
domains; fix the source-map paths.
- Sweep every remaining stale path reference across docs/ to the moved files.
- Fix vfs-protocol.zig's own header (the standalone VFS server retired; the fat
server is the backend today).
zig build + test green.
The adversarial review of the branch confirmed the big one: the shm/DMA
page-table walks and their pmm/heap calls ran outside the big kernel lock —
pre-existing, but fatal once the per-space cursors invited sibling threads to
race them (two concurrent creates could orphan a page table: one thread's
region silently unmapped, the frame leaked — a plausible root for the
long-standing intermittent AP ring-3 fault at the shm base). All three paths
now follow the mmap discipline: allocation, object build, record, and handle
under one lock hold with full rollback; the map itself per-page under brief
holds; dma_free's translate/unmap/free per-page likewise.
Contract edges from the same review: thread_spawn into a dying group returns
-ESRCH (was generic -1); process_kill during the condemned window answers
from the latch's stashed supervisor (0 or -EPERM, was -ESRCH once the leader
slot was reaped); exit derives the group reason from its own argument rather
than the racy exit_code global; a worker's thread_exit no longer overwrites a
concurrent group-kill stamp; checkGroupDead now asserts exactly-one
notification via the drained ring. Full suite: 100/100.
Eight new QEMU cases (thread-fault-group, kill-threaded-group,
kill-via-worker-tid, racing-triggers, exit-group, leader-thread-exit,
thread-exit-solo, shm-mapping-ref) driving seven new thread-test modes; a
shared checkGroupDead asserts the contract everywhere: one notification,
badged with the leader, reason on the leader's record, no member listed,
claims released first.
The shm-mapping-ref case flushed out the per-task DMA/shared-memory arena
cursor bug directly (a sibling's regions mapped over the worker's), so both
cursors moved to the AddressSpaceRef like the mmap/MMIO cursors before them
(threading M7 pattern). Runtime gains Thread.tryExitCurrent for the leader
-EPERM refusal path. Docs updated: threading.md's shared-fate gap is closed,
process-management.md and process-lifecycle.md describe the leader re-key,
plan status = implemented. Full suite: 100/100.
system.img was only mentioned in passing (efi.md, system-requirements.md);
its format, builder, fallback chain, and kernel-side life were spread across
pack-system-image.py, build.zig, efi.zig, initial-ramdisk.zig, and vfs.zig.
New dedicated page covers all of it, cross-linked from every prior mention
and added to the docs index.
Every doc verified claim-by-claim against the code by parallel audit agents,
then fixed and adversarially re-verified. Two waves of staleness corrected:
the originally audited findings (higher-half boot handoff, kernel VFS
takeover, fault isolation + claim release + driver restart, AML/S5 moving to
ring 3, threading's shipped design, USB+FAT landing) and a second pass of
adjacent claims the verifiers caught (smp.md 'not built yet' intro,
system-requirements' PS/2-only and no-storage claims, halting.md's red-panic
and no-IDT text, testing.md's serial mirroring, router-era vfs-protocol
wording, capsule-first boot loading).
threading.md now documents the shared-fate gap explicitly: the design says a
process dies whole, the kernel today kills only the offending thread.
Also fixes three stale code comments (isr.s exceptionHandler, acpi.zig
sleepValue, build.zig boot-volume) — comments only, no behavior change.
Verified every deferred item in the nine docs with a what's-next section and
marked what has since landed (reaper, per-process CR3, higher-half kernel,
kernel heap, contiguous frame alloc, RSDP capture, cap-passing, driver restart
with backoff, PS/2 keyboard) while keeping the genuinely open items. Also
corrects interrupts.md's claim that the keyboard skipped the IO-APIC, and
updates scheduler.zig/heap.zig comments that predated the reaper and the big
kernel lock.
Rename the file to match current naming conventions and update all
cross-references throughout the documentation. Update the content to
reflect the current state of the codebase:
- Module is now called "architecture", not "arch"
- Directory is system/kernel/architecture/, not system/kernel/arch/
- Generic kernel file is kernel.zig, not main.zig
- Updated file listing for x86_64 architecture module (added smp.zig,
per-cpu.zig, ioapic.zig, trampoline.s)
- Updated function list in cpu.zig to include enterUser/userExit and
address-space management
- Clarified that _start lives in architecture-specific isr.s
- Expanded architecture-specific vs generic comparison table with
actual current modules
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Completes hub support (docs/usb-hub.md). On the bus tick, a downstream
hub-port change now dispatches: a connect enumerates the new device
(B4b), a disconnect tears the old one down — recursively, since a hub
that leaves takes its whole subtree with it (children first), reporting
each interface ChildRemoved and Disable-Slotting the device. Route
strings compose across tiers, so a device two hubs deep enumerates with
a two-tier route.
QEMU's hub — unlike root-port hot-plug — DOES raise downstream
status-change events, so both paths are harness-tested: usb-hub (a
keyboard behind a hub binds usb-hid-keyboard), usb-hub-nested (a
keyboard two hubs deep), usb-hub-unplug (device_del behind the hub tears
it down). Full suite 91/91.
Real-hardware validation of the user's SuperSpeed Genesys hub with
full-speed devices on its USB2 companion is flagged for the user (QEMU's
USB2 hub does not model the compound USB3 hub).
First slice of hub support (docs/usb-hub.md), handled IN the xhci-bus
because a device behind a hub is reached by the CONTROLLER via a route
string in its slot context — topology only exists inside the driver
that owns the controller. When the scan enumerates a class-9 device it
now calls setupHub: read the hub descriptor for the downstream port
count and TT arrangement, tell the controller the slot is a hub
(Configure Endpoint with the slot add-flag sets the Hub bit, Number of
Ports, and MTT/TT-Think-Time per xHCI 4.6.6), SET_HUB_DEPTH for a
SuperSpeed hub so it can compose route strings, and SET_FEATURE
PORT_POWER every downstream port.
The Device gains topology fields (route string, root port, parent hub
slot/port for the TT) and buildAddressInputContext now fills them, so
the Address Device path is ready for downstream devices — those are
enumerated in B4b. A root-port device gets route 0 and its own port as
the chain root, unchanged behavior.
Verified in QEMU with a hub on a second controller and a device behind
it (a usb-hub QEMU case, since the boot controller's auto-assigned
devices collide on the low ports): 'hub slot 1: 8 downstream ports
powered (USB2 single-TT)'. Full suite 89/89. The user's SuperSpeed
Genesys hub is the real-hardware target, flagged separately (QEMU's
USB2 hub doesn't model the compound USB3 hub).
Per docs/coding-standards.md (no Unix-abbreviation exception): syscalls
shared_memory_create/map/physical, kernel SharedMemoryObject + handlers,
runtime.shared_memory (library/runtime/shared-memory.zig), the
shared-memory-server/-client test services, the shared-memory QEMU case,
and docs incl. vdso.md's danos_shared_memory_*. 87/87 QEMU tests pass.
logging.md rewritten around the pipeline (std.log -> leveled debug_write
-> tagged ring -> logger service -> /var/log/<boot-stamp>/<binary-path>.log),
why the buffer is a kernel ring rather than a logging server (the
storage stack must log; early boot needs the buffer anyway), the
announce-once and counted-loss disciplines, and the accepted gaps.
vfs-protocol.md: the router is the kernel (fs_resolve names, backends
serve data); endpoint comes from resolve, not a registry lookup; mount/
unmount retired from the wire; the rewrite-prefix mount semantics.
vdso.md: klog_status + the fs naming calls join the future table; the
no-file-I/O line sharpened (kernel resolves names, never blocks on a
filesystem). syscall.md: the placeholder status blurb replaced with the
real call families.
Client 'present' requests and cursor pokes no longer repaint on the spot:
they accumulate damage and arm a one-shot 16 ms timer, and the tick
composites everything pending as one frame. A fast mouse previously
turned every input event into a full present (100+/s); now any number of
draws and moves inside one interval coalesce into a single repaint.
No backend has a real vblank to pace by (docs/display-v2.md, 'Fenced is
not vsync'), so this is the software stand-in — the same strategy Linux
uses atop virtio-gpu. Bring-up paths that need pixels synchronously
(initialise, self-checks) still present directly.
onNotification now handles the message and timer badge bits
independently: one coalesced badge can carry both, and the old
either/or dispatch would have dropped a tick.
All six display QEMU cases pass.
The virtio-gpu present fence completes when the device has consumed the
frame: real completion feedback, and tear-freedom by snapshot semantics.
It is not a vblank — base virtio-gpu 2D has no display-refresh event at
all (Linux fakes one with a timer), so nothing paces presents to the
monitor. The code and docs claimed vsync anyway; now they don't.
- backend.hasVsync -> hasFencedPresent, with an honest doc comment
- marker 'display: vsync present ok' -> 'display: fenced present ok'
(display-modeset test expectation updated, passes)
- display-v2.md gains a 'Fenced is not vsync' note; the vsync claims in
both v2 docs are corrected
- true vsync arrives with a native driver's vblank IRQ, or approximated
by a compositor frame clock
Two real bugs visible in a normal `zig build run-x86-64` boot (but not in the
display-demo test, which spawns no input service):
- Two cursors. The demo drew its own cursor layer while the display service
now draws one too (its mouse-listener thread). The demo's went through the
client IPC protocol and lagged; the service's is in-process and tracks
tightly — "one responds better than the other."
- Animation frozen until the mouse moves. The demo called a *blocking*
`mouse.next()` (ipc_reply_wait) inside its animation loop, so the sliding
box advanced only one frame per mouse event. In the display-demo test
there is no input service, so subscribeMouse() returned null and the loop
ran free on its 30ms timer — which is exactly why the test passed while
the real boot was broken.
The compositor now owns the cursor (docs/display.md), so the demo should draw
none and read no input: it becomes a pure client-animation proof whose loop is
independent of the mouse. Removes its cursor layer, mouse subscription, and the
blocking read.
Also harden displayDemoTest to spawn the `input` service alongside the demo
(matching real boot): a client that blocks its animation loop on a mouse read
would now stall before `display-demo: ok` and fail the test, instead of
passing because no input service happened to be present.
Verified: display / display-service / display-demo / display-cursor all green.
The display's first use of threads (docs/threading.md, docs/display.md). The
compositor stays the single owner of the framebuffer — only the main
service.run loop touches the backend and layer stack — and a dedicated
mouse-listener thread runs beside it:
- Listener: blocks on input.subscribeMouse(), accumulates relative dx/dy
into an absolute cursor position clamped to the screen, and hands it to
the compositor. It never touches the compositor, so no lock guards the
framebuffer; a parked next() lets the core halt.
- CursorChannel: a single-slot latest-value cell under a Thread.Mutex (the
renderer wants where the cursor is now, not a replay of deltas), with a
coalesced self-ipc.send poke that wakes the main loop — parked in
replyWait — as a message-notification. At most one poke is queued while
the last is undrained, so a fast mouse can't flood the endpoint.
- Render: the cursor is a top-z compositor layer; on the poke the main loop
moves it via configure + present (which damages old + new footprints).
The display binary opts into threads (addThreadedUserBinary), and
input-source gains a "mouse" mode that publishes pure motion to drive it.
Two kernel-level findings this surfaced, both fixed:
1. IPC handles do not cross threads. The handle table lives on the Task, so
the listener can't reuse the main loop's endpoint handle — it
ipc.lookup(.display)s its own handle to the same endpoint to poke through.
2. Concurrent IPC from two threads raced unlocked kernel state. The display
is the first process issuing IPC syscalls from two threads at once, which
exposed a data race (flaky #GP in installEntry): create_ipc_endpoint /
ipc_register / ipc_lookup allocate from the kernel heap and mutate the
global registry, endpoint refcounts, and handle tables without the big
kernel lock. They were safe only while a process couldn't race itself.
They now sync.enter() like call/reply_wait/send already did (the kernel
heap has no lock of its own yet — heap.zig: "a lock comes with
threads/SMP" — so the big lock keeps its callers serialized).
Test: -Dtest-case=display-cursor (smp:4) spawns the input service, the
threaded display, and input-source in mouse mode; asserts the display's
"cursor tracking mouse ok" marker once the cursor has tracked a run of motion
end to end. Verified green 6/6 under stress (the race hit ~1-in-4 before the
lock fix) and in the full 29-case QEMU guardrail suite; zig build test clean.
The "fs.base" spelling read like a field/submodule access, but no such
identifier exists — it meant the x86_64 FS segment base (the IA32_FS_BASE
MSR). Two problems fixed:
- Arch-neutrality: in generic docs, the runtime, and the plan, the mechanism
is now named by its arch-neutral concept — the "thread pointer" — matching
the already-renamed `thread_pointer` Task field, `set_thread_pointer`
syscall, and `architecture.setThreadPointer` fn. x86-specific spots keep
the precise names: `IA32_FS_BASE` (the MSR), `%fs:8`/`%fs:0`, variant-II.
- Stale identifiers: the M10 section in threading-plan.md still referenced
`fs_base` on Task and `architecture.setFsBase` — both renamed away in the
arch-neutral pass. Corrected to `thread_pointer` / `setThreadPointer`.
The x86-only `thread-tls` test (which really does write `%fs:8`) now says
"FS base" (no dot) consistently, matching the established form already in
tests.zig. The matched serial markers ("thread-tls: ok" / "thread-tls:
FAIL") are unchanged; only a non-load-bearing FAIL parenthetical was
reworded.
Verified: zig build clean, thread-tls passes.
Expand the abbreviations flagged in docs/coding-standards.md (names spelled
out in full unless an acronym) across the kernel, runtime, ABI, tests, and
docs:
aspace -> address_space (AspaceRef -> AddressSpaceRef, retainAspace ->
retainAddressSpace, loaded_aspace -> loaded_address_space, the
liveAspaceCount/aspaceDestroyCount test hooks, etc.)
vaddr -> virtual_address
paddr -> physical_address
The kernel test case and its serial markers are renamed to match:
aspace-refcount -> address-space-refcount (kernel dispatch string and
test/qemu_test.py case name kept in sync). Prose in docs uses the natural
"address space"/"virtual address"; backticked field/identifier references
use the code spelling.
Also expand the bare "AS" abbreviation in three ABI comments and reframe the
set_thread_pointer ABI/handler docs to lead with the arch-neutral concept
(user-space TLS thread pointer; x86_64 IA32_FS_BASE, aarch64 TPIDR_EL0)
rather than x86 FS-first, matching scheduler.zig's existing framing.
Foreign ABI names preserved: the ELF p_vaddr field and mmap/mmio remain.
Verified: zig build, zig build test, and the full 25-case QEMU guardrail
suite all green.
runtime.Thread.RwLock (reader-preferring, lock/tryLock/unlock +
lockShared/tryLockShared/unlockShared) and WaitGroup (start/finish/wait), both on
the existing Mutex/Condition.
A compile-time Futex seam gated on builtin.os.tag: the futex syscalls on danos, a
spin+yield mock off-target (Zig 0.16 has no std.Thread.Futex; wake is a no-op
since the state machines re-check). thread.zig is wired into zig build test, so
Mutex/RwLock/WaitGroup run as host unit tests with real std.Thread threads (test
blocks compile only under test, so std.Thread there is fine on freestanding).
thread-rwlock QEMU case: 2 writers set both halves of a value under the exclusive
lock while 3 readers check they match under the shared lock; zero half-write
observations across ~150k reads.
Marks Phase 2 (M7-M11) built. threading.md/threading-plan.md status updated.
Gate: host zig build test covers the sync primitives; thread-rwlock PASS (3x);
full Done gate 26/26 (whole thread-* suite + guardrail); build clean.
Each thread gets its own x86_64 thread pointer (FS base) for user-space TLS.
Task.fs_base is restored on every context switch only when it changes (same
conditional-load discipline as CR3; architecture.setFsBase -> wrmsr
IA32_FS_BASE). New set_thread_pointer=44 syscall sets the caller's fs_base and
loads it now. The kernel never touches FS, so no swapgs complication.
The runtime lays a small per-thread TLS block at the top of each thread's stack
(self-pointer at %fs:0 + scratch) and the thread trampoline calls
set_thread_pointer before any user code — so every spawned thread has a private,
switch-stable thread pointer, reclaimed with the stack.
thread-test tls mode: two threads write unique markers to their own %fs:8 and,
after both wrote, read back — a shared fs.base would clobber one (cross-talk).
Deferred: the Zig threadlocal *compiler* layer (ELF variant-II PT_TLS + linker
sections + template copy) — high-uncertainty, no consumer today; this lands the
load-bearing per-thread fs.base it builds on. See docs/threading-plan.md M10.
Gate thread-tls PASS (3x); full guardrail 25/25; build + host tests clean.
join no longer needs a per-thread IPC endpoint. New thread_join(tid) syscall
blocks the caller until the task with id tid exits; the exit paths call
wakeJoinersLocked. join only reclaims the joined thread's USER stack, which the
thread vacates the instant it enters the kernel to exit, so waking at exit time
(not reap time) is safe — no reaper/aspace juggling or user-memory write, and
equally std-shaped (like pthread_join). thread_spawn drops the exit-endpoint arg
(runtime passes no_cap). thread-test's join mode runs 40 spawn+join cycles that
would exhaust the 16-slot handle table under the old endpoint scheme.
Also harden the M8 reaper: its single per-core reap slot could be overwritten by
a second death on that core before draining (a fresh-task/SMP timing window), an
intermittent one-stack leak that made task-reap ~20% flaky. Replace it with a
per-core reap LIST plus a .reaping task state so a pending slot can't be reused
before its stack is freed. task-reap now 11/11 isolated.
Deferred: detached-thread user-stack reclaim (still at process exit, as in M3).
Gate thread-join PASS (3x); full guardrail 26/26; build + host tests clean.
A dead task's kernel stack was leaked (no reaper), so every process/thread death
bled kernel memory. Now exit()/exitUserLocked record the dying task in a per-core
reap_after_switch slot and switch away; the task that resumes on that core frees
the stack in switchTo's tail (on its own stack, lock still held so the slot can't
be reused). reapKillPendingLocked drains the slot on the timer tick as a safety
net for the fresh-task case (a fresh task enters via the trampoline, bypassing
switchTo's tail). A task killed while not running is freed directly in
destroyTaskLocked. live_stack_bytes is the observable.
Fixed a migration bug this exposed: the post-switchContext reap read the pc
parameter, but a migrated task carries a stale pc in its saved switchTo frame ->
it freed the wrong core's pending stack (a #GP under SMP). Re-fetch thisCpu()
after the switch.
Gate task-reap PASS (5x isolated, 2x in the 24-case batch); full guardrail 24/24
incl. fault-recovery/supervision/process-kill/smp/affinity; build + host green.
Move the mmap/mmio grant-arena cursors off Task into the per-address-space object
(scheduler aspace_refs, exposed via aspaceMmapNextPtr/aspaceDeviceMapNextPtr), so
sibling threads in one address space hand out disjoint grants. systemMmap reserves
a range under a brief lock then maps per page under a short-held lock (not the
whole grant): the big lock runs with interrupts disabled, so pinning it across a
multi-MiB memset+map froze other cores. Guard the runtime heap's rawAlloc/rawFree
with a Thread.Mutex, gated on !single_threaded so ordinary binaries compile it out.
thread-test gains an alloc mode: 4 threads x 500 alloc/fill/verify/free cycles;
any overlap between concurrent allocations is caught by the pattern check.
Also fix the affinity guardrail: its 3-billion-iteration busy-loop had
codegen-dependent wall-time (adding a function to tests.zig swung it ~4s -> ~63s
and timed it out). Reworked to wait on the wall clock instead.
Gate thread-alloc PASS (3x); full guardrail 23/23 green; build + host tests clean.
Adjust the unattended loop contract for Phase 2: the Done condition targets M1
through M11 (M1-M6 being checked is no longer Done), the loop branches off the
current main into a new branch (Phase 1's threading is merged), and each green
milestone pushes the working branch to origin (main stays a human merge).
Add M7-M11 to docs/threading-plan.md, designed so threading reinforces danos's
goals: everything a thread owns lives in the address space (reclaimed on process
death via the M1 refcount), the kernel owns mechanism while the runtime owns
policy, and the process stays the isolation/restart boundary.
M7 thread-safe allocation (per-aspace mmap arena + locked runtime heap);
M8 task reaper (reclaim kernel stacks + detached user stacks — the resilience gap);
M9 futex-completion join (retire the per-thread endpoint, built on M8);
M10 per-thread TLS (threadlocal + fs.base, for self-hosting);
M11 RwLock/WaitGroup + host-testable sync.
New thread_self=42 syscall backs runtime.Thread.getCurrentId (the calling
thread's kernel task id). thread-test gains an id mode: two workers read
getCurrentId and the main thread confirms all three ids are non-zero and
distinct. Per-thread threadlocal TLS is deferred by design (no consumer; it would
need context-switched fs.base for an unused feature), as are RwLock/WaitGroup.
Marks the threading feature built (M1-M6): threading.md + docs/README.md status
updated, all thread-* cases wired into qemu_test.py.
Gate thread-id PASS; full suite green: 21/21 (thread-spawn/join/futex/mutex/id,
aspace-refcount, + 15 guardrail cases), zig build clean, zig build test green.
runtime.Thread.Mutex is the classic three-state futex mutex (unlocked/locked/
contended): the fast path is a single CAS and only a contended lock enters the
kernel. Condition is a futex sequence counter (wait/timedWait/signal/broadcast,
spurious wakeups allowed, use in a predicate loop); a signal racing the unlock
bumps the seq so it is never missed. Semaphore is permits guarded by
Mutex+Condition. All mirror std.Thread's shapes, ported onto runtime.Thread.Futex.
thread-test gains a mutex mode: 2 producers + 2 consumers move 2000 unique items
through an 8-slot ring (small enough that both sides block); the consumed
checksum and tally match exactly, proving the lock and condvars correct under
real cross-core contention.
Deferred with rationale (see docs/threading-plan.md): migrating join to a futex
completion word needs kernel clear-on-exit (else use-after-free munmapping a live
stack); host unit tests need a mockable Futex seam.
Gate thread-mutex PASS (3x); 17 guardrail/thread cases green; build + host tests
clean.
New private syscalls futex_wait(addr, expected, timeout_ns)=40 and
futex_wake(addr, count)=41. A waiter is a .blocked task tagged with
Task.futex_addr (no queue linkage); futex_wait reads the user word under the big
lock and parks only if it still equals expected, so a concurrent wake can't slip
between the check and the block. futex_wake scans the task table and readies up
to count waiters in the same address space. A timed wait also sets wake_at so the
existing wakeExpired times it out; futex_addr staying non-zero (only futex_wake
clears it) distinguishes timeout from a real wake. Waiters park in-kernel, so an
idle core still halts (no busy-wait).
runtime.Thread.Futex mirrors std.Thread.Futex (wait/timedWait/wake). thread-test
gains a futex mode: a waiter parks, the main thread wakes it (serial order
waiting/waking/woke, asserted by the case regex), and timedWait reports a
timeout.
Gate thread-futex PASS (3x); 18 guardrail cases green incl. sleep/event/ipc
blocking paths; build + host tests clean.
thread_spawn takes a 4th arg, an exit-endpoint handle: spawnThreadSupervised
resolves and refcounts it under the spawn lock (like spawnProcessSupervised), so
a thread's death posts a child-exit notification carrying its tid. runtime
Thread.join blocks in replyWait on that (private) endpoint for its tid, then
munmaps the stack; detach relinquishes the join (stack reclaimed at process
exit, for now). New current_core=39 syscall + Thread.currentCore() lets a worker
observe which core it ran on.
The closure now lives at the top of the thread's own (private) stack instead of
the heap, so spawn/join never touch the not-yet-thread-safe runtime heap.
thread-test gains a join mode: 4 workers x 100k atomic increments, joined, with
counter == N*K and >1 core stamped (real parallelism), plus a detached worker.
Gate thread-join PASS (4x, non-flaky); 17 guardrail/M1/M2 cases green; build +
host tests clean.
A thread is a task sharing the caller's address space. New private syscalls
thread_spawn(entry, stack_top, arg)=37 and thread_exit=38: thread_spawn goes
through scheduler.spawnThread (retains the shared aspace), thread_exit ends the
task like a process exit(0) (terminateCurrent -> releaseAspace, so the space
survives while siblings hold it). The closure pointer reaches the new thread in
rdi via a new jump_to_user_arg asm path and a per-task user_arg (0 for a normal
process, whose _start ignores it) - so the runtime trampoline is a plain C-ABI
Zig function, no naked asm.
runtime.Thread (library/runtime/thread.zig) mirrors std.Thread.spawn: mmap a
stack, heap-allocate the args closure, hand the kernel the trampoline + closure.
addThreadedUserBinary opts a binary into single_threaded=false; thread-test is
the first, and proves a worker runs in the shared address space via a shared
global the main thread polls.
Gate thread-spawn PASS; 16 guardrail cases green (incl. args/init/process on the
new jump_to_user_arg path) + aspace-refcount; build + host tests clean.
Route address-space lifetime through a refcount keyed by the page-table root
(scheduler.zig aspace_refs): retainAspace on the spawnUserLocked success path,
releaseAspace from both teardown paths (exitUserLocked, destroyTaskLocked),
destroying the space only when the last task on it exits. Behaviour is identical
today (every space has exactly one task); this is the foundation shared-address-
space threads (docs/threading.md) build on.
Test-observable liveAspaceCount/aspaceDestroyCount + a new aspace-refcount kernel
self-test and QEMU case: spawn and reap 5 ring-3 probes, assert live spaces return
to baseline and destructions advance by exactly 5 (destroyed once each, no leak,
no double-free). Gate passes; 13 guardrail cases green; build + host tests clean.
Add docs/threading.md (native runtime.Thread mirroring std.Thread over a
private thread ABI) and docs/threading-plan.md (6 milestones, each with a
serial-checkable gate + guardrail, plus an unattended /loop execution
contract). Index both in docs/README.md.
Two companion research snapshots on what a minimal display-only native driver
(EDID + mode-set + framebuffer scanout, no acceleration) would take as a danos
.scanout backend, and how the two vendors compare.
- nvidia-gpus.md — RTX 3060 (Ampere GA106). Display is not GSP-gated: nouveau's
ga102.c has a direct register path, and danos's GOP boot lets a driver attach to
a live, already-devinit'd display. Tier-4 effort; the register-level display code
lives in GPL nouveau while the permissive OGKM reference is the hard GSP path.
GA10x is the last NVIDIA family that keeps a register-level display path.
- intel-igpu.md — the companion, and a materially easier, lower-tier target. Intel
publishes register-level Display Engine PRMs, so a clean-room permissive driver
is viable; the DMC microcontroller is optional (power states only), scanout comes
from system RAM via the GGTT (no VRAM manager), and coreboot's libgfxinit is a
compact native reference. The catch is hardware: an iGPU may not drive the
monitor on a discrete-GPU desktop.
Sourced, cited snapshots — not implementations; each ends with a "first light"
milestone ladder framed as a danos .scanout service. Not yet linked from
docs/README.md.
The compositor now survives the virtio-gpu driver dying and re-attaches when device-manager
restarts it — the last piece of display v2.
Three parts:
- The driver hellos the device manager (role: bus). It never did, so the manager — which
spawns it supervised and expects a hello — was stopping it at the 3s hello deadline every
run (the gate markers just printed first). Now it is properly supervised: not stopped for
silence, and restarted on death.
- A kernel IPC fix so a call to a dead service errors instead of hanging. An endpoint records
its owner; when that task dies, its registered endpoints are marked dead (and any parked
senders woken with -EPEER), so ipc_call returns -EPEER rather than blocking on a reply that
will never come. Without this the compositor's first present after the driver died blocked
forever. General robustness — any client of any service benefits.
- The compositor re-attaches. Its .scanout calls now fail cleanly (caught), freezing the last
frame; when the restarted driver re-announces, attach_scanout detects the backend is already
virtio and logs "scanout re-attached", mapping the fresh shared surface and re-looking-up
.scanout. (The previous shm mapping leaks — no shm_unmap syscall yet — but its frames are the
dead driver's, reclaimed on exit.)
- device-manager gains a "test-scanout-restart" mode (like test-usb-restart) that kills the
virtio-gpu driver once after it hellos; the displayReattachTest kernel scenario drives it.
Gate: python3 test/qemu_test.py display-reattach — "scanout upgraded to virtio-gpu" then
"scanout re-attached", no CPU exception, passing 3/3. host tests, ipc/ipc-call/ipc-cap,
supervision, shm, display-service, display-demo, virtio-gpu, display-native, and
display-modeset all pass; default zig build clean. v2 (V1-V6) complete.
The native backend can now change resolution and presents tear-free.
Mode-setting without churn. The driver sizes its scanout resource + shared surface to the
largest mode it offers and treats a mode change as re-pointing the scanout rectangle within
that surface — so the resource, its backing, and the shared mapping never change, and the
surface's row stride (the max width) is fixed while the active width/height move. The
compositor is handed that stride in the announce and composes at it; a smaller mode just
paints the top-left rectangle. This sidesteps the surface re-share a true resolution change
would otherwise need (the service harness can't reply with a capability).
- scanout-protocol gains get_modes + set_mode; the driver offers {640x480, 800x600} and
re-points set_scanout on set_mode.
- backend.VirtioGpu carries the surface stride, exposes modes()/setMode(), and reports
canModeSet = hasVsync = true.
- runtime.display gains modes()/setMode() (display-protocol get_modes/set_mode, forwarded to
the backend) — the client-facing API.
- EDID: the driver negotiates VIRTIO_GPU_F_EDID when the device offers it, reads the monitor's
EDID, and logs its preferred mode (parsed from the first detailed timing descriptor).
- vsync: every resource_flush is fenced (VIRTIO_GPU_FLAG_FENCE); the device signals the fence
when the frame is on screen, which the used-ring ack the synchronous present already waits
on gates — so a completed present is a tear-free one.
After the native upgrade the compositor runs a one-shot mode-set self-check: query the modes,
switch to a different one, re-composite, and confirm the backend reports the new geometry —
the gate's markers.
Gate: python3 test/qemu_test.py display-modeset (reuses the display-native boot) — "display:
mode set to 800x600, verified" + "display: vsync present ok", passing 3/3. host tests,
display-service, display-demo, shm, virtio-gpu, and display-native still pass.
The compositor now boots on the GOP framebuffer and upgrades to the virtio-gpu driver
the moment it announces itself — the pluggable-scanout payoff.
The shared surface. The scanout resource is an shm region the driver creates
(shm_physical, a new syscall, hands it the guest-physical for attach_backing) and passes
to the compositor as a capability. The compositor maps it and composites straight into
it: on x86 DMA is cache-coherent, so the cacheable shared pages the CPU paints are exactly
what the device transfers-and-flushes — no copy, no explicit flush.
The handshake. After bring-up the driver looks up .display and sends attach_scanout with
the geometry + the surface capability. The compositor maps the surface, looks up the
driver's .scanout endpoint itself (the driver registered it — no need to pass it), switches
to backend.VirtioGpu, and re-composites the current frame. present() over the native
backend is a present request on .scanout -> transfer-to-host + resource flush. The first
native present is deferred to a one-shot timer: presenting inline from the announce handler
would deadlock, since the driver is still blocked on our reply and not yet serving .scanout.
After it lands, the compositor reads a pixel back from the shared surface to confirm the
frame reached the device's backing.
- shm_physical (syscall 36) + runtime.shm.physical.
- scanout-protocol (the compositor->driver present channel), separate from the
client-facing display protocol; the display protocol gains attach_scanout.
- backend.VirtioGpu joins backend.Gop in the tagged union; select() still boots GOP.
- the virtio-gpu driver's scanout backing is now shm (was DMA); it announces + serves
.scanout present requests (transfer-to-host + flush of the shared surface).
Also fixes a latent framebuffer-geometry corruption the display service hit only when it
enumerated the device tree alongside a busy device-manager: Gop.init now captures the
geometry into a small value the instant device_enumerate returns (rather than re-reading
the 328-byte descriptor across the later claim/mmio_map syscalls) and retries on a zero
geometry. The underlying device-table clobber is a separate kernel bug, tracked apart.
Gate: python3 test/qemu_test.py display-native (QEMU -device virtio-gpu-pci) — "display:
scanout upgraded to virtio-gpu" + "display: native present verified" + "display-demo: ok",
passing 3/3. host tests, display-service, display-demo, shm, and virtio-gpu still pass.
Daniel Samson
daniel