The EFI loader now ENUMERATES /system rather than opening files by name,
so the on-disk name must be exact, not merely case-insensitively
findable. make-fat-image.py stored 8.3-fitting lowercase names as bare
uppercase short entries ('init' -> INIT), garbling the ramdisk paths;
now any name whose case the 8.3 entry cannot reproduce gets a long-name
chain carrying the exact name.
device-manager's driver table stored names in 24 bytes — silently
truncating '/system/drivers/usb-xhci-bus' and failing the spawn; widen
to 64 (= abi.maximum_process_name). The usb-report harness regex learns
that restart lines name drivers by path.
Retire the build-time ramdisk packer and the packed initial-ramdisk.img.
make-fat-image.py now lays every user binary out at its FHS path on the
boot volume (system/services, system/drivers, system/tests), and the EFI
loader walks \system at boot, packing what it finds into an in-RAM v2
initial_ramdisk whose entry names are full FHS paths. init rides the
table like every other binary: its dedicated handoff fields are gone and
the kernel spawns PID 1 via the same lookup as everyone else
(process.spawnBundled).
system_spawn resolves names by exact path first, then unique basename,
and normalizes argv[0] to the stored path — so task names (and, next,
the tagged log ring's attribution) are honest binary paths everywhere.
initrd v2 rejects the old magic so a stale image fails loudly.
Groundwork for per-process logging (/var/log/<boot-stamp>/<binary-path>.log)
and the kernel-VFS /system mount.
Both EDID moments the system has are now captured and carried to the
compositor's frame clock:
- EFI: the loader derives refresh from the preferred detailed timing
(pixel clock / total pixels) while GOP is still alive — the only moment
it is readable — and hands it through the boot handoff into the
display0 node's DisplayInfo (new refresh_hz field, 0 = unknown).
- GPU: the virtio-gpu driver derives the same figure from its own EDID
read and carries it in the attach_scanout announce (request.y).
updateFrameClock() re-derives the interval from the active backend's
info at bring-up and again on every backend change — the boot
framebuffer's clock dies with the GOP floor at upgrade, replaced by the
GPU's rate. Unknown rate defaults to 60 Hz; the result is clamped to
[30, 120] Hz so a mis-parsed EDID can neither starve nor flood the
compositor. Rate only, never phase: without vblank, presents still
free-run (docs/display-v2.md, 'Fenced is not vsync').
Observed in QEMU: OVMF exposes no EDID for the VGA adapter, so the GOP
floor logs 'frame clock 62 Hz (default)' (real firmware does expose it);
the virtio-gpu EDID advertises 75 Hz and the upgrade logs 'frame clock
76 Hz (panel EDID)'. The display kernel test asserts refresh_hz rides
the seeded node; all 7 display QEMU cases pass.
Client 'present' requests and cursor pokes no longer repaint on the spot:
they accumulate damage and arm a one-shot 16 ms timer, and the tick
composites everything pending as one frame. A fast mouse previously
turned every input event into a full present (100+/s); now any number of
draws and moves inside one interval coalesce into a single repaint.
No backend has a real vblank to pace by (docs/display-v2.md, 'Fenced is
not vsync'), so this is the software stand-in — the same strategy Linux
uses atop virtio-gpu. Bring-up paths that need pixels synchronously
(initialise, self-checks) still present directly.
onNotification now handles the message and timer badge bits
independently: one coalesced badge can carry both, and the old
either/or dispatch would have dropped a tick.
All six display QEMU cases pass.
The virtio-gpu present fence completes when the device has consumed the
frame: real completion feedback, and tear-freedom by snapshot semantics.
It is not a vblank — base virtio-gpu 2D has no display-refresh event at
all (Linux fakes one with a timer), so nothing paces presents to the
monitor. The code and docs claimed vsync anyway; now they don't.
- backend.hasVsync -> hasFencedPresent, with an honest doc comment
- marker 'display: vsync present ok' -> 'display: fenced present ok'
(display-modeset test expectation updated, passes)
- display-v2.md gains a 'Fenced is not vsync' note; the vsync claims in
both v2 docs are corrected
- true vsync arrives with a native driver's vblank IRQ, or approximated
by a compositor frame clock
Boots the usual VGA/GOP floor plus a virtio-gpu-pci adapter, so the
device-manager stack spawns the driver and the compositor upgrades to the
native fenced-present backend interactively. QEMU shows one head per
adapter: the live output is the virtio-gpu console in the View menu (the
VGA head freezes at the moment of upgrade). 512M, matching display-native.
Tearing mitigation for the GOP floor, attacking the copy window from three sides:
- Damage is no longer one bounding box. Two trackers, A/B-switchable at
compile time (display.zig damage_mode): DamageList (free-form dirty rects,
overlap-merged) and TileGrid (fixed 64-px tiles, exact O(1) marking, runs
coalesced back into rects). Far-apart changes — the cursor here, an
animating layer there — no longer unite into one huge repaint.
- fillRect/composite/blitTile now work in row spans (@memset/@memcpy), so
the compiler vectorizes them and ReleaseSafe bounds checks drop to per-row.
- The back->front present streams 8-byte volatile stores (presentSpan);
Backend.present takes the rect list, so each present copies only what
changed, faster.
Host tests cover both trackers; the display QEMU cases all pass.
The display becomes danos's first multi-threaded service. A mouse-listener
thread runs beside the compositor loop; the compositor stays the single owner
of the framebuffer, and the cursor it draws is a top-z layer moved from the
listener's accumulated position over a single-slot latest-value CursorChannel
(Thread.Mutex) with a coalesced self-ipc.send poke that wakes the parked
replyWait (docs/display.md, docs/threading.md).
Two kernel-level findings this surfaced, both fixed:
- IPC handles are per-Task, not per-address-space — a thread reaches another
thread's endpoint via ipc.lookup() to install its own handle.
- Concurrent IPC from two threads raced unlocked kernel state (flaky #GP);
create_ipc_endpoint / ipc_register / ipc_lookup now take the big kernel
lock like call/reply_wait/send already did (the kernel heap has no lock of
its own yet — heap.zig: "a lock comes with threads/SMP").
Also decouples display-demo: it no longer draws a cursor (the service owns it)
or blocks its animation loop on a mouse read, so a normal boot shows one
responsive cursor and animation that runs independent of the mouse. The
display-demo test now spawns the input service to keep that independence honest.
New test display-cursor (smp:4) drives synthetic motion end to end. Verified:
zig build, zig build test, and the full 29-case QEMU guardrail suite all green.
Two real bugs visible in a normal `zig build run-x86-64` boot (but not in the
display-demo test, which spawns no input service):
- Two cursors. The demo drew its own cursor layer while the display service
now draws one too (its mouse-listener thread). The demo's went through the
client IPC protocol and lagged; the service's is in-process and tracks
tightly — "one responds better than the other."
- Animation frozen until the mouse moves. The demo called a *blocking*
`mouse.next()` (ipc_reply_wait) inside its animation loop, so the sliding
box advanced only one frame per mouse event. In the display-demo test
there is no input service, so subscribeMouse() returned null and the loop
ran free on its 30ms timer — which is exactly why the test passed while
the real boot was broken.
The compositor now owns the cursor (docs/display.md), so the demo should draw
none and read no input: it becomes a pure client-animation proof whose loop is
independent of the mouse. Removes its cursor layer, mouse subscription, and the
blocking read.
Also harden displayDemoTest to spawn the `input` service alongside the demo
(matching real boot): a client that blocks its animation loop on a mouse read
would now stall before `display-demo: ok` and fail the test, instead of
passing because no input service happened to be present.
Verified: display / display-service / display-demo / display-cursor all green.
The display's first use of threads (docs/threading.md, docs/display.md). The
compositor stays the single owner of the framebuffer — only the main
service.run loop touches the backend and layer stack — and a dedicated
mouse-listener thread runs beside it:
- Listener: blocks on input.subscribeMouse(), accumulates relative dx/dy
into an absolute cursor position clamped to the screen, and hands it to
the compositor. It never touches the compositor, so no lock guards the
framebuffer; a parked next() lets the core halt.
- CursorChannel: a single-slot latest-value cell under a Thread.Mutex (the
renderer wants where the cursor is now, not a replay of deltas), with a
coalesced self-ipc.send poke that wakes the main loop — parked in
replyWait — as a message-notification. At most one poke is queued while
the last is undrained, so a fast mouse can't flood the endpoint.
- Render: the cursor is a top-z compositor layer; on the poke the main loop
moves it via configure + present (which damages old + new footprints).
The display binary opts into threads (addThreadedUserBinary), and
input-source gains a "mouse" mode that publishes pure motion to drive it.
Two kernel-level findings this surfaced, both fixed:
1. IPC handles do not cross threads. The handle table lives on the Task, so
the listener can't reuse the main loop's endpoint handle — it
ipc.lookup(.display)s its own handle to the same endpoint to poke through.
2. Concurrent IPC from two threads raced unlocked kernel state. The display
is the first process issuing IPC syscalls from two threads at once, which
exposed a data race (flaky #GP in installEntry): create_ipc_endpoint /
ipc_register / ipc_lookup allocate from the kernel heap and mutate the
global registry, endpoint refcounts, and handle tables without the big
kernel lock. They were safe only while a process couldn't race itself.
They now sync.enter() like call/reply_wait/send already did (the kernel
heap has no lock of its own yet — heap.zig: "a lock comes with
threads/SMP" — so the big lock keeps its callers serialized).
Test: -Dtest-case=display-cursor (smp:4) spawns the input service, the
threaded display, and input-source in mouse mode; asserts the display's
"cursor tracking mouse ok" marker once the cursor has tracked a run of motion
end to end. Verified green 6/6 under stress (the race hit ~1-in-4 before the
lock fix) and in the full 29-case QEMU guardrail suite; zig build test clean.
Completes the std.Thread-shaped runtime.Thread. Phase 1 (M1-M6: address-space
refcount, thread_spawn/exit, join/detach, futex + Mutex/Condition/Semaphore,
getCurrentId) was already on main; this brings the Phase 2 hardening and the
coding-standards/arch-neutrality passes on top:
M7 thread-safe allocation (per-address-space mmap arena + locked heap)
M8 the task reaper — reclaim dead tasks' kernel stacks
M9 thread_join syscall — retire the per-thread endpoint
M10 per-thread thread pointer — the TLS mechanism (x86_64 IA32_FS_BASE)
M11 RwLock, WaitGroup, and host-testable sync
Plus: the TLS thread pointer named arch-neutrally (not fs.base) so the kernel
stays architecture-agnostic; aspace/vaddr/paddr spelled out per
docs/coding-standards.md across kernel, runtime, ABI, tests, and docs; and the
misleading fs.base dot-notation dropped in favour of "thread pointer"
(arch-neutral) / "FS base" (x86-specific).
Deferred by design (no consumer yet): the Zig threadlocal *compiler* layer
(M10) and detached-thread user-stack reclaim (M9) — both noted in place.
Verified: zig build, zig build test, and the full 25-case QEMU guardrail suite
all green.
The "fs.base" spelling read like a field/submodule access, but no such
identifier exists — it meant the x86_64 FS segment base (the IA32_FS_BASE
MSR). Two problems fixed:
- Arch-neutrality: in generic docs, the runtime, and the plan, the mechanism
is now named by its arch-neutral concept — the "thread pointer" — matching
the already-renamed `thread_pointer` Task field, `set_thread_pointer`
syscall, and `architecture.setThreadPointer` fn. x86-specific spots keep
the precise names: `IA32_FS_BASE` (the MSR), `%fs:8`/`%fs:0`, variant-II.
- Stale identifiers: the M10 section in threading-plan.md still referenced
`fs_base` on Task and `architecture.setFsBase` — both renamed away in the
arch-neutral pass. Corrected to `thread_pointer` / `setThreadPointer`.
The x86-only `thread-tls` test (which really does write `%fs:8`) now says
"FS base" (no dot) consistently, matching the established form already in
tests.zig. The matched serial markers ("thread-tls: ok" / "thread-tls:
FAIL") are unchanged; only a non-load-bearing FAIL parenthetical was
reworded.
Verified: zig build clean, thread-tls passes.
Expand the abbreviations flagged in docs/coding-standards.md (names spelled
out in full unless an acronym) across the kernel, runtime, ABI, tests, and
docs:
aspace -> address_space (AspaceRef -> AddressSpaceRef, retainAspace ->
retainAddressSpace, loaded_aspace -> loaded_address_space, the
liveAspaceCount/aspaceDestroyCount test hooks, etc.)
vaddr -> virtual_address
paddr -> physical_address
The kernel test case and its serial markers are renamed to match:
aspace-refcount -> address-space-refcount (kernel dispatch string and
test/qemu_test.py case name kept in sync). Prose in docs uses the natural
"address space"/"virtual address"; backticked field/identifier references
use the code spelling.
Also expand the bare "AS" abbreviation in three ABI comments and reframe the
set_thread_pointer ABI/handler docs to lead with the arch-neutral concept
(user-space TLS thread pointer; x86_64 IA32_FS_BASE, aarch64 TPIDR_EL0)
rather than x86 FS-first, matching scheduler.zig's existing framing.
Foreign ABI names preserved: the ELF p_vaddr field and mmap/mmio remain.
Verified: zig build, zig build test, and the full 25-case QEMU guardrail
suite all green.
The M10 TLS work leaked x86 naming into the generic kernel: Task.fs_base,
PerCpu.loaded_fs_base, and architecture.setFsBase. The *mechanism* was already
abstracted (the generic scheduler calls through the architecture layer; the
wrmsr IA32_FS_BASE lives in architecture/x86_64/cpu.zig), but the *names* would
force an aarch64 port to implement a 'setFsBase' that writes TPIDR_EL0.
Rename to the neutral concept: Task.thread_pointer, PerCpu.loaded_thread_pointer,
architecture.setThreadPointer (x86_64 impl writes IA32_FS_BASE; aarch64 -> TPIDR_EL0).
Also neutralise the user_arg comment (first argument register, rdi on x86_64).
No behaviour change; thread-tls/thread-mutex/smp/process-kill + host tests pass.
runtime.Thread.RwLock (reader-preferring, lock/tryLock/unlock +
lockShared/tryLockShared/unlockShared) and WaitGroup (start/finish/wait), both on
the existing Mutex/Condition.
A compile-time Futex seam gated on builtin.os.tag: the futex syscalls on danos, a
spin+yield mock off-target (Zig 0.16 has no std.Thread.Futex; wake is a no-op
since the state machines re-check). thread.zig is wired into zig build test, so
Mutex/RwLock/WaitGroup run as host unit tests with real std.Thread threads (test
blocks compile only under test, so std.Thread there is fine on freestanding).
thread-rwlock QEMU case: 2 writers set both halves of a value under the exclusive
lock while 3 readers check they match under the shared lock; zero half-write
observations across ~150k reads.
Marks Phase 2 (M7-M11) built. threading.md/threading-plan.md status updated.
Gate: host zig build test covers the sync primitives; thread-rwlock PASS (3x);
full Done gate 26/26 (whole thread-* suite + guardrail); build clean.
Each thread gets its own x86_64 thread pointer (FS base) for user-space TLS.
Task.fs_base is restored on every context switch only when it changes (same
conditional-load discipline as CR3; architecture.setFsBase -> wrmsr
IA32_FS_BASE). New set_thread_pointer=44 syscall sets the caller's fs_base and
loads it now. The kernel never touches FS, so no swapgs complication.
The runtime lays a small per-thread TLS block at the top of each thread's stack
(self-pointer at %fs:0 + scratch) and the thread trampoline calls
set_thread_pointer before any user code — so every spawned thread has a private,
switch-stable thread pointer, reclaimed with the stack.
thread-test tls mode: two threads write unique markers to their own %fs:8 and,
after both wrote, read back — a shared fs.base would clobber one (cross-talk).
Deferred: the Zig threadlocal *compiler* layer (ELF variant-II PT_TLS + linker
sections + template copy) — high-uncertainty, no consumer today; this lands the
load-bearing per-thread fs.base it builds on. See docs/threading-plan.md M10.
Gate thread-tls PASS (3x); full guardrail 25/25; build + host tests clean.
join no longer needs a per-thread IPC endpoint. New thread_join(tid) syscall
blocks the caller until the task with id tid exits; the exit paths call
wakeJoinersLocked. join only reclaims the joined thread's USER stack, which the
thread vacates the instant it enters the kernel to exit, so waking at exit time
(not reap time) is safe — no reaper/aspace juggling or user-memory write, and
equally std-shaped (like pthread_join). thread_spawn drops the exit-endpoint arg
(runtime passes no_cap). thread-test's join mode runs 40 spawn+join cycles that
would exhaust the 16-slot handle table under the old endpoint scheme.
Also harden the M8 reaper: its single per-core reap slot could be overwritten by
a second death on that core before draining (a fresh-task/SMP timing window), an
intermittent one-stack leak that made task-reap ~20% flaky. Replace it with a
per-core reap LIST plus a .reaping task state so a pending slot can't be reused
before its stack is freed. task-reap now 11/11 isolated.
Deferred: detached-thread user-stack reclaim (still at process exit, as in M3).
Gate thread-join PASS (3x); full guardrail 26/26; build + host tests clean.
A dead task's kernel stack was leaked (no reaper), so every process/thread death
bled kernel memory. Now exit()/exitUserLocked record the dying task in a per-core
reap_after_switch slot and switch away; the task that resumes on that core frees
the stack in switchTo's tail (on its own stack, lock still held so the slot can't
be reused). reapKillPendingLocked drains the slot on the timer tick as a safety
net for the fresh-task case (a fresh task enters via the trampoline, bypassing
switchTo's tail). A task killed while not running is freed directly in
destroyTaskLocked. live_stack_bytes is the observable.
Fixed a migration bug this exposed: the post-switchContext reap read the pc
parameter, but a migrated task carries a stale pc in its saved switchTo frame ->
it freed the wrong core's pending stack (a #GP under SMP). Re-fetch thisCpu()
after the switch.
Gate task-reap PASS (5x isolated, 2x in the 24-case batch); full guardrail 24/24
incl. fault-recovery/supervision/process-kill/smp/affinity; build + host green.
Move the mmap/mmio grant-arena cursors off Task into the per-address-space object
(scheduler aspace_refs, exposed via aspaceMmapNextPtr/aspaceDeviceMapNextPtr), so
sibling threads in one address space hand out disjoint grants. systemMmap reserves
a range under a brief lock then maps per page under a short-held lock (not the
whole grant): the big lock runs with interrupts disabled, so pinning it across a
multi-MiB memset+map froze other cores. Guard the runtime heap's rawAlloc/rawFree
with a Thread.Mutex, gated on !single_threaded so ordinary binaries compile it out.
thread-test gains an alloc mode: 4 threads x 500 alloc/fill/verify/free cycles;
any overlap between concurrent allocations is caught by the pattern check.
Also fix the affinity guardrail: its 3-billion-iteration busy-loop had
codegen-dependent wall-time (adding a function to tests.zig swung it ~4s -> ~63s
and timed it out). Reworked to wait on the wall clock instead.
Gate thread-alloc PASS (3x); full guardrail 23/23 green; build + host tests clean.
Adjust the unattended loop contract for Phase 2: the Done condition targets M1
through M11 (M1-M6 being checked is no longer Done), the loop branches off the
current main into a new branch (Phase 1's threading is merged), and each green
milestone pushes the working branch to origin (main stays a human merge).
Add M7-M11 to docs/threading-plan.md, designed so threading reinforces danos's
goals: everything a thread owns lives in the address space (reclaimed on process
death via the M1 refcount), the kernel owns mechanism while the runtime owns
policy, and the process stays the isolation/restart boundary.
M7 thread-safe allocation (per-aspace mmap arena + locked runtime heap);
M8 task reaper (reclaim kernel stacks + detached user stacks — the resilience gap);
M9 futex-completion join (retire the per-thread endpoint, built on M8);
M10 per-thread TLS (threadlocal + fs.base, for self-hosting);
M11 RwLock/WaitGroup + host-testable sync.
New thread_self=42 syscall backs runtime.Thread.getCurrentId (the calling
thread's kernel task id). thread-test gains an id mode: two workers read
getCurrentId and the main thread confirms all three ids are non-zero and
distinct. Per-thread threadlocal TLS is deferred by design (no consumer; it would
need context-switched fs.base for an unused feature), as are RwLock/WaitGroup.
Marks the threading feature built (M1-M6): threading.md + docs/README.md status
updated, all thread-* cases wired into qemu_test.py.
Gate thread-id PASS; full suite green: 21/21 (thread-spawn/join/futex/mutex/id,
aspace-refcount, + 15 guardrail cases), zig build clean, zig build test green.
runtime.Thread.Mutex is the classic three-state futex mutex (unlocked/locked/
contended): the fast path is a single CAS and only a contended lock enters the
kernel. Condition is a futex sequence counter (wait/timedWait/signal/broadcast,
spurious wakeups allowed, use in a predicate loop); a signal racing the unlock
bumps the seq so it is never missed. Semaphore is permits guarded by
Mutex+Condition. All mirror std.Thread's shapes, ported onto runtime.Thread.Futex.
thread-test gains a mutex mode: 2 producers + 2 consumers move 2000 unique items
through an 8-slot ring (small enough that both sides block); the consumed
checksum and tally match exactly, proving the lock and condvars correct under
real cross-core contention.
Deferred with rationale (see docs/threading-plan.md): migrating join to a futex
completion word needs kernel clear-on-exit (else use-after-free munmapping a live
stack); host unit tests need a mockable Futex seam.
Gate thread-mutex PASS (3x); 17 guardrail/thread cases green; build + host tests
clean.
New private syscalls futex_wait(addr, expected, timeout_ns)=40 and
futex_wake(addr, count)=41. A waiter is a .blocked task tagged with
Task.futex_addr (no queue linkage); futex_wait reads the user word under the big
lock and parks only if it still equals expected, so a concurrent wake can't slip
between the check and the block. futex_wake scans the task table and readies up
to count waiters in the same address space. A timed wait also sets wake_at so the
existing wakeExpired times it out; futex_addr staying non-zero (only futex_wake
clears it) distinguishes timeout from a real wake. Waiters park in-kernel, so an
idle core still halts (no busy-wait).
runtime.Thread.Futex mirrors std.Thread.Futex (wait/timedWait/wake). thread-test
gains a futex mode: a waiter parks, the main thread wakes it (serial order
waiting/waking/woke, asserted by the case regex), and timedWait reports a
timeout.
Gate thread-futex PASS (3x); 18 guardrail cases green incl. sleep/event/ipc
blocking paths; build + host tests clean.
thread_spawn takes a 4th arg, an exit-endpoint handle: spawnThreadSupervised
resolves and refcounts it under the spawn lock (like spawnProcessSupervised), so
a thread's death posts a child-exit notification carrying its tid. runtime
Thread.join blocks in replyWait on that (private) endpoint for its tid, then
munmaps the stack; detach relinquishes the join (stack reclaimed at process
exit, for now). New current_core=39 syscall + Thread.currentCore() lets a worker
observe which core it ran on.
The closure now lives at the top of the thread's own (private) stack instead of
the heap, so spawn/join never touch the not-yet-thread-safe runtime heap.
thread-test gains a join mode: 4 workers x 100k atomic increments, joined, with
counter == N*K and >1 core stamped (real parallelism), plus a detached worker.
Gate thread-join PASS (4x, non-flaky); 17 guardrail/M1/M2 cases green; build +
host tests clean.
A thread is a task sharing the caller's address space. New private syscalls
thread_spawn(entry, stack_top, arg)=37 and thread_exit=38: thread_spawn goes
through scheduler.spawnThread (retains the shared aspace), thread_exit ends the
task like a process exit(0) (terminateCurrent -> releaseAspace, so the space
survives while siblings hold it). The closure pointer reaches the new thread in
rdi via a new jump_to_user_arg asm path and a per-task user_arg (0 for a normal
process, whose _start ignores it) - so the runtime trampoline is a plain C-ABI
Zig function, no naked asm.
runtime.Thread (library/runtime/thread.zig) mirrors std.Thread.spawn: mmap a
stack, heap-allocate the args closure, hand the kernel the trampoline + closure.
addThreadedUserBinary opts a binary into single_threaded=false; thread-test is
the first, and proves a worker runs in the shared address space via a shared
global the main thread polls.
Gate thread-spawn PASS; 16 guardrail cases green (incl. args/init/process on the
new jump_to_user_arg path) + aspace-refcount; build + host tests clean.
Route address-space lifetime through a refcount keyed by the page-table root
(scheduler.zig aspace_refs): retainAspace on the spawnUserLocked success path,
releaseAspace from both teardown paths (exitUserLocked, destroyTaskLocked),
destroying the space only when the last task on it exits. Behaviour is identical
today (every space has exactly one task); this is the foundation shared-address-
space threads (docs/threading.md) build on.
Test-observable liveAspaceCount/aspaceDestroyCount + a new aspace-refcount kernel
self-test and QEMU case: spawn and reap 5 ring-3 probes, assert live spaces return
to baseline and destructions advance by exactly 5 (destroyed once each, no leak,
no double-free). Gate passes; 13 guardrail cases green; build + host tests clean.
Add docs/threading.md (native runtime.Thread mirroring std.Thread over a
private thread ABI) and docs/threading-plan.md (6 milestones, each with a
serial-checkable gate + guardrail, plus an unattended /loop execution
contract). Index both in docs/README.md.
Two companion research snapshots on what a minimal display-only native driver
(EDID + mode-set + framebuffer scanout, no acceleration) would take as a danos
.scanout backend, and how the two vendors compare.
- nvidia-gpus.md — RTX 3060 (Ampere GA106). Display is not GSP-gated: nouveau's
ga102.c has a direct register path, and danos's GOP boot lets a driver attach to
a live, already-devinit'd display. Tier-4 effort; the register-level display code
lives in GPL nouveau while the permissive OGKM reference is the hard GSP path.
GA10x is the last NVIDIA family that keeps a register-level display path.
- intel-igpu.md — the companion, and a materially easier, lower-tier target. Intel
publishes register-level Display Engine PRMs, so a clean-room permissive driver
is viable; the DMC microcontroller is optional (power states only), scanout comes
from system RAM via the GGTT (no VRAM manager), and coreboot's libgfxinit is a
compact native reference. The catch is hardware: an iGPU may not drive the
monitor on a discrete-GPU desktop.
Sourced, cited snapshots — not implementations; each ends with a "first light"
milestone ladder framed as a danos .scanout service. Not yet linked from
docs/README.md.
The kernel enabled SSE at boot and both kernel and userspace keep live values in
XMM (LLVM emits movdqu/movaps for >=16-byte struct copies, plus floats and SIMD),
yet the kernel never saved the SSE/FPU register file anywhere — not across
switch_context, not across the syscall boundary, not across interrupts. Any value
the compiler parked in an XMM register across a kernel entry could be silently
clobbered by kernel code, or by whatever the scheduler ran while the task blocked:
a whole-kernel, timing-dependent data-corruption bug.
It was the real root cause of the "device_enumerate corruption" Heisenbug: the
display service read garbage framebuffer geometry (height=0) because findDisplay
held the 16-byte .display field live in xmm0 across the claim/mmio_map syscalls,
and a timer preemption to the busy device-manager clobbered it. The tell that it
was register-only: memory always read correct, and the bug vanished whenever an
added syscall spilled the value to the stack.
Fix: fxsave/fxrstor the register file in the asm stubs (isr_common and
syscall_entry), right after pushing the GP trap frame — before any Zig kernel code
can touch XMM — and right before the pops. rbx bridges the exact rsp across the
call to interruptDispatch: it is callee-saved, so it survives even a blocking
dispatch that context-switches away and back, and `and $-16,%rsp; sub $512,%rsp`
gives fxsave its 16-byte-aligned scratch on the kernel stack. Context-switch-time
save/restore alone is not enough — kernel code between the interrupt and switchTo
already clobbers XMM.
The commit-58927ed Gop.init workaround (copy scalar geometry fields rather than the
whole descriptor by value) is now redundant but harmless; left in place. Deferred:
a fresh task inherits the previous task's XMM (minor info-leak / nondeterminism),
and the fxsave runs on every interrupt including ring0->ring0.
pciScanTest put three [64]DeviceDescriptor arrays on the stack; at 328
bytes each that is a 66,096-byte frame, larger than the 64 KiB bootstrap
stack the boot context runs on, so the prologue overflowed the stack and
triple-faulted on entry before the test could print anything — the CPU
reset with no exception line. Reuse one descriptor buffer (frame ~21 KiB).
With the fault gone, the restart-drill checks proved flaky. They polled
process.write_buffer (which holds only the latest write-syscall message)
for the transient restarting/re-scan lines, which the concurrent
class-driver output overwrote before the starved low-priority poller could
observe them. The kernel broker count can't witness the restart either:
register is idempotent and the table has no unregister, so the count is
invariant across kill/prune/respawn.
Follow the sibling restart drills (usbReportTest/driverRestartTest): assert
the ordered drill in the harness regex over the full serial log (a
backreference requires the respawn to re-scan the same count), and keep only
race-free broker checks in the kernel — empty before the scan, populated
once it settles, never growing past N — sampled via scheduler.sleep at a
fixed cadence instead of a busy-poll.
The compositor now survives the virtio-gpu driver dying and re-attaches when device-manager
restarts it — the last piece of display v2.
Three parts:
- The driver hellos the device manager (role: bus). It never did, so the manager — which
spawns it supervised and expects a hello — was stopping it at the 3s hello deadline every
run (the gate markers just printed first). Now it is properly supervised: not stopped for
silence, and restarted on death.
- A kernel IPC fix so a call to a dead service errors instead of hanging. An endpoint records
its owner; when that task dies, its registered endpoints are marked dead (and any parked
senders woken with -EPEER), so ipc_call returns -EPEER rather than blocking on a reply that
will never come. Without this the compositor's first present after the driver died blocked
forever. General robustness — any client of any service benefits.
- The compositor re-attaches. Its .scanout calls now fail cleanly (caught), freezing the last
frame; when the restarted driver re-announces, attach_scanout detects the backend is already
virtio and logs "scanout re-attached", mapping the fresh shared surface and re-looking-up
.scanout. (The previous shm mapping leaks — no shm_unmap syscall yet — but its frames are the
dead driver's, reclaimed on exit.)
- device-manager gains a "test-scanout-restart" mode (like test-usb-restart) that kills the
virtio-gpu driver once after it hellos; the displayReattachTest kernel scenario drives it.
Gate: python3 test/qemu_test.py display-reattach — "scanout upgraded to virtio-gpu" then
"scanout re-attached", no CPU exception, passing 3/3. host tests, ipc/ipc-call/ipc-cap,
supervision, shm, display-service, display-demo, virtio-gpu, display-native, and
display-modeset all pass; default zig build clean. v2 (V1-V6) complete.
The native backend can now change resolution and presents tear-free.
Mode-setting without churn. The driver sizes its scanout resource + shared surface to the
largest mode it offers and treats a mode change as re-pointing the scanout rectangle within
that surface — so the resource, its backing, and the shared mapping never change, and the
surface's row stride (the max width) is fixed while the active width/height move. The
compositor is handed that stride in the announce and composes at it; a smaller mode just
paints the top-left rectangle. This sidesteps the surface re-share a true resolution change
would otherwise need (the service harness can't reply with a capability).
- scanout-protocol gains get_modes + set_mode; the driver offers {640x480, 800x600} and
re-points set_scanout on set_mode.
- backend.VirtioGpu carries the surface stride, exposes modes()/setMode(), and reports
canModeSet = hasVsync = true.
- runtime.display gains modes()/setMode() (display-protocol get_modes/set_mode, forwarded to
the backend) — the client-facing API.
- EDID: the driver negotiates VIRTIO_GPU_F_EDID when the device offers it, reads the monitor's
EDID, and logs its preferred mode (parsed from the first detailed timing descriptor).
- vsync: every resource_flush is fenced (VIRTIO_GPU_FLAG_FENCE); the device signals the fence
when the frame is on screen, which the used-ring ack the synchronous present already waits
on gates — so a completed present is a tear-free one.
After the native upgrade the compositor runs a one-shot mode-set self-check: query the modes,
switch to a different one, re-composite, and confirm the backend reports the new geometry —
the gate's markers.
Gate: python3 test/qemu_test.py display-modeset (reuses the display-native boot) — "display:
mode set to 800x600, verified" + "display: vsync present ok", passing 3/3. host tests,
display-service, display-demo, shm, virtio-gpu, and display-native still pass.
The compositor now boots on the GOP framebuffer and upgrades to the virtio-gpu driver
the moment it announces itself — the pluggable-scanout payoff.
The shared surface. The scanout resource is an shm region the driver creates
(shm_physical, a new syscall, hands it the guest-physical for attach_backing) and passes
to the compositor as a capability. The compositor maps it and composites straight into
it: on x86 DMA is cache-coherent, so the cacheable shared pages the CPU paints are exactly
what the device transfers-and-flushes — no copy, no explicit flush.
The handshake. After bring-up the driver looks up .display and sends attach_scanout with
the geometry + the surface capability. The compositor maps the surface, looks up the
driver's .scanout endpoint itself (the driver registered it — no need to pass it), switches
to backend.VirtioGpu, and re-composites the current frame. present() over the native
backend is a present request on .scanout -> transfer-to-host + resource flush. The first
native present is deferred to a one-shot timer: presenting inline from the announce handler
would deadlock, since the driver is still blocked on our reply and not yet serving .scanout.
After it lands, the compositor reads a pixel back from the shared surface to confirm the
frame reached the device's backing.
- shm_physical (syscall 36) + runtime.shm.physical.
- scanout-protocol (the compositor->driver present channel), separate from the
client-facing display protocol; the display protocol gains attach_scanout.
- backend.VirtioGpu joins backend.Gop in the tagged union; select() still boots GOP.
- the virtio-gpu driver's scanout backing is now shm (was DMA); it announces + serves
.scanout present requests (transfer-to-host + flush of the shared surface).
Also fixes a latent framebuffer-geometry corruption the display service hit only when it
enumerated the device tree alongside a busy device-manager: Gop.init now captures the
geometry into a small value the instant device_enumerate returns (rather than re-reading
the 328-byte descriptor across the later claim/mmio_map syscalls) and retries on a zero
geometry. The underlying device-table clobber is a separate kernel bug, tracked apart.
Gate: python3 test/qemu_test.py display-native (QEMU -device virtio-gpu-pci) — "display:
scanout upgraded to virtio-gpu" + "display: native present verified" + "display-demo: ok",
passing 3/3. host tests, display-service, display-demo, shm, and virtio-gpu still pass.
The first native scanout backend's driver half. The device manager matches the
virtio-gpu PCI function by its Display/Other class triple and spawns the driver,
which claims the device, confirms vendor 0x1AF4/device 0x1050 from config space,
enables memory-space + bus mastering, and walks the virtio vendor capabilities to
find the common-config and notify structures in its BAR.
From there it is the standard modern-virtio bring-up: reset, negotiate VERSION_1,
stand up the control virtqueue in coherent DMA, then drive the GPU end to end —
RESOURCE_CREATE_2D, ATTACH_BACKING (a coherent DMA buffer; V4 swaps in the shm-
shared surface), SET_SCANOUT, paint a known pattern, TRANSFER_TO_HOST_2D,
RESOURCE_FLUSH, and wait for the device's used-ring ack. Reading the backing back
proves it is CPU-visible RAM; the ack proves the device consumed the frame —
together the automated stand-in for "it's on screen", no screenshot.
- system/drivers/virtio-gpu/: the driver, plus virtio-gpu-protocol.zig (control
commands) and virtio-pci.zig (the 1.0 PCI transport + split-virtqueue), both with
host-tested struct sizes.
- device-manager matches the display/other class triple to "virtio-gpu"; the driver
self-confirms the vendor/device id, since the class alone cannot distinguish it.
- ServiceId.scanout (11): the driver registers it so the compositor finds it in V4.
Gate: python3 test/qemu_test.py virtio-gpu (QEMU -device virtio-gpu-pci) — the
device-manager stack discovers the function, the driver brings up a 640x480 scanout
and flushes a test pattern: "virtio-gpu: scanout 640x480 online" + "flush acked,
pixel check ok". host tests, display-service, shm, and device-list still pass.
Note: the pre-existing pci-scan case triple-faults on main (verified at 88ad432,
before this change); it is unrelated and tracked separately.
Generalize capability passing from endpoints to memory objects. The per-task
handle table now holds kind-tagged entries (scheduler.HandleObject{kind, ptr});
closeHandles and shareCapability dispatch by kind, so a shared-memory object
rides an ipc_call send_cap exactly like an endpoint and is refcount-freed only
when its last capability drops.
- shm_create(len) -> vaddr, handle: contiguous, zeroed, cacheable frames wrapped
in a refcounted ShmObject, mapped into the caller's shm arena (PML4[230]).
- shm_map(cap) -> vaddr: map the same physical pages into a receiver that got the
capability. mapUserSharedInto maps WB-cacheable + device_grant, so a sharer's
teardown never frees the shared frames — the object owns them.
- runtime.shm: create(len) -> Region{ptr, handle, len}, map(handle) -> ptr.
Gate: qemu_test.py shm — shm-client creates a region, writes a pattern, passes
its capability to shm-server, which maps it and reads the same bytes back
(shm: shared 4096 bytes ok). ipc/ipc-call/ipc-cap/supervision/dma/usermem/
display-service and host tests all still pass — the handle change broke no IPC.
Extract scanout from the compositor into backend.zig — a `Backend` tagged union
with info()/surface()/present(damage) and canModeSet/hasVsync flags. The v1 GOP
path becomes `backend.Gop` (claim the display node, WC-map the LFB, keep the
cacheable back buffer, present = the damage-rect WC copy); display.zig now
composes into backend.surface() and calls backend.present(damage), with no LFB or
framebuffer geometry left in the compositor core. The selection decision is the
pure chooseKind(native_available), split from the syscall-bound bring-up, ready
for the native-if-present branch at V4.
Pure refactor: display-service, display-demo, and zig build test all pass
unchanged.
Rewrite V3/V4/V5 gates so none need a screenshot: the driver/compositor read
their own pixels back (the scanout resource is shm-backed CPU-visible RAM), and a
virtio resource_flush is confirmed by the device's used-ring ack / fence. Pixel
readback + flush-ack is the serial stand-in for "it's on screen," so the whole
V1→V6 plan can run and self-verify without a human eyeballing QEMU.
Keep the GOP framebuffer as the floor, make scanout a pluggable backend, and
upgrade to a native virtio-gpu driver when it announces itself (dynamic
hot-attach; GOP stays the fallback for "no driver ever"). v2 builds the shm
cross-process memory capability, shared with the future client-surface path.
Milestones V1 (backend seam) → V6 (resilience + tests), each with a gate.
init supervises its boot services (spawned against an exit endpoint) but the
child-exit handler was `if (got.isNotification()) continue;` — it silently
dropped a dead service. Now init restarts it: on a child-exit notification it
finds the service, and unless it exited cleanly (chose to stop) or has hit the
crash-loop cap (maximum_restarts), respawns it and logs the death + reason. The
reincarnation half of resilience (docs/resilience.md) at the service level, the
counterpart to the device manager's driver restarts. A `shutting_down` flag
skips restarts during the orderly stop sequence, whose child deaths are expected.
Daniel Samson
daniel