The engine gained mkdir/unlink in Phase 2a; this exposes them as first-class
filesystem operations so programs can use them.
- vfs protocol: two new path-based operations, mkdir and unlink (appended, so
existing opcodes/offsets are unchanged).
- VFS router: a forwardPath helper relays a path-based op under a mount to its
backend; the mkdir/unlink cases forward to the mounted filesystem (the flat
ramfs refuses them — it has no directories).
- fat server: mkdir -> engine.createDirectory, unlink -> engine.removeFile, each
resolving the parent via a shared splitParent helper (also used by open-create).
- runtime.fs: makeDirectory(path) and remove(path).
- fat-test now exercises the whole path — mkdir /mnt/usb/TESTDIR, create + write +
read a file inside it, then remove it — behind a new `fat-mutations` QEMU case.
Verified: zig build, zig build test, zig build check-fat-image, and a sequential
QEMU sweep — fat-mount, fat-mutations (mkdir/write/read/unlink through the mount),
vfs, vfs-client-death, log-flush, orderly-shutdown, initial-ramdisk, smoke — green.
The FAT engine could create, read, write, and grow files, but never free clusters
or make directories — so overwriting a shorter file left a stale tail (a real bug:
corrupt boot-log re-flushes, and later corrupt compiler cache/.o files). This adds
the mutation half of the engine, with the corruption fix wired all the way through.
Engine (system/services/fat/engine.zig), all host-tested:
- freeChain: return a cluster chain to the pool (bounded against a corrupt cycle) —
the shared primitive under truncate and remove.
- truncate: free the chain and zero the entry's size/first-cluster (O_TRUNC).
- createDirectory (mkdir): allocate + initialise a cluster with "." and ".." and add
the directory entry to the parent.
- removeFile (unlink): free the chain and mark the 8.3 entry plus any preceding
long-name entries deleted, so a reused slot can't inherit an orphaned long name.
Refuses directories.
- createFile now shares a common addEntry helper with createDirectory.
O_TRUNC wired end to end: a truncate open-flag (vfs protocol) that the router already
forwards; runtime.fs.OpenOptions.truncate; and fat's handleOpen calls engine.truncate
on an existing file. The boot-log flush (log-flush + init) now opens with truncate, so
a shorter log on a later boot of the same stick leaves no stale tail — closing the
caveat from the boot-log work.
mkdir/unlink are engine-complete and host-tested but not yet exposed as VFS
operations / runtime.fs methods (they are new path-based ops needing router cases);
rename and the richer stat (mtime/mode, blocked on wall-clock) remain. See
docs/zig-self-hosting.md (Phase 2).
Verified: zig build, zig build test (8 engine host tests, incl. truncate, the
overwrite-no-stale-tail regression, remove, and mkdir), zig build check-fat-image, and
a sequential QEMU sweep — fat-mount, log-flush (DANOS.LOG read back at 11804 bytes,
clean, with the truncate-based final flush), vfs, vfs-client-death, usb-storage,
orderly-shutdown, initial-ramdisk, smoke — all green.
The first step of the Zig self-hosting roadmap (docs/zig-self-hosting.md): give
danos programs a danos-native file API and remove the premature POSIX compatibility
shim. This also resolves the earlier misplacement of a full-write helper into the
compat layer — that behaviour now lives natively in runtime.fs.File.writeAll.
- library/runtime/fs.zig: the danos-native file client over the VFS (open/read/
write/writeAll/seekTo/attributes/close, directory listing, mount). Handles are
*values* — a File/Directory owns its VFS node id and byte offset — so there is no
per-process fd table or descriptor limit, unlike the POSIX fd model the shim
emulated. This is where the operations that later become std.os.danos are staged.
- Retire library/posix/ (unistd, stdio): only five call sites used it, all file
operations, all migrated to runtime.fs — fat (mount), the vfs-test and fat-test
clients, and init/log-flush (the boot-log flush). stdio was already dead.
- build.zig: drop the posix module, its addUserBinary parameter, the per-binary
import, and the ~26 call-site arguments.
- Docs: the VFS protocol's client is now runtime.fs; the docs index and
coding-standards note posix is retired and the foreign-ABI naming exception now
applies to the future std.os.danos seam; the process-lifecycle note points the
future musl layer at that same seam rather than the deleted directory.
Deferred by design (see the roadmap): the C-ABI runtime.os errno seam is built at
fork time (its shape must match std/os/danos.zig); truncate/mkdir/rename are
Phase 2; stdio-byte fds and cwd are later slices.
Verified: zig build, zig build test, zig build check-fat-image, and a sequential
QEMU sweep — vfs, vfs-client-death (the park/hold-handle path), fat-mount, log-flush,
orderly-shutdown, initial-ramdisk (log-flush silent in the bare sweep), smoke, init,
usb-storage, device-manager — all green.
On a headless or real board nothing captures serial, so the boot log — the whole
diagnostic stream — is lost at power-off. This retains it in the kernel and copies
it to the boot USB volume as /mnt/usb/DANOS.LOG, the on-disk equivalent of QEMU's
`-serial file:`. Pull the stick, read DANOS.LOG on another machine.
How it fits together:
- Kernel RAM sink (log.zig): a fixed 256 KiB in-image buffer registered as a log
sink in kmain, right after serial. Because userspace debug_write funnels through
log.write, it captures the entire stream — kernel lines and every service's
output — from the first line. Fills linearly and stops when full (earliest boot
output, the most valuable, is kept); no allocation, so it is panic-safe.
- klog_read syscall (32): copies that buffer out to a user buffer, the mirror of
debug_write — same overflow-safe user-half bounds check, kernel -> user copy,
under the kernel lock so the snapshot can't grow mid-copy. Wrapped by
runtime.system.klogRead.
- log-flush (new one-shot, in the initial-ramdisk): waits for the fat server to
mount /mnt/usb, then copies the whole log to /mnt/usb/DANOS.LOG. init spawns it
once the boot services are up (fire-and-forget; it polls the mount itself). If
no volume is mounted — no stick, or the no-VFS ramdisk sweep — it exits silently.
- init shutdown flush: init repeats the copy inline at the top of shutDown(),
BEFORE it tears down the storage services (the fat server is stopped first), so
a clean poweroff captures the fullest log while /mnt/usb is still writable.
- unistd.writeAll: loops write() past the 224-byte VFS payload cap; both flush
paths use it.
The filename is 8.3 (DANOS.LOG) at the mount root — the FAT short-name rule, and
there is no mkdir on the FAT path yet. Extend-only writes mean the two same-session
flushes never leave stale bytes (the shutdown log is a superset of the boot log);
a shorter log on a later boot of the same stick can leave a stale tail — a noted,
cosmetic limitation, not worth pulling O_TRUNC into the FAT write path for now.
Verified end to end under QEMU: a new `log-flush` case (reusing the orderly-shutdown
build) asserts both markers then S5, and DANOS.LOG is read back out of the image
afterwards (11804 bytes, containing the kernel init line, the FAT mount line, and
the boot flush marker). Regression stays green: zig build, zig build test, zig
build check-fat-image, and a sequential QEMU sweep — smoke, init, initial-ramdisk
(log-flush silent in the bare sweep), orderly-shutdown, fat-mount, usb-storage,
usb-hid, vfs, input, device-manager, process, signals, dma, fault-pf.
Add a FAT12/16/32 filesystem the VFS mounts at /mnt/usb, reading and writing a
USB stick through the block device. Verified end to end under QEMU: the fat
server mounts the volume, the VFS routes /mnt/usb to it, and a client lists the
root and reads a file (the ELF magic of /mnt/usb/system/kernel).
- engine.zig: the FAT engine over a BlockDevice interface — mount (a bare FAT or,
as QEMU's VVFAT and most real sticks present it, an MBR-partitioned disk), FAT
chain walk (12/16/32), cluster allocation, directory traversal with long-name
read, and file read / write / create. Host-tested against a RAM-backed FAT16
image (create, cluster-spanning write, mid-file overwrite, read-back, list).
- on-disk.zig: the align(1) boot-sector / directory / long-name / FSInfo structs
and the cluster-count FAT-type detection.
- fat.zig: the server — wraps the .block device (a DMA bounce buffer) in a
BlockDevice, mounts the FAT, serves the vfs-protocol as a backend, and mounts
itself into the VFS at /mnt/usb. Spawned by init as a boot service.
- runtime.block: the block-device client (geometry / read / write by physical
address, so whole sectors never cross IPC).
- Raise the kernel service-name registry (maximum_services) 8 -> 16: it is
indexed directly by ServiceId, and fat = 8 was being rejected, so the fat
server exited before registering.
- VFS: an absolute path with no matching mount is now not-found rather than
silently created in the flat ramfs — so /mnt/usb fails cleanly until mounted.
Tests: fat-mount (the full stack: block -> FAT -> VFS mount -> list + file read)
passes; host units cover the engine and on-disk structs; the vfs, shutdown, and
USB regression suite stays green (10/10).
Turn the flat-ramfs VFS into a router: a mount table maps an absolute path prefix
(e.g. /mnt/usb) to a backend server's endpoint, and open/read/write/status/
readdir/close on a path under a mount are forwarded to that backend, which speaks
the same vfs-protocol. This is what a FAT filesystem mounts into.
- protocol: append readdir / mount / unmount operations, a NodeKind enum (the FSH
file types) that now fills FileStatus.kind, a DirectoryEntry record, and a
directory open flag. Appended values keep existing clients and tests unchanged.
- vfs.zig: a mount table, longest-prefix routing, forwarding of every op on a
backend handle, mount/unmount handlers (the backend arrives as the call's
capability), and release-on-death that also closes the backend's handles.
- path.zig: pure, host-tested mount-prefix matching that never captures a
non-boundary like /mnt/usbextra.
- unistd: mount(), opendir / readdir / closedir clients.
Bare names still resolve in the flat ramfs — the backward-compat contract; the
vfs and vfs-client-death tests pass unchanged. End-to-end mount+read is exercised
by the FAT server (M6). Host units cover path matching and protocol sizes.
Flesh out the xHCI host-controller driver into a full transfer engine and build
the three USB class drivers on top, all verified end to end under QEMU.
- xHCI engine (usb-xhci-library.zig): controller reset, command/event rings with
cycle-bit bookkeeping (gated on a No-Op-command proof), device slots, Address
Device, control transfers, full chapter-9 enumeration, Configure Endpoint, and
interrupt/bulk transfers. Each interface is device_registered with its
(class,subclass,protocol) identity, unique per (port,interface).
- Bus<->class transfer protocol (usb-transfer-protocol.zig + runtime.usb): open /
control / interrupt-subscribe (async report pump on a poll timer) / bulk-by-
physical-address, so sector data never crosses the 256-byte IPC limit.
- USB HID keyboard + mouse (usb-hid/): decode boot-protocol reports and publish
to the input service. A USB usage is already the input protocol's keycode.
- USB mass storage (usb-storage/): Bulk-Only Transport + transparent SCSI,
serving a block device under the new .block service id (block-protocol).
- device-manager matches USB interfaces to class drivers (usbDriverForIdentity).
- usb-abi / usb-ids made importable modules; add HID and mass-storage class
requests, packTriple, and a usb_device DeviceClass.
- Fix test/qemu_test.py on macOS: the QMP unix-socket path was built from the
deep worktree path and exceeded the 104-byte sun_path limit, so QEMU exited
before booting. It now lives under a short temp path.
Tests: usb-report, usb-hid, usb-storage pass under python3 test/qemu_test.py;
host units (usb-abi, usb-ids, hid-report, bulk-only-transport, scsi) green.
Time is a kernel concern in danos: the kernel owns the scheduling timer and
already exposes monotonic time via the clock/sleep/timer_bind syscalls, so a
userspace time service would be a redundant, slower path. This adds the generic
runtime.time module over those syscalls, retires the two demonstration drivers,
reorganizes the milestone docs, and makes the monotonic clock correct on Intel,
AMD, and inside any VM.
runtime.time (library/runtime/time.zig)
- Instant/Duration interface: now, sleep, spin, after, monotonicNanos, available
- a thin layer over system.clock/sleep/timerOnce; unit-tested arithmetic
Remove the demo drivers hpet and bus (a teaching example belongs in the docs,
not shipped in the tree)
- system/drivers/ now holds only real drivers: pci-bus, ps2-bus, usb-xhci-bus
- device-manager end-to-end test repointed to pci-bus (asserts on kernel state:
the process table and the device tree, not a racy serial marker)
- device_register containment moved to a new in-kernel `containment` test
- the driver-model worked example moved inline into docs/drivers.md
Reorganize milestone docs into topic docs
- m17-m18 / m19-m20 / m21 plans dissolved into process-lifecycle, device-manager,
discovery, and acpi docs; new docs/power.md and docs/timers.md; ~20 citations
repointed; plan docs deleted
TSC reliability (apic.zig, smp.zig, cpu.zig, kernel.zig)
- check the invariant-TSC bit (CPUID 0x80000007 EDX[8]) on Intel and AMD
- cross-core "warp" check at SMP bring-up, pairwise BSP<->AP as each core comes up
- fall back to the HPET clocksource when the TSC is not invariant (a bare VM) or
not synchronized (a warp), switched continuously so time never jumps
- boot log reports the outcome; new tsc-sync test exercises the TSC + warp path
Verified: zig build; zig build test; 60/60 QEMU cases (incl. new containment and
tsc-sync).
The kernel publishes the FADT as one more acpi-tables memory resource
(tagged by its intact FACP header — the AML blobs are header-stripped);
the acpi service reads the PM1 event/control and GPE register ports from
that copy, so the kernel's own FADT parse is untouched. A power-protocol
module (ServiceId.power = 5, domain-named so an ARM PSCI service can serve
the same id) carries subscribe / shutdown / events.
The acpi service converts to runtime.service.run — device discovery, the
.power protocol, and the SCI notification all fold into one loop. At
startup it enables ACPI mode if SCI_EN is clear (the SMI dance), binds the
SCI (found as the node's len-1 irq resource, distinct from the broad
window), and sets PWRBTN_EN. On the SCI it reads PM1_STS, clears
PWRBTN_STS write-1, logs the press, publishes power_button to
subscribers, and always acks. The power-button scenario proves it with a
real QMP system_powerdown injected mid-run through the M21.0 channel.
Applications ask the device manager for the tree (enumerate: a header
plus ChildEntry records) and subscribe to published add/remove events by
handing their endpoint over as the call's capability — the input-service
pattern; events are the same ChildAdded/ChildRemoved structs the bus
drivers send, one encoding in both directions. device-list is the first
client: it prints the tree, subscribes, and narrates the events through
a driver restart. The protocol's message maximum is capped at the
kernel's IPC MESSAGE_MAXIMUM (256 bytes, ten entries per reply; paging
joins the protocol when a tree outgrows one message). The startUserTask
debug print is gone: it wrote to serial unserialized against user-space
lines and sheared concurrent log markers in half — the root cause of the
scenario flakes.
The manager is now a harness service on the well-known .device_manager
endpoint. Every driver spawns supervised; drivers with an assignment must
hello (device-manager-protocol, versioned) within a deadline enforced by
a timer sweep. Exit reasons drive the restart decision: clean exits stay
down, faults restart with 300/600/1200ms backoff, and three fast deaths
mark a driver failed instead of respawning forever. usb-xhci-bus is the
first conforming driver; the crash-test fixture claims a device, hellos,
and faults on purpose — each respawn re-proving claim release on death
through the manager's own path. maximum_tasks grows 16 -> 32: the
initial-ramdisk sweep (15 binaries at once) was intermittently
overflowing the static pool.
Signals are statements delivered as coalescing notifications to the
endpoint a process nominates with signal_bind — never a hijacked stack,
never a question (liveness is the zero-length ping the harness answers).
process_signal is supervisor-or-self gated, like kill; unbound targets
accumulate a pending mask delivered on bind. timer_bind is the missing
timed wait: a one-shot deadline landing in the same replyWait as
everything else — what stop(), hello deadlines, and restart backoff are
built from. runtime.service.run folds requests, signals, and
notifications into callbacks; the VFS conversion deletes its hand-rolled
loop and gains the whole lifecycle contract. The signals scenario drives
ping, reload, terminate->exited, the timer, and the deaf-child
deadline->killed path from ring 3. docs/process-lifecycle.md increments
1-4 are now as-built.
process_subscribe adds an endpoint to a bounded, ref-counted subscriber
table; every death posts the same badge encoding a supervisor's exit
notification uses, equally late, so subscribers observe a fully-released
child. A dying subscriber's own subscriptions are removed first — it never
hears about itself. The VFS is the first subscriber: open handles now
record their owner and are swept when the owner dies, because a service
must never depend on clients cleaning up after themselves
(docs/process-lifecycle.md). Proven by the vfs-client-death scenario.
The kernel records an ExitReason at all three death sites — clean exit,
fault (classified by vector), and process_kill — into a bounded ring
before the exit notification posts, so a supervisor's query never races
the notice. process_exit_reason is gated by the same supervisor check as
kill; runtime.process.exitReason is the stable interface. This is the
input restart policy reads (docs/process-lifecycle.md iron rule 2).
Turn a keycode + modifiers into a character. The input module delivers HID
usage keycodes but nothing mapped them to characters; rather than hand-maintain
layout tables, vendor the X11 xkeyboard-config database and compile it to native
Zig at build time (no X11 runtime), the way make-initial-ramdisk.py packs the
ramdisk.
- tools/make-xkeyboard-config.py: `fetch` downloads the pinned xkeyboard-config
release (2.44, sha256-verified), resolves the include graph for the configured
layouts, and vendors only the reached symbols files + keysymdef.h + COPYING +
PROVENANCE into library/xkeyboard-config/vendor/. `generate` parses that
(keycodes via a HID->xkb-name table, symbols with include/augment/override and
per-key type, keysymdef for keysym->Unicode) and emits generated/layouts.zig
deterministically.
- library/xkeyboard-config/xkeyboard-config.zig: the API over the generated data
— map(layout, hid_usage, mods) -> { keysym, character }, byName, and the
level-selection semantics (the generated tables stay pure data). Host tests
assert US letters/digits with Shift/Caps, GB £ vs US # on Shift+3, and French
AZERTY q-where-US-has-a — the end-to-end proof of the parse->emit->lookup path.
- build.zig: `xkeyboard-config` + `layouts` modules, the test wired into
`zig build test`, and a `zig build gen-xkeyboard-config` convenience step.
- Layouts: us, gb, de, fr, es, dvorak. Scope (documented): group 1, no dead-key
composition, curated key types. Standalone library; wiring it into the input
path to fill KeyEvent.character is a documented follow-up.
zig build test green (incl. the new keymap tests); regeneration is byte-identical;
full QEMU suite 48/48 (unaffected — no kernel/runtime/service change).
Extend the input service beyond the keyboard so mouse and joystick/gamepad
drivers can broadcast too, with per-device publish and subscribe methods.
- protocol: KeyEvent joins MouseEvent (motion/buttons/scroll) and
JoystickEvent (axes/buttons), all carried in a common InputEvent envelope
tagged with a DeviceKind. A subscribe request carries a device_mask, so a
subscriber names the classes it wants and the service routes each event only
to interested subscribers (a mouse-only listener never wakes for keystrokes).
- runtime: per-device publish methods (publishKeyboardEvent/publishMouseEvent/
publishJoystickEvent) and subscribe helpers (subscribeKeyboard/Mouse/Joystick,
each typed, plus subscribe(mask)/subscribeAll returning the tagged envelope).
- service: subscriber table gains a device_mask; broadcast routes by the
event's device class.
- mouse driver now publishes (synthetic) mouse events like the keyboard driver;
input-source cycles all three classes; input-test subscribes to all and only
emits its "ok" marker once it has received one of each class — so the passing
test proves per-device routing, not just delivery. Real HID decoding stays a
follow-up.
No kernel changes: ipc_send is generic and the 36-byte InputEvent fits its
64-byte payload. Full QEMU suite 48/48; serial log confirms keyboard, mouse,
and joystick all reach one subscription.
Programs can now subscribe to keyboard events (key_down/key_up/key_press)
and drivers can broadcast them, through a new user-space input service.
The delivery model is forced by danos IPC: a synchronous rendezvous holds
one pending reply, so a server cannot park N subscribers blocked in a
"wait for next event" call — delivery must be push. But a synchronous push
has no timeout and the kernel never wakes a sender parked on a dead peer's
endpoint, so one dying subscriber would hang all input. So this lands the
roadmap's planned asynchronous buffered send and builds the service on it:
- ipc_send (syscall 26): non-blocking post to an endpoint's bounded payload
ring, delivered through reply_wait as a buffered message (notify_message_bit).
A full ring drops the oldest. It can never hang on a dead/slow peer.
- input-protocol + runtime.input helpers (subscribe/next, connectSource/
publish) — the first real consumer of M13 capability passing: a subscriber
hands the service its own endpoint as a capability.
- input service (fan-out via ipc_send, dead-subscriber pruning), a synthetic
input-source, and input-test; the ps2-bus keyboard driver publishes to it.
Real IRQ1 scancode decoding (which must live in the bus, the PNP0303 owner)
is a documented follow-up; the source is synthetic for now.
- build/init wiring, an `input` QEMU case, and docs/input.md.
Full QEMU suite 48/48, including the new input case and every IPC/endpoint
regression (ipc, ipc-call, ipc-cap, vfs, hpet, bus, irqfree).
process_enumerate snapshots the task table (the device_enumerate shape, so
ps is a user program); system_spawn returns the child id, records the caller
as supervisor, and takes an exit endpoint; process_kill is allowed only for
the supervisor. Every death — exit, fault, or kill — posts a child-exit badge
to that endpoint (the IRQ-as-IPC pattern as SIGCHLD). A target caught off-CPU
is reaped in place; a running one is condemned and finished at its next
system call or tick, guarded so teardown never lands mid-kernel-operation.
Tested by process-list, process-kill, and supervision (a ring-3 supervisor
exercising the whole surface); design notes in docs/process-management.md.
Processes now start with C-compatible arguments: the kernel builds the
System V AMD64 entry block (argc, argv, empty envp, auxiliary vector)
at the top of the stack, argv[0] is the path or initial-ramdisk name
the process was spawned as, and system_spawn carries an optional
NUL-separated blob that becomes argv[1..]. The runtime parses the block
(runtime.argumentCount/argument) and its spawn wrappers pass arguments
through. The name is also recorded on the task, so a fault report says
which binary died, not just its id.
The user stack grows from one page to eight (32 KiB,
parameters.user_stack_pages), with the page below left unmapped as a
guard so an overflow faults into a clean process kill rather than
corrupting the image. Task.name_buffer is zero-initialised, not
undefined: an undefined default is materialised as a 0xAA fill that
moved the static task pool out of .bss and made the whole kernel ~7x
slower under QEMU TCG (caught by the affinity test).
Proven end to end by the new args test: args-echo respawns itself with
arguments via the syscall blob, burns more stack than one page could
hold, and echoes its argv intact. Full suite: 44/44.
Ring 3 still has no direct in/out (no TSS I/O bitmap, IOPL never raised — a #GP), but a
driver no longer needs it: io_read(device_id, resource_index, offset, width) and
io_write(..., value) grant port access the same way mmio_map grants memory. The claim
plus the device's discovered io_port resource are the capability — resolveIoPort checks
the device is claimed by the caller, the resource is io_port, and [offset, offset+width)
stays inside it, then issues the in/out via architecture.pioRead/pioWrite. So a PS/2 or
16550 driver is now writable; the low-rate legacy hardware that needs port I/O is fine
with a syscall per access. io_port resources were recorded by discovery and ignored —
now they're used. Runtime: device.ioRead/ioWrite.
New `ioport` test claims QEMU's PS/2 controller (io_port 0x64, discovered via ACPI) and
checks the gate admits an in-range access, refuses over-wide / out-of-range / unclaimed,
and that the kernel actually reads the status port (0x1c). Suite 41/41 plus host tests.
Docs refreshed for the whole M13-M16 + port-I/O reality: drivers.md "Limits" no longer
lists port I/O, DMA memory, or barriers as missing (they exist) and its "what's next"
reflects that; the FSH doc's character-device and block-device sections are corrected
("cannot host a block driver at all" is no longer true — writable now, not yet memory-
safe pending IOMMU enforcement); device-interrupts.md unblocks the keyboard and narrows
the interrupt gap to MSI-X.
Two parts, both blockers for real PCI drivers.
ECAM config space per function: enumeratePci now gives every pci_device its own 4 KiB
configuration window as resource 0. A claimed PCI driver mmio_maps that to reach its
command register, BARs, and — the point — its capability list (MSI/MSI-X, PCIe extended
caps), with no new syscall. Verified in the discovery test (QEMU q35's functions each
carry it).
MSI: msi_bind(device_id, endpoint) -> address (rax), data (rdx) allocates a per-device
edge-triggered vector, binds it to the endpoint, and returns the (address, data) the
driver programs into its own MSI capability. Unlike irq_bind there's no GSI, no I/O APIC
entry, no sharing, and no ack cycle — dispatch recognises an MSI vector (vector_gsi ==
none, msi_bound set), EOIs, and notifies. Owner-keyed release drops the binding on exit.
Legacy INTx (_PRT parsing + shared lines) is deliberately skipped; MSI is the real
answer.
QEMU's HPET has no MSI, so the new `msi` test proves the vector-routing path with a
self-IPI (new apic.selfIpi) standing in for the device's MSI write: bind a vector,
fire it, the bound endpoint is notified. The msi_bind syscall wraps irq.msiBind with
the claim check and lands its first real use with the first PCI driver. Suite 39/39
plus host tests.
An HCD programs a bus-master engine: it needs a descriptor ring that is physically
contiguous, at a physical address it knows, uncacheable, and pinned. mmap gives none
of those. Add dma_alloc(len, flags) -> vaddr (rax), paddr (rdx) and dma_free(vaddr,
len): grant contiguous, zeroed, pinned, strong-uncacheable memory in a per-process DMA
arena (PML4[228]) and hand back both addresses.
Pieces: pmm.allocContiguous(count, max_phys) finds a run of contiguous free frames
below a cap (dma_below_4g for 32-bit engines); mapUserDmaInto maps them uncacheable
(PCD|PWT) but WITHOUT device_grant, so unlike an MMIO grant these frames are real RAM
and freeSubtree returns them on teardown — a driver that dies leaks nothing. dma_free
is bounded to the DMA arena so it can never unmap the caller's stack/heap/MMIO.
dma_write_combining is accepted but falls back to coherent (WC needs PAT programming).
Runtime: runtime.dma.alloc/free (a two-return-value stub, like replyWait). New `dma`
kernel test drives the mechanism directly — contiguity, the below-4G cap, coherent
mapping, and reclaim-on-teardown (no leak). The thin syscall wrappers follow the tested
mmap/mmio_map shape and land their first real use with the first DMA driver. Suite
38/38 plus host tests.
The tree had zero memory barriers — correct-by-accident on x86 (TSO + strong-
uncacheable MMIO), but a landmine for the first DMA driver and for ARM, which is the
win condition. Add /lib/mmio: typed volatile register access (read/write) plus mb /
rmb / wmb, lowered per-architecture (mfence/lfence/sfence on x86_64, dsb sy/ld/st on
aarch64) so the ordering rules are a named primitive, not scattered `asm volatile`.
`volatile` is not a barrier — it says nothing about ordinary stores (a DMA descriptor
in WB RAM) relative to a volatile doorbell write; wmb() between them is the fix.
Prove it on the one existing caller: hpet now does its register access through
mmio.read/write. It needs no barriers itself (pure MMIO, no DMA, UC grant on x86) —
the point is the typed, arch-portable access every driver should use; the barriers are
there for the DMA drivers to come.
New `mmio` module injected into addUserBinary; host test asserts the barriers assemble
and a register round-trips. Suite 37/37 plus host tests.
ipc_call and ipc_reply_wait grow a `send_cap` argument (r9) and a `received_cap`
return (r8): an endpoint travels alongside a message, installed into the receiver's
handle table. The transfer is a share, not a move — the endpoint's refcount is bumped
and the sender keeps its handle. If the receiver's table is full the call fails
-ENOSPC and the message is NOT delivered (a half-delivered capability is worse than a
failed send); a bad handle fails -EBADF. Both directions carry a cap: a client's call
hands one to the server (seen in the server's replyWait), and the server's reply hands
one back (seen in the client's call return).
This is the "open" primitive the driver model was blocked on: a bus driver mints a
per-device endpoint and hands it to a class driver, giving it a private channel to one
device without the 8-slot global name registry.
Kernel: shareCapability in ipc-synchronous.zig at both copy points; new
setSystemCallResult3 (r8, saved/restored by the syscall stub); Task gains
ipc_send_cap / ipc_received_cap. Runtime: callCap + Reply, replyWait gains send_cap
and Received.cap; plain call/replyWait delegate with no_cap. New abi.no_cap.
New ipc-cap test (two kernel tasks exercise both directions, each verifying the
endpoint it received is the same object shared, refcount bumped to 2). No class driver
consumes callCap yet — it lands with the first one. Suite 37/37 plus host tests.
Add a `system_spawn(name)` system call: the kernel loads a binary bundled in the
initial-ramdisk, by name, as a fresh ring-3 process. It's the mechanism a user-space
supervisor needs — discovery and policy stay in user space, the kernel only spawns.
The kernel already holds the initial-ramdisk image from the boot handoff; it now
stashes it (process.setInitialRamdisk) so the handler can resolve names, bounds-checks
the name into the user half like debug_write, and returns -1 for an unknown name or a
load failure. Ungated for now (any process may spawn any bundled binary); a spawn
capability belongs here once the model grows one.
The device manager stops logging "would spawn it" and calls runtime.system.spawn on
its matched driver. On QEMU it discovers the HPET, matches `hpet`, and spawns it — and
the driver comes all the way up (claims the timer, maps its MMIO, binds and services
its IRQ, prints "hpet: ok"). The device-manager test now keys on that final marker:
since only the manager is spawned, `hpet: ok` appearing proves the whole
discover -> match -> system_spawn -> driver-up chain end to end.
Transitional: the kernel still auto-spawns the whole initial-ramdisk at boot, so a
real boot briefly double-spawns hpet (the second claim fails harmlessly). Increment 3
removes that redundancy so the manager is the sole owner of driver spawning. Suite
36/36 plus host tests.
The `system` module (formerly `danos`) had become a grab-bag: it held the
loader<->kernel handoff *and* the kernel<->user ABI *and* the device wire types, in
one module three different audiences imported. Usage proved the seam — the
bootloader never touched the syscall/device ABI, and user space never touched the
boot handoff — so split it by audience, one module per contract:
system/boot-handoff.zig loader <-> kernel: BootInformation, Framebuffer,
MemoryMap, the VM layout + physicalToVirtual, kernel_abi
system/abi.zig kernel <-> user, core: SystemCall, mmap prot flags,
page_size, notify_badge_bit, ServiceId
system/devices/device-abi.zig kernel <-> user, devices: DeviceDescriptor,
DeviceClass, ResourceDescriptor, ResourceKind, ...
device-abi is the devices sub-project's public interface, exposed as its own module
the way vfs exposes vfs-protocol — importable by user space, unlike the
kernel-internal device model it also feeds. That collapses a real duplication:
DeviceClass and ResourceKind were defined twice (device-model.zig and the contract,
kept "in sync by hand"); device-model now re-exports them from device-abi, so the
enum a driver matches on and the one the kernel classifies with are one type.
Each import now declares which contract it speaks: the bootloader imports only
boot-handoff; a driver only abi + device-abi (via the runtime); the kernel all
three. This also retires the `system` / `runtime.system` name overlap. page_size
lands in abi (it's part of the mmap contract user space aligns to); the bootloader
keeps its own local 4 KiB constant so it depends on nothing but the handoff.
All 21 importers rewired, docs updated to keep /system mapping to source. Build,
host tests, and the QEMU suite (36/36) all green.
The shared kernel<->user ABI contract (BootInformation, the SystemCall numbers,
DeviceDescriptor, page_size, ...) is now the `system` module at
system/system.zig, following the convention that a directory's root file takes
the directory's name.
One overlap to note: the runtime's syscall wrappers are already `runtime.system`,
so the single file that uses both the contract and those wrappers
(library/runtime/heap.zig) aliases the wrappers locally as `system_calls`. The
two are distinct (top-level `system` vs `runtime.system`); everywhere else the
contract is just `system`.
Also: the QEMU run's serial capture now lands in the FHS log location,
zig-out/var/log/system/serial0-<timestamp>.log — a stand-in for the kernel's own
logging system, which will eventually write there itself.
Suite 35/35 plus host tests green.
Follow-up to the monorepo re-org. Suite 35/35 plus host tests green.
POSIX compatibility is now its own library, library/posix/ (unistd, stdio),
layered strictly over the runtime — it calls the runtime's IPC/heap, never
system calls directly. The runtime is now POSIX-free (the danos-native
application ABI). The VFS wire protocol is danos-native throughout
(Stat -> FileStatus, .stat -> .status, O_CREAT -> create); the POSIX layer
maps the POSIX spellings at the boundary. The coding standard's ABI-name
exception is scoped to one place: a file is allowed POSIX spellings only if it
lives under library/posix/ — everywhere else, danos naming with no exception.
Naming fixes, all mechanical:
- initrd -> initial-ramdisk: the source file, the module, the tool
(make-initial-ramdisk.py), the artifact (initial-ramdisk.img, including the
bootloader's load path), and the identifiers.
- system/kernel/device-service.zig -> devices-broker.zig: it is ring-0 kernel
code (the trusted device table + claim capability), not a ring-3 service. The
future user-space device *manager* (policy) will live in system/services/.
- Dropped the daemon `d` suffix: hpetd -> hpet, busd -> bus. A driver lives in
system/drivers/, so the folder already says what it is; encoding the role in
the name too is redundant. The coding standard drops that exception.
- system/devices/aml/interp.zig -> interpreter.zig (the type was already
Interpreter).
The source layout now mirrors the runtime filesystem hierarchy
(docs/danos-file-system-hierarchy-FSH.md): what lives under system/ in the
source is what a running danos represents under /system. Each service and
driver is a sub-project directory that is its own Zig module — cross-project
references go by module name, never by a path into another project's files.
Moves (all git mv, history preserved):
- src/ -> system/ (danos internals; the self-representation)
root.zig -> danos.zig (the kernel<->user contract module)
kernel/arch/ -> kernel/architecture/ (arch -> architecture)
device/ -> devices/ (what /system/devices reflects)
boot/ -> /boot (the loaders, top level)
- sbin/ -> split by role:
init, vfs -> system/services/<name>/<name>.zig
hpetd, busd -> system/drivers/<name>/<name>.zig
vfs-test -> system/services/vfs/vfs-test.zig (inside the vfs project)
- lib/ -> library/runtime/ (room for other libraries beside runtime)
The VFS wire protocol becomes its own module, system/services/vfs/protocol.zig
("vfs-protocol"): the vfs sub-project exposes its interface, and the runtime's
file layer imports it by name. First instance of the "protocol module" pattern
(docs/driver-model.md); usb/block will expose theirs the same way.
Also: fix a naming-standard violation in the protocol — Op -> Operation (and
req -> request, _pad -> _padding). Docs updated: /system/services added to the
FHS doc, a repository-layout section added to the docs index, and stale source
paths swept across comments and docs.
Runtime boot paths are unchanged (the bootloader still loads /sbin/init);
aligning the runtime filesystem to the FHS is a separate follow-up. Suite 35/35
plus host tests green.
Daniel Samson
Daniel Samson