The `system` module (formerly `danos`) had become a grab-bag: it held the
loader<->kernel handoff *and* the kernel<->user ABI *and* the device wire types, in
one module three different audiences imported. Usage proved the seam — the
bootloader never touched the syscall/device ABI, and user space never touched the
boot handoff — so split it by audience, one module per contract:
system/boot-handoff.zig loader <-> kernel: BootInformation, Framebuffer,
MemoryMap, the VM layout + physicalToVirtual, kernel_abi
system/abi.zig kernel <-> user, core: SystemCall, mmap prot flags,
page_size, notify_badge_bit, ServiceId
system/devices/device-abi.zig kernel <-> user, devices: DeviceDescriptor,
DeviceClass, ResourceDescriptor, ResourceKind, ...
device-abi is the devices sub-project's public interface, exposed as its own module
the way vfs exposes vfs-protocol — importable by user space, unlike the
kernel-internal device model it also feeds. That collapses a real duplication:
DeviceClass and ResourceKind were defined twice (device-model.zig and the contract,
kept "in sync by hand"); device-model now re-exports them from device-abi, so the
enum a driver matches on and the one the kernel classifies with are one type.
Each import now declares which contract it speaks: the bootloader imports only
boot-handoff; a driver only abi + device-abi (via the runtime); the kernel all
three. This also retires the `system` / `runtime.system` name overlap. page_size
lands in abi (it's part of the mmap contract user space aligns to); the bootloader
keeps its own local 4 KiB constant so it depends on nothing but the handoff.
All 21 importers rewired, docs updated to keep /system mapping to source. Build,
host tests, and the QEMU suite (36/36) all green.
The device manager is the ring-3 process that turns the device tree into a running
system — the udev-analog. It is mechanism-vs-policy done right: the kernel
enumerates the hardware and enforces the claim capability; this decides which
driver serves which device, using no special privilege (the same device_enumerate
any process could call).
This first increment does the discovery + matching half: system/services/
device-manager enumerates /system/devices, matches each device to a driver by
class (a small static policy table), and logs the decision — finding the HPET
(a timer) and deciding `hpet` serves it. It does not spawn yet: spawning needs a
`system_spawn` system call (the kernel spawns every initial-ramdisk binary in a
loop today), which is the next increment. New `device-manager` test; suite 36/36
plus host tests.
The shared kernel<->user ABI contract (BootInformation, the SystemCall numbers,
DeviceDescriptor, page_size, ...) is now the `system` module at
system/system.zig, following the convention that a directory's root file takes
the directory's name.
One overlap to note: the runtime's syscall wrappers are already `runtime.system`,
so the single file that uses both the contract and those wrappers
(library/runtime/heap.zig) aliases the wrappers locally as `system_calls`. The
two are distinct (top-level `system` vs `runtime.system`); everywhere else the
contract is just `system`.
Also: the QEMU run's serial capture now lands in the FHS log location,
zig-out/var/log/system/serial0-<timestamp>.log — a stand-in for the kernel's own
logging system, which will eventually write there itself.
Suite 35/35 plus host tests green.
`zig build` now installs into a FHS-shaped zig-out that *is* the danos filesystem
and the boot volume — no more zig-out/bin or a separate esp/:
zig-out/EFI/BOOT/BOOTX64.efi (firmware entry; UEFI fixes this path)
zig-out/boot/initial-ramdisk.img
zig-out/system/kernel (the kernel binary)
zig-out/system/services/init vfs
zig-out/system/drivers/hpet bus
Binaries land at their addressed, leaf-collapsed paths per the sub-project
resolution rule (system/services/init/init.zig -> system/services/init); vfs, hpet,
and bus are installed to their FHS homes too, so the image is complete even though
at boot they arrive inside the initial-ramdisk.
The bootloader (boot/efi.zig) now loads each artifact from its FHS path
(system\kernel, system\services\init, boot\initial-ramdisk.img); run-x86-64 mounts
zig-out directly; the QEMU test harness assembles its ESP from the FHS zig-out.
Also renames system/kernel/main.zig -> kernel.zig so the kernel follows the
name/name.zig convention (kernel/ = ring-0 code, services/ = ring-3 OS services).
Documents the resolution rule in the repository-layout section (README + coding
standard). Suite 35/35 plus host tests green.
Follow-up to the monorepo re-org. Suite 35/35 plus host tests green.
POSIX compatibility is now its own library, library/posix/ (unistd, stdio),
layered strictly over the runtime — it calls the runtime's IPC/heap, never
system calls directly. The runtime is now POSIX-free (the danos-native
application ABI). The VFS wire protocol is danos-native throughout
(Stat -> FileStatus, .stat -> .status, O_CREAT -> create); the POSIX layer
maps the POSIX spellings at the boundary. The coding standard's ABI-name
exception is scoped to one place: a file is allowed POSIX spellings only if it
lives under library/posix/ — everywhere else, danos naming with no exception.
Naming fixes, all mechanical:
- initrd -> initial-ramdisk: the source file, the module, the tool
(make-initial-ramdisk.py), the artifact (initial-ramdisk.img, including the
bootloader's load path), and the identifiers.
- system/kernel/device-service.zig -> devices-broker.zig: it is ring-0 kernel
code (the trusted device table + claim capability), not a ring-3 service. The
future user-space device *manager* (policy) will live in system/services/.
- Dropped the daemon `d` suffix: hpetd -> hpet, busd -> bus. A driver lives in
system/drivers/, so the folder already says what it is; encoding the role in
the name too is redundant. The coding standard drops that exception.
- system/devices/aml/interp.zig -> interpreter.zig (the type was already
Interpreter).
The source layout now mirrors the runtime filesystem hierarchy
(docs/danos-file-system-hierarchy-FSH.md): what lives under system/ in the
source is what a running danos represents under /system. Each service and
driver is a sub-project directory that is its own Zig module — cross-project
references go by module name, never by a path into another project's files.
Moves (all git mv, history preserved):
- src/ -> system/ (danos internals; the self-representation)
root.zig -> danos.zig (the kernel<->user contract module)
kernel/arch/ -> kernel/architecture/ (arch -> architecture)
device/ -> devices/ (what /system/devices reflects)
boot/ -> /boot (the loaders, top level)
- sbin/ -> split by role:
init, vfs -> system/services/<name>/<name>.zig
hpetd, busd -> system/drivers/<name>/<name>.zig
vfs-test -> system/services/vfs/vfs-test.zig (inside the vfs project)
- lib/ -> library/runtime/ (room for other libraries beside runtime)
The VFS wire protocol becomes its own module, system/services/vfs/protocol.zig
("vfs-protocol"): the vfs sub-project exposes its interface, and the runtime's
file layer imports it by name. First instance of the "protocol module" pattern
(docs/driver-model.md); usb/block will expose theirs the same way.
Also: fix a naming-standard violation in the protocol — Op -> Operation (and
req -> request, _pad -> _padding). Docs updated: /system/services added to the
FHS doc, a repository-layout section added to the docs index, and stale source
paths swept across comments and docs.
Runtime boot paths are unchanged (the bootloader still loads /sbin/init);
aligning the runtime filesystem to the FHS is a separate follow-up. Suite 35/35
plus host tests green.
Two driver-model milestones plus a tree-wide naming pass. Suite 35/35
(QEMU) + host tests green.
M11 — IRQ-as-IPC. A ring-3 driver now sleeps until its device interrupts
it. New src/kernel/irq.zig: per-GSI endpoint bindings, comptime per-vector
trampolines, dispatch = mask GSI -> LAPIC EOI -> notifyLocked, all under one
lock region. irq_bind/irq_ack syscalls, gated by the device claim like
mmio_map. interruptDispatch no longer EOIs — each handler owns its EOI,
because a level line must be masked before it is acknowledged (irq_ack is
the unmask). Bindings are keyed on the owning task and released on exit
(a shared endpoint's siblings survive). hpetd rewritten interrupt-driven.
Tests: hpet (rewritten, reads back the I/O APIC routing) and irqfree.
M12 — bus drivers. DeviceDesc gains a parent, making the device table a
tree. dev_register (device_register) lets a process publish children below
a device it claimed; the kernel enforces resource containment (a child's
resources must nest in its parent's), so a descriptor can't fabricate a
window over kernel RAM. Descriptor copied in via copyFromUser (physmap
walk — an unmapped user pointer fails the call instead of faulting the
kernel). Per-parent child cap bounds table exhaustion. sbin/busd.zig is a
worked bus driver. Test: bus.
Naming — per docs/coding-standards.md: non-acronym abbreviations spelled
out (message, descriptor, device_service, scheduler, runtime, physical,
interpreter, ...); acronyms kept (IPC, MMIO, DMA, HCD, ...); files are
kebab-case (ipc-synchronous.zig, device-service.zig, vfs-protocol.zig, ...).
Exceptions: POSIX/C ABI names and Zig idioms (init/len/ptr) kept. Module
collisions resolved by specific naming (config -> parameters, device.zig
alias -> device_model). AML op/Op disambiguated: op = opcode, Op =
operation; per-opcode parse handlers renamed opX -> parseX.
New driver docs: drivers.md, driver-model.md (bus/class/HCD shapes + the
proposed M13–M16 ABI), coding-standards.md.
Start the user-space driver track (VFS + IPC + heap). This lays the
process/syscall foundation the runtime heap will grow on.
- Rename usermode.zig -> process.zig; drop the retired hello/ping blob and
its `user` test (subsumed by the real /sbin/init exerciser). Keep the
isolation-proof pf blob and the user-pf test.
- Add danos.Syscall as the single source of truth for syscall numbers, shared
by the kernel dispatcher and (later) the user runtime lib. Dispatch on the
enum. New calls: 1=yield, 4=mmap, 5=munmap. Widen debug_write's bounds
check to the whole user low half so heap buffers are writable.
- mmap grants zeroed RW+NX pages from a per-process bump arena
(Task.heap_next, PML4[224] above image+stack); munmap frees the frames.
Add paging.translateIn / unmapInto (+ arch.translate / unmapUserPageInto)
as the primitives munmap and future cross-AS copies need.
- New `usermem` test: grant three pages into a fresh AS, translate them,
release via the munmap path, tear down, and assert no frames leak.
Suite 28/28 (user -> usermem).
init is now a real scheduled ring-3 process: it prints a heartbeat and
sleeps, forever. The kernel spawns it at boot via spawnProcess (its own
address space, preemption on) and the boot context drops to idle — the
system's steady state is "kernel idle, init alive in ring 3", beating
~1 Hz. Adds the sleep(ms) syscall. The init/process tests are reshaped
around the heartbeat (repeated syscalls, still-alive, two concurrent
processes on distinct address spaces). Pins init to LLD and folds the
.large code model's .ltext/.lrodata/.ldata into the linker script so its
code segment is R+X. Removes the now-dead borrowed-thread ELF loader
(runInitElf); spawnProcess is the one path. Docs updated. Suite 28/28.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-process address spaces (AddressSpace = a PML4 with an empty user
half and the shared kernel half copied in; create/destroy in paging.zig)
with CR3 switched on context switch and TSS.rsp0/kernel_rsp published per
switch. The GS base now points at an arch per-CPU block and every ring
transition observes the swapgs discipline, so a ring-3 `mov %ax,%gs` can
no longer poison per-CPU access. syscall/sysret is the primary user entry
(int 0x80 kept as a test path); one handler, installed once at boot,
serves both and dispatches on whether the caller is a scheduled process
or a borrowed test thread. spawnProcess loads an ELF into a fresh address
space and schedules it; exit frees the address space after switching to
the kernel tables. New `process` test: init runs twice as a real process
(create/exit/recreate) on its own page tables, coexisting with a kernel
task under preemption. Suite 28/28.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
paging.init now maps only the physmap, the framebuffer/LAPIC windows,
and the kernel's own segments; the entire low canonical half is left to
user space. Every higher-half PML4 entry is pre-created so a per-process
address space can share the kernel half by copying PML4[256..512), with
an assert against late top-half entries and a 4 GiB guard on
pre-switch table frames. The AP trampoline's low identity page is now
created transiently by arm() and unmapped by disarm(); startAp asserts
the page-table root is 32-bit addressable. Docs (paging.md) updated.
Suite 27/27; 4-core normal boot reaches /sbin/init.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ring 3 works: user GDT descriptors (sysret-ready layout), TSS.rsp0,
U/S-bit user mappings (W^X preserved), an int 0x80 syscall gate with a
mutable trap frame, and a setjmp-style enter/exit path. /sbin/init is a
real freestanding Zig binary built from sbin/, shipped on the ESP,
loaded by the bootloader (BootInfo.init_base/len), validated and mapped
by an in-kernel user-ELF loader, and run at CPL 3 — syscalls: exit,
ping, write. Tests: user, user-pf (U/S isolation proof, error code
0x5), init. Suite 27/27.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hoist max_cpus into danos (root.zig), raise it to 128, and stop reserving a static [max_cpus][16 KiB] IST array. Only the BSP's IST stack is static (needed before the allocator); each AP's is heap-allocated when it comes online. Shrinks the kernel from ~1.1 MB to ~139 KiB in ReleaseSmall.
fault-ap-df pins a #DF to an AP (its own IST must catch it); affinity checks a pinned task never migrates. onException now names the core, so an AP fault is attributed and shown contained. Both teeth-checked.
spawnOn(entry, priority, cpu) routes to a per-core pinned queue, merged with the global queue at O(1) selection. Falls back to unpinned for an offline/invalid core.
Arm the low frame (copy blob, make executable) only while a core climbs, then zero it and restore RW+NX; the frame stays reserved so cores can be re-woken. startSecondary is one re-runnable attempt; boot retries a non-responding core 3x. Validated the retry path by forcing a first-attempt failure.
Daniel Samson
Daniel Samson