danos/docs

danos documentation

Notes on how danos boots and draws, written to explain the why behind the code rather than restate it. Roughly in the order things happen at runtime:

1. efi.md — EFI / the boot process. How UEFI firmware finds and runs the bootloader, what the loader gathers before ExitBootServices, how it loads the kernel ELF, and the ABI contract for the jump into the kernel. Start here.
2. system-image.md — system.img, the boot capsule. The bundled user binaries packed into one file in the initial-ramdisk wire format, because one open + one sequential read is the only file I/O shape firmware is fast at. The trivial container format, the three artifacts one build list derives (tree, manifest, capsule), the loader's three-strategy fallback chain, and the capsule's kernel-side life as both the spawn table and the read-only /system mount.
3. gop.md — the Graphics Output Protocol. How UEFI exposes graphics modes (unlike fixed VGA modes), how we detect the monitor's native resolution from EDID and switch to it, and the pixel formats we accept or reject.
4. framebuffer.md — the framebuffer. What the linear framebuffer the loader hands over actually is, and what pitch (stride) means versus width — the detail you have to get right to avoid a skewed image.
5. memory-map.md — the memory map. How the loader learns what physical RAM exists and hands it to the kernel in danos's own neutral format, rather than leaking UEFI's memory descriptors across the boundary.
6. frame-allocator.md — the physical frame allocator. The bitmap allocator that hands out and reclaims 4 KiB physical frames from that map — the primitive page tables and the heap are built on.
7. interrupts.md — interrupts and exceptions. The GDT, IDT and TSS, the exception stubs, and the handler that reports a CPU fault in red instead of letting it triple-fault into a silent reset.
8. paging.md — the kernel's page tables. Building our own 4-level page tables, identity-mapping the low 4 GiB, and switching CR3 off the firmware's tables onto ours.
9. device-interrupts.md — device interrupts. The Local APIC and its timer — the kernel's first interrupt that is handled and returned from, giving it a heartbeat.
10. heap.md — the kernel heap. A growable free-list allocator built on the VMM, exposed as a std.mem.Allocator so std containers work — dynamic allocation for the kernel.
11. scheduling.md — the scheduler. Fixed-priority preemptive multitasking: kernel threads, the context switch, O(1) priority selection, and blocking (sleep, wait queues) — the leap to a running system.
12. ipc.md — inter-process communication. Bounded blocking message-passing channels, then synchronous call/reply between processes over endpoints — the backbone the microkernel's isolated servers talk over.
13. syscall.md — system calls. How ring 3 asks the kernel for something: the syscall/sysret fast path, the trap frame, and why the table is deliberately tiny. The numbers are a private ABI — vdso.md designs the public boundary that will hide them.
14. vfs-protocol.md — the VFS wire protocol. The language-neutral byte-level spec of the file protocol spoken over IPC: request/reply headers, the operation table, mount routing, and the append-only evolution rules — the first IPC protocol documented as public ABI.
15. drivers.md — writing a driver. The payoff: a driver is an ordinary ring-3 process that claims a device, maps its registers, and sleeps until its hardware interrupts it. The claim is the capability; irq_ack is the unmask.
16. driver-model.md — buses, classes and host controllers. How real driver stacks factor into three shapes and how families share code. The three primitives it proposed are long since built (M13 capability passing, M14 DMA + barriers, M15 MSI), and the driver contract on top of them — hello, supervision, restart — is built too (device-manager.md, M18).
17. usb-hub.md — USB hubs. Built (M22): why hub topology is handled inside the usb-xhci-bus driver rather than a separate hub class driver — a device behind a hub is reached by the controller, programmed with a route string in its slot context — plus the compound-hub reality (a USB 3.0 hub is physically two hubs) and detection via the hub's status-change interrupt endpoint.
18. process-management.md — process management. The microkernel's ps/kill/SIGCHLD: enumerate as a table snapshot, the supervision link as the kill authority, and child-exit notifications over the same endpoints IRQs arrive on.
19. process-lifecycle.md — the process lifecycle. Built (M17): signals over IPC as the one lifecycle vocabulary every process speaks — the POSIX.1-1990 words with message delivery instead of stack hijack, the stable process module interface, exit reasons, published exit events any stateful service can subscribe to (the VFS releasing dead clients' handles), and the two iron rules (cleanup is the kernel's job; kill is not a signal).
20. device-manager.md — the device manager. Built (M18, through the app surface): the tree, the matcher, and the supervisor. Tree structure lives in the manager, authority stays in the kernel; bus drivers report what they see; drivers are restarted through the lifecycle vocabulary — the plan that turns resilience.md's restart goal into increments.
21. input.md — the input module. Broadcasting input events (keyboard, mouse, joystick): why a synchronous rendezvous can't fan out to many listeners, the asynchronous ipc_send primitive built to fix it, and the per-device subscribe/publish service layered on top.
22. display.md — the display service. The display half of the GUI track: a user-space compositor that owns the framebuffer, composes a layer stack into a double buffer, and presents it. Why GOP and the PCI display device are two views of one controller, the device-node + write-combining handoff, and what flicker-free buys that tear-free doesn't. Plan: display-plan.md. v2 (complete) makes scanout a pluggable backend — GOP floor + a native virtio-gpu driver, hot-attached, with runtime mode-set, EDID, fenced vsync presents, and restart re-attach: display-v2.md, plan display-v2-plan.md. Looking further out, three research snapshots survey what a native driver for real GPU silicon would take as another .scanout backend: nvidia-gpus.md (RTX 3060 / Ampere), amd-gpus.md (RX 6600 / RDNA2), and intel-igpu.md (Intel iGPU).
23. halting.md — halting. Why a kernel can't just "exit", and how while (true) hlt parks the CPU safely once there's nothing left to do.

Start with the north star:

• vision.md — the vision. danos is a learning-by-doing microkernel: minimal kernel, drivers/services isolated in user space, chosen for resilience (restartable components). Win condition: runs on the author's PC and both Raspberry Pis, ideally with a GUI. Real-time is an option to explore, not a requirement. The why that shapes everything below.
• resilience.md — resilience. A design note (not built yet) on fault isolation + live restart — the reincarnation-server + capability model that makes "if I break it, I can restart it" real. danos's core motivation.
• zig-self-hosting.md — running Zig on danos. A design note (not built yet) on making danos a real Zig target (-target x86_64-danos) and eventually running the compiler on it. The key realisation: Zig 0.16 reduces an OS port to one seam (std.os.danos), so we build an os seam module (→ that seam) plus the thin file-system module, retire the posix shim, and follow a phased path to zig build-exe hello.zig running on danos — not Linux-ABI emulation.
• threading.md — threads, the std-shaped way. Built (M1–M6): the thread module's Thread mirrors std.Thread's API (spawn/join/detach, Mutex/Condition/ Semaphore) over a private thread ABI — several tasks sharing one address space via a thread_spawn syscall, futex-backed blocking, address-space refcounting. Why it's the native type and not literal std.Thread (the private ABI), and why threads stay a narrow opt-in against the resilience default. Build plan + gates: threading-plan.md.
• vdso.md — the vDSO, the public system-call boundary. A design note (not built yet) on keeping abi.zig genuinely private: a kernel-supplied, C-ABI entry blob mapped into every process as the only way into the kernel — so the syscall numbers can be renumbered or randomised at will, and Rust/C binaries get a stable boundary without danos growing a dynamic linker. danos's public ABI = the vDSO + the documented IPC wire protocols (vfs-protocol.md first).

Cutting across all of these:

• system-requirements.md — system requirements. The hardware needed to run danos: minimum specs (UEFI x86-64, ACPI, PCIe ECAM, xHCI, ~128 MiB RAM) grounded in what the boot path actually assumes, plus a plain-language guide matching Intel/AMD CPU generations by name.
• release-iso.md — the release ISO. The flashable boot media: zig build release-x86-64 wraps the FAT32 boot volume in a hybrid ISO (MBR ESP partition + El Torito EFI entry, one embedded image) that Etcher/dd flash to USB or a burner writes to disc — built by an in-repo pure-Python tool, like the FAT image itself.
• architecture.md — the architecture split. How CPU-specific code is kept behind a build-time arch module so the generic kernel never names x86_64, leaving room for other systems (e.g. an AArch64 Raspberry Pi) later.
• arm.md — ARM targets. The Raspberry Pi landscape the arch split is aiming at: arm (32-bit, Pi Zero W) vs aarch64 (64-bit, Pi 3-5), UEFI vs device-tree boot, and what each layer needs.
• discovery.md — device discovery. A design note on learning what hardware exists via ACPI (x86) or device tree (ARM) behind one neutral device model — when to build it, and how to keep it architecture-agnostic.
• acpi.md — finding the ACPI tables. The concrete x86 locator chain: how the loader captures the RSDP, hands its physical address across in BootInformation, and how the platform derives the RSDT/XSDT from it and walks the SDTs — plus the live event side (the SCI, the power button, GPE/Notify) the ring-3 acpi service runs.
• power.md — the power service. System power as a domain-named service: button/lid/battery events published to subscribers, and init's orderly shutdown composing the lifecycle stop sequence with an ACPI S5 write. Firmware-neutral — a PSCI backend drops in on ARM.
• timers.md — timers and time. The ring-3 surface for reading the clock and waiting: why now() is a syscall rather than a service, and the one-shot timer notification (timer_bind) that gives supervisors a timed wait — built on the LAPIC heartbeat and calibrated TSC of device-interrupts.md.
• smp.md — multiple cores. A design/research note on how microkernels (L4, seL4) handle SMP — big kernel lock vs per-CPU vs multikernel — and how the right choice depends on whether danos is chasing real-time or resilience.
• coding-standards.md — coding standards. The naming rule the tree follows: non-acronyms are spelled out in full (message, not msg), files are kebab-case, code follows Zig's case conventions, and the handful of exceptions (POSIX/C ABI names, init/len/ptr, acronyms).
• sysv.md — the calling convention. What "the kernel is SysV" means, and why the loader→kernel boundary has to pin it (the RDI-vs-RCX handoff).
• testing.md — testing. How the kernel is tested by booting it in QEMU and asserting on its serial output — reproducibly, and structured so the same tests run across architectures.
• logging.md — logging. The multi-sink diagnostic log (serial, 0xE9 debugcon, file later) kept separate from the framebuffer display, plus the robustness path: optional framebuffer, POST-code checkpoints, and a persistent panic breadcrumb so the kernel survives — and can be diagnosed — with no output.

How the pieces relate

The boot flow ties them together: UEFI runs the loader (efi.md), which queries the GOP to pick a graphics mode (gop.md), hands the kernel a framebuffer to draw into (framebuffer.md) and a memory map of physical RAM (memory-map.md); the kernel turns that map into a frame allocator (frame-allocator.md), installs its descriptor tables so CPU faults are caught (interrupts.md), builds its own page tables and switches onto them (paging.md), brings up the heap for dynamic allocation (heap.md), starts the scheduler (scheduling.md) and the timer that preempts it (device-interrupts.md) — with tasks blocking, sleeping and passing messages over IPC channels — runs, its CPU-specific bits behind the architecture boundary, and when idle, or on a panic, it halts (halting.md).

Above that line the microkernel proper begins: discovery (discovery.md, acpi.md) learns what hardware exists, ring-3 processes ask the kernel for things through the small syscall table, isolated servers reach each other over IPC endpoints (ipc.md), and a driver claims a device, maps its registers, and sleeps until the hardware interrupts it — which is the whole reason for the arrangement (vision.md).

Repository layout

danos is a monorepo of sub-projects. Each service or driver is a directory that is its own Zig module — it can hold as many files as it needs, and other sub-projects reach it by module name, never by a path into its files. The source tree deliberately mirrors the runtime file-system hierarchy (file-system-hierarchy.md): what you see under system/ in the source is what a running danos represents under /system.

A sub-project is addressed by its directory; its entry point repeats the directory's name. system/services/init/ contains init.zig (its root), and produces a binary addressed as system/services/init — the repeated leaf resolves away:

Source (root file)	Addressed as (module / binary / hierarchy path)
system/services/init/init.zig	system/services/init → /system/services/init
system/drivers/ps2-bus/ps2-bus.zig	system/drivers/ps2-bus → /system/drivers/ps2-bus
test/system/services/vfs-test/vfs-test.zig	test/system/services/vfs-test → /test/system/services/vfs-test
library/device/pci/pci.zig	library/device/pci (the pci module)

In source, a sub-project is a directory so it can hold many files — the entry is fat/fat.zig, beside it fat/engine.zig, fat/on-disk.zig, and so on. When addressed or installed, that collapses to the single canonical path: the init binary installs to /system/services/init (a file at that path), not /system/services/init/init. The repeated leaf exists only in source; the directory is the identity, the entry file is its implementation. (Same idea as a Go package being its directory, or a macOS .app bundle addressed by the bundle, not the executable within.) A sub-project's extra files are reached through the module, never as separate paths.

system/          → /system   danos's own internals (the self-representation)
  boot-handoff.zig             the loader↔kernel contract (the `boot-handoff` module)
  abi.zig                      the private kernel↔userspace syscall ABI (the `abi` module)
  parameters.zig  initial-ramdisk.zig   shared contracts
  kernel/                     IPC, memory, scheduling, the VFS root, the private syscall dispatch
    architecture/x86_64/      the `architecture` module (never named by generic code)
    devices-broker.zig        the syscall-facing device table
    platform.zig acpi.zig fdt.zig device-model.zig   firmware discovery + the kernel's
                              device model — the implementation of what /system/devices reflects
  drivers/     pci-bus/ ps2-bus/ usb-xhci-bus/   one sub-project per driver → /system/drivers
  services/    init/ fat/ device-manager/   system servers → /system/services (fat/ holds
                              fat.zig, engine.zig, on-disk.zig)
library/         → /lib       libraries, one sub-directory each
  kernel/                     the danos-native system library (kernel32-style): the syscall
                              surface split by concern — ipc, memory (heap/dma/shared-memory),
                              process, time, logging, file-system, thread, service, plus the
                              system-call stubs and the start/root entry shim
  device/                     device code by domain — mmio/ model/ pci/ usb/ acpi/ driver/
                              block/ — each a shareable data module (device-abi, pci-class,
                              usb-abi/ids, acpi-ids) plus a logic module (mmio, pci, usb, aml,
                              driver — the device-access + device-manager-hello client)
  client/                     userspace service clients (display, input) — a program's view of
                              a service, layered over that service's protocol
  protocol/                   driver↔service wire contracts (vfs block display scanout input
                              power device-manager usb-transfer), one module per directory
boot/            → /boot      the loaders
test/            → /test      the test tree: the QEMU harness (qemu_test.py, host-side)
  system/services/            beside the on-image test fixtures — vfs-test/ thread-test/
                              crash-test/ … — whose repo path IS their boot-volume path
                              (/test/system/services/<name>)
build-support/                the danos build API (build-time only, nothing on the image):
                              the shared user-binary recipe + default-import wiring every
                              build file consumes (docs/build-packages-plan.md)
build/                        root-build helpers: image assembly (images.zig) + the QEMU
                              run steps (qemu.zig)
tools/                        host-side build scripts

Builds are packages (docs/build-packages-plan.md): each library/ domain owns a build.zig/build.zig.zon exporting its modules (with a standalone zig build test), every binary directory is a ~15-line package build, and the root build.zig orchestrates — the kernel + loader, what ships, and the aggregate test step — with image assembly in build/images.zig and the QEMU run steps in build/qemu.zig.

Wire protocols live in library/protocol/, one module per directory (library/protocol/vfs/vfs-protocol.zig is the vfs-protocol module), imported by module name. A protocol is the seam between a low-level driver and the higher-level service it serves — block ↔ the filesystem, a scanout driver ↔ the compositor — so both sides depend on the contract, not on each other, and the contract belongs to neither sub-project. A client module may wrap one for application convenience (the file-system module over vfs-protocol, and the block, display, input clients over theirs), but the protocol module is the boundary — a client re-exports no protocol, it imports it by name. A driver's private wire to its hardware (virtio-gpu's command set) is not a service seam and stays a driver-private file, beside the transport that reaches the same device.

Device code lives in library/device/<domain>/, grouped by what it is about (pci, usb, acpi, and the cross-cutting device model) and split by dependency weight: a data module of enums and wire types that is std-only and cheap for anyone to import, and a logic module that needs mmio or IPC. This is what keeps the microkernel out of device business — it imports exactly one library/ module, device-abi (the descriptor types its broker marshals across the syscall boundary), and nothing with logic or a taxonomy in it. That lone pure-data import is the only edge from system/kernel/ into library/.

There is no POSIX/C compatibility layer today: danos programs do file I/O through the danos-native file-system module (open/read/write/list over the VFS). A hand-rolled POSIX shim (library/posix/) was retired as premature — the real POSIX/C surface will come later from the std.os.danos seam (and, eventually, musl) when danos becomes a Zig target (see zig-self-hosting.md). When it does, the foreign-ABI naming exception in coding-standards.md applies to that seam.

Source map

Area	Code
Boot methods (one per way of booting the kernel)	boot/ — efi.zig (UEFI) → BOOTX64.efi
Kernel entry, panic, bring-up	system/kernel/kernel.zig
Loader↔kernel handoff (BootInformation, Framebuffer, MemoryMap, VM layout)	system/boot-handoff.zig
Private kernel↔userspace syscall ABI (SystemCall, mmap prot flags, page_size) — the system library speaks it, not apps	system/abi.zig
Device wire types (DeviceDescriptor, DeviceClass, …)	library/device/model/device-abi.zig
Physical frame allocator	system/kernel/pmm.zig
Kernel heap (std.mem.Allocator)	system/kernel/heap.zig
Scheduler (fixed-priority preemptive; blocking, wait queues)	system/kernel/scheduler.zig
Big kernel lock + interrupt-safe critical sections	system/kernel/sync.zig
IPC channels between kernel threads (message passing)	system/kernel/ipc.zig
IPC endpoints: cross-address-space call/reply, handles, notifications	system/kernel/ipc-synchronous.zig
User processes: ELF loading, address spaces, the syscall table	system/kernel/process.zig
VFS root: mount table + kernel-served nodes (fs_resolve/fs_node); wire protocol in library/protocol/vfs/vfs-protocol.zig	system/kernel/vfs.zig
Device tree + claim capability + device_register containment	system/kernel/devices-broker.zig
IRQ-as-IPC: routing a device interrupt to a driver's endpoint	system/kernel/irq.zig
Hardware discovery (ACPI/device tree) behind one neutral device model	system/kernel/platform.zig
Framebuffer text console (mirrors to serial)	system/kernel/console.zig
In-kernel test cases	system/kernel/tests.zig
Arch-specific kernel code (halt, GDT/IDT/TSS, exception + interrupt stubs, page tables, APIC/IO-APIC/timer, serial, linker script)	system/kernel/architecture/x86_64/
danos-native system library (kernel32-style): the syscall surface by concern — ipc, memory, process, time, logging, file-system, thread, service — the stable application ABI	library/kernel/
Service clients (a program's view of a service) and device clients	library/client/ (display, input), library/device/driver
System services (init, the fat filesystem, the device-manager)	system/services/
Device drivers, one sub-project each (pci-bus, ps2-bus, usb-xhci-bus bus drivers)	system/drivers/
On-image test fixtures for the QEMU cases (vfs-test, crash-test, thread-test, …) → /test/system/services	test/system/services/
Build orchestration (kernel + loader, what ships, the aggregate test step)	build.zig (root; the shared user-binary recipe is build-support/, and each library/ domain + binary package carries its own build.zig)
Image assembly + release-x86-64 (the flashable ISO)	build/images.zig
run-x86-64 / run-x86-64-gpu (QEMU/OVMF)	build/qemu.zig
QEMU integration test harness	test/qemu_test.py