# danos documentation

Notes on how danos boots and draws, written to explain the *why* behind the code
rather than restate it. Roughly in the order things happen at runtime:

1. **[efi.md](os-development/efi.md) — EFI / the boot process.** How UEFI firmware finds and
   runs the bootloader, what the loader gathers before `ExitBootServices`, how it
   loads the kernel ELF, and the ABI contract for the jump into the kernel. Start
   here.
2. **[system-image.md](os-development/system-image.md) — system.img, the boot capsule.** The
   bundled user binaries packed into one file in the initial-ramdisk wire
   format, because one open + one sequential read is the only file I/O shape
   firmware is fast at. The trivial container format, the three artifacts one
   build list derives (tree, manifest, capsule), the loader's three-strategy
   fallback chain, and the capsule's kernel-side life as both the spawn table
   and the read-only `/system` mount.
3. **[gop.md](os-development/gop.md) — the Graphics Output Protocol.** How UEFI exposes graphics
   modes (unlike fixed VGA modes), how we detect the monitor's native resolution
   from EDID and switch to it, and the pixel formats we accept or reject.
4. **[framebuffer.md](os-development/framebuffer.md) — the framebuffer.** What the linear
   framebuffer the loader hands over actually is, and what **pitch** (stride)
   means versus width — the detail you have to get right to avoid a skewed image.
5. **[memory-map.md](os-development/memory-map.md) — the memory map.** How the loader learns what
   physical RAM exists and hands it to the kernel in danos's own neutral format,
   rather than leaking UEFI's memory descriptors across the boundary.
6. **[frame-allocator.md](os-development/frame-allocator.md) — the physical frame allocator.** The
   bitmap allocator that hands out and reclaims 4 KiB physical frames from that
   map — the primitive page tables and the heap are built on.
7. **[interrupts.md](os-development/interrupts.md) — interrupts and exceptions.** The GDT, IDT and
   TSS, the exception stubs, and the handler that reports a CPU fault in red instead
   of letting it triple-fault into a silent reset.
8. **[paging.md](os-development/paging.md) — the kernel's page tables.** Building our own 4-level
   page tables, identity-mapping the low 4 GiB, and switching CR3 off the firmware's
   tables onto ours.
9. **[device-interrupts.md](device-driver-development/device-interrupts.md) — device interrupts.** The Local
   APIC and its timer — the kernel's first interrupt that is *handled and returned
   from*, giving it a heartbeat.
10. **[heap.md](os-development/heap.md) — the kernel heap.** A growable free-list allocator built on
   the VMM, exposed as a `std.mem.Allocator` so std containers work — dynamic
   allocation for the kernel.
11. **[scheduling.md](os-development/scheduling.md) — the scheduler.** Fixed-priority preemptive
    multitasking: kernel threads, the context switch, O(1) priority selection, and
    blocking (sleep, wait queues) — the leap to a running system.
12. **[ipc.md](device-driver-development/ipc.md) — inter-process communication.** Bounded blocking
    message-passing channels, then synchronous call/reply between *processes* over
    endpoints — the backbone the microkernel's isolated servers talk over.
13. **[syscall.md](os-development/syscall.md) — system calls.** How ring 3 asks the kernel for
    something: the `syscall`/`sysret` fast path, the trap frame, and why the table is
    deliberately tiny. The numbers are a **private** ABI — [vdso.md](os-development/vdso.md) designs
    the public boundary that will hide them.
14. **[vfs-protocol.md](file-system-development/vfs-protocol.md) — the VFS wire protocol.** The language-neutral
    byte-level spec of the file protocol spoken over IPC: request/reply headers,
    the operation table, mount routing, and the append-only evolution rules — the
    first IPC protocol documented as public ABI.
15. **[drivers.md](device-driver-development/drivers.md) — writing a driver.** The payoff: a driver is an
    ordinary ring-3 process that claims a device, maps its registers, and **sleeps
    until its hardware interrupts it**. The claim is the capability; `irq_ack` is the
    unmask.
16. **[driver-model.md](device-driver-development/driver-model.md) — buses, classes and host controllers.** How
    real driver stacks factor into three shapes and how families share code. The
    three primitives it proposed are long since built (M13 capability passing,
    M14 DMA + barriers, M15 MSI), and the driver *contract* on top of them —
    hello, supervision, restart — is built too (device-manager.md, M18).
17. **[usb-hub.md](device-driver-development/usb-hub.md) — USB hubs.** Built (M22): why hub topology is handled
    *inside* the `usb-xhci-bus` driver rather than a separate hub class driver — a
    device behind a hub is reached by the **controller**, programmed with a route
    string in its slot context — plus the compound-hub reality (a USB 3.0 hub is
    physically two hubs) and detection via the hub's status-change interrupt endpoint.
18. **[process-management.md](os-development/process-management.md) — process management.** The
    microkernel's `ps`/`kill`/SIGCHLD: enumerate as a table snapshot, the
    supervision link as the kill authority, and child-exit notifications over the
    same endpoints IRQs arrive on.
19. **[process-lifecycle.md](os-development/process-lifecycle.md) — the process lifecycle.** Built
    (M17): signals over IPC as the one lifecycle vocabulary every process speaks — the
    POSIX.1-1990 words with message delivery instead of stack hijack, the stable
    `process` module interface, exit reasons, published exit events any stateful
    service can subscribe to (the VFS releasing dead clients' handles), and the two
    iron rules (cleanup is the kernel's job; kill is not a signal).
20. **[device-manager.md](device-driver-development/device-manager.md) — the device manager.** Built (M18,
    through the app surface): the
    tree, the matcher, and the supervisor. Tree structure lives in the manager,
    authority stays in the kernel; bus drivers report what they see; drivers are
    restarted through the lifecycle vocabulary — the plan that turns
    [resilience.md](os-development/resilience.md)'s restart goal into increments.
21. **[input.md](device-driver-development/input.md) — the input module.** Broadcasting input events (keyboard,
    mouse, joystick): why a synchronous rendezvous can't fan out to many listeners, the
    asynchronous `ipc_send` primitive built to fix it, and the per-device subscribe/publish
    service layered on top.
22. **[display.md](device-driver-development/display.md) — the display service.** The display half of the GUI
    track: a user-space compositor that owns the framebuffer, composes a layer stack into
    a double buffer, and presents it. Why GOP and the PCI display device are two views of
    one controller, the device-node + write-combining handoff, and what flicker-free buys
    that tear-free doesn't. Plan: [display-plan.md](device-driver-development/display-plan.md). **v2** (complete) makes
    scanout a pluggable backend — GOP floor + a native virtio-gpu driver, hot-attached, with
    runtime mode-set, EDID, fenced vsync presents, and restart re-attach:
    [display-v2.md](device-driver-development/display-v2.md), plan [display-v2-plan.md](device-driver-development/display-v2-plan.md). Looking
    further out, three research snapshots survey what a *native* driver for real GPU silicon
    would take as another `.scanout` backend: [nvidia-gpus.md](device-driver-development/nvidia-gpus.md) (RTX 3060 /
    Ampere), [amd-gpus.md](device-driver-development/amd-gpus.md) (RX 6600 / RDNA2), and [intel-igpu.md](device-driver-development/intel-igpu.md)
    (Intel iGPU).
23. **[halting.md](os-development/halting.md) — halting.** Why a kernel can't just "exit", and
    how `while (true) hlt` parks the CPU safely once there's nothing left to do.

Start with the north star:

- **[vision.md](vision.md) — the vision.** danos is a **learning-by-doing**
  microkernel: minimal kernel, drivers/services isolated in user space, chosen for
  **resilience** (restartable components). Win condition: runs on the author's PC and
  both Raspberry Pis, ideally with a GUI. Real-time is an option to explore, not a
  requirement. The *why* that shapes everything below.
- **[resilience.md](os-development/resilience.md) — resilience.** A design note (not built yet) on
  fault isolation + live restart — the reincarnation-server + capability model that
  makes "if I break it, I can restart it" real. danos's core motivation.
- **[zig-self-hosting.md](zig-self-hosting.md) — running Zig on danos.** A design note
  (not built yet) on making danos a real Zig target (`-target x86_64-danos`) and
  eventually running the compiler on it. The key realisation: Zig 0.16 reduces an OS
  port to **one seam** (`std.os.danos`), so we build an `os` seam module (→ that seam) plus
  the thin `file-system` module, retire the `posix` shim, and follow a phased path to
  `zig build-exe hello.zig` running on danos — **not** Linux-ABI emulation.
- **[threading.md](os-development/threading.md) — threads, the std-shaped way.** **Built** (M1–M6):
  the `thread` module's `Thread` mirrors `std.Thread`'s API (spawn/join/detach, Mutex/Condition/
  Semaphore) over a **private** thread ABI — several tasks sharing one address space via
  a `thread_spawn` syscall, futex-backed blocking, address-space refcounting. Why it's the
  native type and not literal `std.Thread` (the [private ABI](os-development/syscall.md)), and why
  threads stay a narrow opt-in against the [resilience](os-development/resilience.md) default. Build
  plan + gates: [threading-plan.md](os-development/threading-plan.md).
- **[vdso.md](os-development/vdso.md) — the vDSO, the public system-call boundary.** A design note
  (not built yet) on keeping `abi.zig` genuinely private: a kernel-supplied, C-ABI
  entry blob mapped into every process as the *only* way into the kernel — so the
  syscall numbers can be renumbered or randomised at will, and Rust/C binaries get a
  stable boundary without danos growing a dynamic linker. danos's public ABI = the
  vDSO + the documented IPC wire protocols ([vfs-protocol.md](file-system-development/vfs-protocol.md) first).

Cutting across all of these:

- **[system-requirements.md](system-requirements.md) — system requirements.** The
  hardware needed to run danos: minimum specs (UEFI x86-64, ACPI, PCIe ECAM,
  xHCI, ~128 MiB RAM) grounded in what the boot path actually assumes, plus a
  plain-language guide matching Intel/AMD CPU generations by name.
- **[release-iso.md](os-development/release-iso.md) — the release ISO.** The flashable boot
  media: `zig build release-x86-64` wraps the FAT32 boot volume in a hybrid ISO
  (MBR ESP partition + El Torito EFI entry, one embedded image) that Etcher/dd
  flash to USB or a burner writes to disc — built by an in-repo pure-Python
  tool, like the FAT image itself.
- **[architecture.md](os-development/architecture.md) — the architecture split.** How CPU-specific code is kept
  behind a build-time `arch` module so the generic kernel never names x86_64,
  leaving room for other systems (e.g. an AArch64 Raspberry Pi) later.
- **[arm.md](os-development/arm.md) — ARM targets.** The Raspberry Pi landscape the arch split is
  aiming at: `arm` (32-bit, Pi Zero W) vs `aarch64` (64-bit, Pi 3-5), UEFI vs
  device-tree boot, and what each layer needs.
- **[discovery.md](os-development/discovery.md) — device discovery.** A design note on learning what
  hardware exists via ACPI (x86) or device tree (ARM) behind one neutral device model —
  when to build it, and how to keep it architecture-agnostic.
- **[acpi.md](os-development/acpi.md) — finding the ACPI tables.** The concrete x86 locator chain:
  how the loader captures the **RSDP**, hands its physical address across in `BootInformation`,
  and how the platform derives the **RSDT/XSDT** from it and walks the SDTs — plus the
  live event side (the SCI, the power button, GPE/Notify) the ring-3 acpi service runs.
- **[power.md](os-development/power.md) — the power service.** System power as a domain-named
  service: button/lid/battery events published to subscribers, and init's orderly
  shutdown composing the [lifecycle](os-development/process-lifecycle.md) stop sequence with an ACPI
  S5 write. Firmware-neutral — a PSCI backend drops in on ARM.
- **[timers.md](os-development/timers.md) — timers and time.** The ring-3 surface for reading the
  clock and waiting: why `now()` is a syscall rather than a service, and the one-shot
  timer notification (`timer_bind`) that gives supervisors a timed wait — built on the
  LAPIC heartbeat and calibrated TSC of [device-interrupts.md](device-driver-development/device-interrupts.md).
- **[smp.md](os-development/smp.md) — multiple cores.** A design/research note on how microkernels
  (L4, seL4) handle SMP — big kernel lock vs per-CPU vs multikernel — and how the
  right choice depends on whether danos is chasing real-time or resilience.
- **[coding-standards.md](coding-standards.md) — coding standards.** The naming rule the
  tree follows: non-acronyms are spelled out in full (`message`, not `msg`), files are
  `kebab-case`, code follows Zig's case conventions, and the handful of exceptions
  (POSIX/C ABI names, `init`/`len`/`ptr`, acronyms).
- **[sysv.md](os-development/sysv.md) — the calling convention.** What "the kernel is SysV" means,
  and why the loader→kernel boundary has to pin it (the RDI-vs-RCX handoff).
- **[testing.md](testing.md) — testing.** How the kernel is tested by booting it in
  QEMU and asserting on its serial output — reproducibly, and structured so the
  same tests run across architectures.
- **[logging.md](os-development/logging.md) — logging.** The multi-sink diagnostic log (serial,
  0xE9 debugcon, file later) kept separate from the framebuffer display, plus the
  robustness path: optional framebuffer, POST-code checkpoints, and a persistent
  panic breadcrumb so the kernel survives — and can be diagnosed — with no output.

## How the pieces relate

The boot flow ties them together: UEFI runs the loader ([efi.md](os-development/efi.md)), which
queries the **GOP** to pick a graphics mode ([gop.md](os-development/gop.md)), hands the kernel a
**framebuffer** to draw into ([framebuffer.md](os-development/framebuffer.md)) and a **memory
map** of physical RAM ([memory-map.md](os-development/memory-map.md)); the kernel turns that map
into a **frame allocator** ([frame-allocator.md](os-development/frame-allocator.md)), installs
its **descriptor tables** so CPU faults are caught ([interrupts.md](os-development/interrupts.md)),
builds its own **page tables** and switches onto them ([paging.md](os-development/paging.md)),
brings up the **heap** for dynamic allocation ([heap.md](os-development/heap.md)), starts the
**scheduler** ([scheduling.md](os-development/scheduling.md)) and the **timer** that preempts it
([device-interrupts.md](device-driver-development/device-interrupts.md)) — with tasks blocking, sleeping and
passing messages over **[IPC](device-driver-development/ipc.md)** channels — runs, its CPU-specific bits
behind the [architecture](os-development/architecture.md) boundary, and when idle, or on a panic, it **halts**
([halting.md](os-development/halting.md)).

Above that line the microkernel proper begins: **discovery** ([discovery.md](os-development/discovery.md),
[acpi.md](os-development/acpi.md)) learns what hardware exists, ring-3 processes ask the kernel for
things through the small **[syscall](os-development/syscall.md)** table, isolated servers reach each
other over IPC **endpoints** ([ipc.md](device-driver-development/ipc.md)), and a **[driver](device-driver-development/drivers.md)** claims
a device, maps its registers, and sleeps until the hardware interrupts it — which is
the whole reason for the arrangement ([vision.md](vision.md)).

## Repository layout

danos is a **monorepo of sub-projects**. Each service or driver is a directory that is
its own Zig module — it can hold as many files as it needs, and other sub-projects
reach it *by module name*, never by a path into its files. The source tree deliberately
**mirrors the runtime file-system hierarchy** ([file-system-hierarchy.md](file-system-development/file-system-hierarchy.md)):
what you see under `system/` in the source is what a running danos represents under
`/system`.

**A sub-project is addressed by its directory; its entry point repeats the directory's
name.** `system/services/init/` contains `init.zig` (its root), and produces a binary
addressed as **`system/services/init`** — the repeated leaf resolves away:

| Source (root file)                     | Addressed as (module / binary / hierarchy path) |
|----------------------------------------|--------------------------------------------|
| `system/services/init/init.zig`        | `system/services/init` → `/system/services/init` |
| `system/drivers/ps2-bus/ps2-bus.zig`   | `system/drivers/ps2-bus` → `/system/drivers/ps2-bus` |
| `test/system/services/vfs-test/vfs-test.zig` | `test/system/services/vfs-test` → `/test/system/services/vfs-test` |
| `library/device/pci/pci.zig`           | `library/device/pci` (the `pci` module)          |

In **source**, a sub-project is a directory so it can hold many files — the entry is
`fat/fat.zig`, beside it `fat/engine.zig`, `fat/on-disk.zig`, and so on. When
**addressed or installed**, that collapses to the single canonical path: the `init`
binary installs to `/system/services/init` (a file at that path), not
`/system/services/init/init`. The repeated leaf exists only in source; the directory is
the identity, the entry file is its implementation. (Same idea as a Go package being its
directory, or a macOS `.app` bundle addressed by the bundle, not the executable within.)
A sub-project's extra files are reached through the module, never as separate paths.

```
system/          → /system   danos's own internals (the self-representation)
  boot-handoff.zig             the loader↔kernel contract (the `boot-handoff` module)
  abi.zig                      the private kernel↔userspace syscall ABI (the `abi` module)
  parameters.zig  initial-ramdisk.zig   shared contracts
  kernel/                     IPC, memory, scheduling, the VFS root, the private syscall dispatch
    architecture/x86_64/      the `architecture` module (never named by generic code)
    devices-broker.zig        the syscall-facing device table
    platform.zig acpi.zig fdt.zig device-model.zig   firmware discovery + the kernel's
                              device model — the implementation of what /system/devices reflects
  drivers/     pci-bus/ ps2-bus/ usb-xhci-bus/   one sub-project per driver → /system/drivers
  services/    init/ fat/ device-manager/   system servers → /system/services (fat/ holds
                              fat.zig, engine.zig, on-disk.zig)
library/         → /lib       libraries, one sub-directory each
  kernel/                     the danos-native system library (kernel32-style): the syscall
                              surface split by concern — ipc, memory (heap/dma/shared-memory),
                              process, time, logging, file-system, thread, service, plus the
                              system-call stubs and the start/root entry shim
  device/                     device code by domain — mmio/ model/ pci/ usb/ acpi/ driver/
                              block/ — each a shareable data module (device-abi, pci-class,
                              usb-abi/ids, acpi-ids) plus a logic module (mmio, pci, usb, aml,
                              driver — the device-access + device-manager-hello client)
  client/                     userspace service clients (display, input) — a program's view of
                              a service, layered over that service's protocol
  protocol/                   driver↔service wire contracts (vfs block display scanout input
                              power device-manager usb-transfer), one module per directory
boot/            → /boot      the loaders
test/            → /test      the test tree: the QEMU harness (qemu_test.py, host-side)
  system/services/            beside the on-image test fixtures — vfs-test/ thread-test/
                              crash-test/ … — whose repo path IS their boot-volume path
                              (/test/system/services/<name>)
build-support/                the danos build API (build-time only, nothing on the image):
                              the shared user-binary recipe + default-import wiring every
                              build file consumes (docs/build-packages-plan.md)
build/                        root-build helpers: image assembly (images.zig) + the QEMU
                              run steps (qemu.zig)
tools/                        host-side build scripts
```

**Builds are packages** (docs/build-packages-plan.md): each `library/` domain owns a
`build.zig`/`build.zig.zon` exporting its modules (with a standalone `zig build test`),
every binary directory is a ~15-line package build, and the root `build.zig`
orchestrates — the kernel + loader, what ships, and the aggregate test step — with
image assembly in `build/images.zig` and the QEMU run steps in `build/qemu.zig`.

**Wire protocols live in `library/protocol/`**, one module per directory
(`library/protocol/vfs/vfs-protocol.zig` is the `vfs-protocol` module), imported by module
name. A protocol is the seam between a low-level driver and the higher-level service it
serves — block ↔ the filesystem, a scanout driver ↔ the compositor — so both sides depend
on the contract, not on each other, and the contract belongs to neither sub-project. A
client module may *wrap* one for application convenience (the `file-system` module over
`vfs-protocol`, and the `block`, `display`, `input` clients over theirs), but the protocol
module is the boundary — a client re-exports no protocol, it imports it by name. A driver's
private wire to its *hardware* (virtio-gpu's command set) is not a service seam and stays a
driver-private file, beside the transport that reaches the same device.

**Device code lives in `library/device/<domain>/`**, grouped by what it is about (pci, usb,
acpi, and the cross-cutting device model) and split by dependency weight: a data module of
enums and wire types that is `std`-only and cheap for anyone to import, and a logic module
that needs `mmio` or IPC. This is what keeps the microkernel out of device business — it
imports exactly one `library/` module, `device-abi` (the descriptor types its broker
marshals across the syscall boundary), and nothing with logic or a taxonomy in it. That
lone pure-data import is the only edge from `system/kernel/` into `library/`.

There is **no POSIX/C compatibility layer today**: danos programs do file I/O through the
danos-native `file-system` module (open/read/write/list over the VFS). A hand-rolled POSIX shim
(`library/posix/`) was retired as premature — the real POSIX/C surface will come later
from the `std.os.danos` seam (and, eventually, musl) when danos becomes a Zig target (see
[zig-self-hosting.md](zig-self-hosting.md)). When it does, the foreign-ABI naming
exception in [coding-standards.md](coding-standards.md) applies to that seam.

## Source map

| Area | Code |
|------|------|
| Boot methods (one per way of booting the kernel) | `boot/` — `efi.zig` (UEFI) → `BOOTX64.efi` |
| Kernel entry, panic, bring-up | `system/kernel/kernel.zig` |
| Loader↔kernel handoff (`BootInformation`, `Framebuffer`, `MemoryMap`, VM layout) | `system/boot-handoff.zig` |
| Private kernel↔userspace syscall ABI (`SystemCall`, mmap prot flags, `page_size`) — the system library speaks it, not apps | `system/abi.zig` |
| Device wire types (`DeviceDescriptor`, `DeviceClass`, …) | `library/device/model/device-abi.zig` |
| Physical frame allocator | `system/kernel/pmm.zig` |
| Kernel heap (`std.mem.Allocator`) | `system/kernel/heap.zig` |
| Scheduler (fixed-priority preemptive; blocking, wait queues) | `system/kernel/scheduler.zig` |
| Big kernel lock + interrupt-safe critical sections | `system/kernel/sync.zig` |
| IPC channels between kernel threads (message passing) | `system/kernel/ipc.zig` |
| IPC endpoints: cross-address-space call/reply, handles, notifications | `system/kernel/ipc-synchronous.zig` |
| User processes: ELF loading, address spaces, the syscall table | `system/kernel/process.zig` |
| VFS root: mount table + kernel-served nodes (`fs_resolve`/`fs_node`); wire protocol in `library/protocol/vfs/vfs-protocol.zig` | `system/kernel/vfs.zig` |
| Device tree + claim capability + `device_register` containment | `system/kernel/devices-broker.zig` |
| IRQ-as-IPC: routing a device interrupt to a driver's endpoint | `system/kernel/irq.zig` |
| Hardware discovery (ACPI/device tree) behind one neutral device model | `system/kernel/platform.zig` |
| Framebuffer text console (mirrors to serial) | `system/kernel/console.zig` |
| In-kernel test cases | `system/kernel/tests.zig` |
| Arch-specific kernel code (`halt`, GDT/IDT/TSS, exception + interrupt stubs, page tables, APIC/IO-APIC/timer, serial, linker script) | `system/kernel/architecture/x86_64/` |
| danos-native system library (kernel32-style): the syscall surface by concern — `ipc`, `memory`, `process`, `time`, `logging`, `file-system`, `thread`, `service` — the stable application ABI | `library/kernel/` |
| Service clients (a program's view of a service) and device clients | `library/client/` (display, input), `library/device/driver` |
| System services (init, the `fat` filesystem, the device-manager) | `system/services/` |
| Device drivers, one sub-project each (`pci-bus`, `ps2-bus`, `usb-xhci-bus` bus drivers) | `system/drivers/` |
| On-image test fixtures for the QEMU cases (`vfs-test`, `crash-test`, `thread-test`, …) → `/test/system/services` | `test/system/services/` |
| Build orchestration (kernel + loader, what ships, the aggregate test step) | `build.zig` (root; the shared user-binary recipe is `build-support/`, and each `library/` domain + binary package carries its own `build.zig`) |
| Image assembly + `release-x86-64` (the flashable ISO) | `build/images.zig` |
| `run-x86-64` / `run-x86-64-gpu` (QEMU/OVMF) | `build/qemu.zig` |
| QEMU integration test harness | `test/qemu_test.py` |