daniel
/
danos
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects 1 Releases Wiki Activity
danos/docs/README.md

17 KiB
Raw Blame History

danos documentation

Notes on how danos boots and draws, written to explain the why behind the code rather than restate it. Roughly in the order things happen at runtime:

  1. efi.md — EFI / the boot process. How UEFI firmware finds and runs the bootloader, what the loader gathers before ExitBootServices, how it loads the kernel ELF, and the ABI contract for the jump into the kernel. Start here.
  2. gop.md — the Graphics Output Protocol. How UEFI exposes graphics modes (unlike fixed VGA modes), how we detect the monitor's native resolution from EDID and switch to it, and the pixel formats we accept or reject.
  3. framebuffer.md — the framebuffer. What the linear framebuffer the loader hands over actually is, and what pitch (stride) means versus width — the detail you have to get right to avoid a skewed image.
  4. memory-map.md — the memory map. How the loader learns what physical RAM exists and hands it to the kernel in danos's own neutral format, rather than leaking UEFI's memory descriptors across the boundary.
  5. frame-allocator.md — the physical frame allocator. The bitmap allocator that hands out and reclaims 4 KiB physical frames from that map — the primitive page tables and the heap are built on.
  6. interrupts.md — interrupts and exceptions. The GDT, IDT and TSS, the exception stubs, and the handler that reports a CPU fault in red instead of letting it triple-fault into a silent reset.
  7. paging.md — the kernel's page tables. Building our own 4-level page tables, identity-mapping the low 4 GiB, and switching CR3 off the firmware's tables onto ours.
  8. device-interrupts.md — device interrupts. The Local APIC and its timer — the kernel's first interrupt that is handled and returned from, giving it a heartbeat.
  9. heap.md — the kernel heap. A growable free-list allocator built on the VMM, exposed as a std.mem.Allocator so std containers work — dynamic allocation for the kernel.
  10. scheduling.md — the scheduler. Fixed-priority preemptive multitasking: kernel threads, the context switch, O(1) priority selection, and blocking (sleep, wait queues) — the leap to a running system.
  11. ipc.md — inter-process communication. Bounded blocking message-passing channels, then synchronous call/reply between processes over endpoints — the backbone the microkernel's isolated servers talk over.
  12. syscall.md — system calls. How ring 3 asks the kernel for something: the syscall/sysret fast path, the trap frame, and why the table is deliberately tiny.
  13. drivers.md — writing a driver. The payoff: a driver is an ordinary ring-3 process that claims a device, maps its registers, and sleeps until its hardware interrupts it. The claim is the capability; irq_ack is the unmask.
  14. driver-model.md — buses, classes and host controllers. How real driver stacks factor into three shapes and how families share code. The three primitives it proposed are long since built (M13 capability passing, M14 DMA + barriers, M15 MSI), and the driver contract on top of them — hello, supervision, restart — is built too (device-manager.md, M18).
  15. process-management.md — process management. The microkernel's ps/kill/SIGCHLD: enumerate as a table snapshot, the supervision link as the kill authority, and child-exit notifications over the same endpoints IRQs arrive on.
  16. process-lifecycle.md — the process lifecycle. Built (M17): signals over IPC as the one lifecycle vocabulary every process speaks — the POSIX.1-1990 words with message delivery instead of stack hijack, the stable runtime.process interface, exit reasons, published exit events any stateful service can subscribe to (the VFS releasing dead clients' handles), and the two iron rules (cleanup is the kernel's job; kill is not a signal).
  17. device-manager.md — the device manager. Built (M18, through the app surface): the tree, the matcher, and the supervisor. Tree structure lives in the manager, authority stays in the kernel; bus drivers report what they see; drivers are restarted through the lifecycle vocabulary — the plan that turns resilience.md's restart goal into increments.
  18. input.md — the input module. Broadcasting input events (keyboard, mouse, joystick): why a synchronous rendezvous can't fan out to many listeners, the asynchronous ipc_send primitive built to fix it, and the per-device subscribe/publish service layered on top.
  19. display.md — the display service. The display half of the GUI track: a user-space compositor that owns the framebuffer, composes a layer stack into a double buffer, and presents it. Why GOP and the PCI display device are two views of one controller, the device-node + write-combining handoff, and what flicker-free buys that tear-free doesn't. Plan: display-plan.md.
  20. halting.md — halting. Why a kernel can't just "exit", and how while (true) hlt parks the CPU safely once there's nothing left to do.

Start with the north star:

  • vision.md — the vision. danos is a learning-by-doing microkernel: minimal kernel, drivers/services isolated in user space, chosen for resilience (restartable components). Win condition: runs on the author's PC and both Raspberry Pis, ideally with a GUI. Real-time is an option to explore, not a requirement. The why that shapes everything below.
  • resilience.md — resilience. A design note (not built yet) on fault isolation + live restart — the reincarnation-server + capability model that makes "if I break it, I can restart it" real. danos's core motivation.
  • zig-self-hosting.md — running Zig on danos. A design note (not built yet) on making danos a real Zig target (-target x86_64-danos) and eventually running the compiler on it. The key realisation: Zig 0.16 reduces an OS port to one seam (std.os.danos), so we build runtime.os (→ that seam) plus a thin runtime.fs, retire the posix shim, and follow a phased path to zig build-exe hello.zig running on danos — not Linux-ABI emulation.

Cutting across all of these:

  • system-requirements.md — system requirements. The hardware needed to run danos: minimum specs (UEFI x86-64, ACPI, PCIe ECAM, xHCI, ~128 MiB RAM) grounded in what the boot path actually assumes, plus a plain-language guide matching Intel/AMD CPU generations by name.
  • arch.md — the architecture split. How CPU-specific code is kept behind a build-time arch module so the generic kernel never names x86_64, leaving room for other systems (e.g. an AArch64 Raspberry Pi) later.
  • arm.md — ARM targets. The Raspberry Pi landscape the arch split is aiming at: arm (32-bit, Pi Zero W) vs aarch64 (64-bit, Pi 3-5), UEFI vs device-tree boot, and what each layer needs.
  • discovery.md — device discovery. A design note on learning what hardware exists via ACPI (x86) or device tree (ARM) behind one neutral device model — when to build it, and how to keep it architecture-agnostic.
  • acpi.md — finding the ACPI tables. The concrete x86 locator chain: how the loader captures the RSDP, hands its physical address across in BootInfo, and how the platform derives the RSDT/XSDT from it and walks the SDTs — plus the live event side (the SCI, the power button, GPE/Notify) the ring-3 acpi service runs.
  • power.md — the power service. System power as a domain-named service: button/lid/battery events published to subscribers, and init's orderly shutdown composing the lifecycle stop sequence with an ACPI S5 write. Firmware-neutral — a PSCI backend drops in on ARM.
  • timers.md — timers and time. The ring-3 surface for reading the clock and waiting: why now() is a syscall rather than a service, and the one-shot timer notification (timer_bind) that gives supervisors a timed wait — built on the LAPIC heartbeat and calibrated TSC of device-interrupts.md.
  • smp.md — multiple cores. A design/research note on how microkernels (L4, seL4) handle SMP — big kernel lock vs per-CPU vs multikernel — and how the right choice depends on whether danos is chasing real-time or resilience.
  • coding-standards.md — coding standards. The naming rule the tree follows: non-acronyms are spelled out in full (message, not msg), files are kebab-case, code follows Zig's case conventions, and the handful of exceptions (POSIX/C ABI names, init/len/ptr, acronyms).
  • sysv.md — the calling convention. What "the kernel is SysV" means, and why the loader→kernel boundary has to pin it (the RDI-vs-RCX handoff).
  • testing.md — testing. How the kernel is tested by booting it in QEMU and asserting on its serial output — reproducibly, and structured so the same tests run across architectures.
  • logging.md — logging. The multi-sink diagnostic log (serial, 0xE9 debugcon, file later) kept separate from the framebuffer display, plus the robustness path: optional framebuffer, POST-code checkpoints, and a persistent panic breadcrumb so the kernel survives — and can be diagnosed — with no output.

How the pieces relate

The boot flow ties them together: UEFI runs the loader (efi.md), which queries the GOP to pick a graphics mode (gop.md), hands the kernel a framebuffer to draw into (framebuffer.md) and a memory map of physical RAM (memory-map.md); the kernel turns that map into a frame allocator (frame-allocator.md), installs its descriptor tables so CPU faults are caught (interrupts.md), builds its own page tables and switches onto them (paging.md), brings up the heap for dynamic allocation (heap.md), starts the scheduler (scheduling.md) and the timer that preempts it (device-interrupts.md) — with tasks blocking, sleeping and passing messages over IPC channels — runs, its CPU-specific bits behind the arch boundary, and when idle, or on a panic, it halts (halting.md).

Above that line the microkernel proper begins: discovery (discovery.md, acpi.md) learns what hardware exists, ring-3 processes ask the kernel for things through the small syscall table, isolated servers reach each other over IPC endpoints (ipc.md), and a driver claims a device, maps its registers, and sleeps until the hardware interrupts it — which is the whole reason for the arrangement (vision.md).

Repository layout

danos is a monorepo of sub-projects. Each service or driver is a directory that is its own Zig module — it can hold as many files as it needs, and other sub-projects reach it by module name, never by a path into its files. The source tree deliberately mirrors the runtime FHS (danos-file-system-hierarchy-FSH.md): what you see under system/ in the source is what a running danos represents under /system.

A sub-project is addressed by its directory; its entry point repeats the directory's name. system/services/init/ contains init.zig (its root), and produces a binary addressed as system/services/init — the repeated leaf resolves away:

Source (root file) Addressed as (module / binary / FHS path)
system/services/init/init.zig system/services/init → /system/services/init
system/drivers/ps2-bus/ps2-bus.zig system/drivers/ps2-bus → /system/drivers/ps2-bus
library/runtime/runtime.zig library/runtime (the runtime module)

In source, a sub-project is a directory so it can hold many files — the entry is init/init.zig, beside it vfs/vfs-test.zig, vfs/protocol.zig, and so on. When addressed or installed, that collapses to the single canonical path: the init binary installs to /system/services/init (a file at that path), not /system/services/init/init. The repeated leaf exists only in source; the directory is the identity, the entry file is its implementation. (Same idea as a Go package being its directory, or a macOS .app bundle addressed by the bundle, not the executable within.) A sub-project's extra files are reached through the module, never as separate paths.

system/          → /system   danos's own internals (the self-representation)
  boot-handoff.zig             the loader↔kernel contract (the `boot-handoff` module)
  abi.zig                      the private kernel↔runtime syscall ABI (the `abi` module)
  parameters.zig  initial-ramdisk.zig  shared contracts
  kernel/                     IPC, memory, scheduling, the private syscall dispatch
    architecture/x86_64/      the `architecture` module (never named by generic code)
  devices/                    the device model /system/devices reflects (+ aml/)
    device-abi.zig            the device wire types (the `device-abi` module)
  drivers/     hpet/ bus/   one sub-project per driver → /system/drivers
  services/    init/ vfs/ device-manager/   system servers → /system/services (vfs/ holds
                              vfs.zig, vfs-test.zig, protocol.zig)
library/         → /lib       libraries, one sub-directory each
  runtime/                    the danos-native runtime + file API (fs) — the stable application ABI
boot/            → /boot      the loaders
tools/  test/                 host-side build + QEMU test harness

A sub-project exposes its public interface as a module: system/services/vfs/ owns the VFS wire protocol (protocol.zig, the vfs-protocol module), which the runtime's file API (runtime.fs) imports by name. usb/block drivers expose their protocols the same way.

There is no POSIX/C compatibility layer today: danos programs do file I/O through the danos-native runtime.fs (open/read/write/list over the VFS). A hand-rolled POSIX shim (library/posix/) was retired as premature — the real POSIX/C surface will come later from the std.os.danos seam (and, eventually, musl) when danos becomes a Zig target (see zig-self-hosting.md). When it does, the foreign-ABI naming exception in coding-standards.md applies to that seam.

Source map

Area Code
Boot methods (one per way of booting the kernel) boot/ — efi.zig (UEFI) → BOOTX64.efi
Kernel entry, panic, bring-up system/kernel/kernel.zig
Loader↔kernel handoff (BootInfo, Framebuffer, MemoryMap, VM layout) system/boot-handoff.zig
Private kernel↔runtime syscall ABI (SystemCall, mmap prot flags, page_size) — the runtime speaks it, not apps system/abi.zig
Device wire types (DeviceDescriptor, DeviceClass, …) system/devices/device-abi.zig
Physical frame allocator system/kernel/pmm.zig
Kernel heap (std.mem.Allocator) system/kernel/heap.zig
Scheduler (fixed-priority preemptive; blocking, wait queues) system/kernel/scheduler.zig
Big kernel lock + interrupt-safe critical sections system/kernel/sync.zig
IPC channels between kernel threads (message passing) system/kernel/ipc.zig
IPC endpoints: cross-address-space call/reply, handles, notifications system/kernel/ipc-synchronous.zig
User processes: ELF loading, address spaces, the syscall table system/kernel/process.zig
Device tree + claim capability + device_register containment system/kernel/devices-broker.zig
IRQ-as-IPC: routing a device interrupt to a driver's endpoint system/kernel/irq.zig
Hardware discovery (ACPI/device tree) behind one neutral device model system/devices/
Framebuffer text console (mirrors to serial) system/kernel/console.zig
In-kernel test cases system/kernel/tests.zig
Arch-specific kernel code (halt, GDT/IDT/TSS, exception + interrupt stubs, page tables, APIC/IO-APIC/timer, serial, linker script) system/kernel/architecture/x86_64/
danos-native runtime (runtime): syscall wrappers, heap, IPC, device access, the file API (fs) — the stable application ABI library/runtime/
System services (init, the VFS server + protocol, the device-manager) system/services/
Device drivers, one sub-project each (pci-bus, ps2-bus, usb-xhci-bus bus drivers) system/drivers/
Build + run-x86-64 (QEMU/OVMF) build.zig
QEMU integration test harness test/qemu_test.py