204 lines
14 KiB
Markdown
204 lines
14 KiB
Markdown
# danos documentation
|
|
|
|
Notes on how danos boots and draws, written to explain the *why* behind the code
|
|
rather than restate it. Roughly in the order things happen at runtime:
|
|
|
|
1. **[efi.md](efi.md) — EFI / the boot process.** How UEFI firmware finds and
|
|
runs the bootloader, what the loader gathers before `ExitBootServices`, how it
|
|
loads the kernel ELF, and the ABI contract for the jump into the kernel. Start
|
|
here.
|
|
2. **[gop.md](gop.md) — the Graphics Output Protocol.** How UEFI exposes graphics
|
|
modes (unlike fixed VGA modes), how we detect the monitor's native resolution
|
|
from EDID and switch to it, and the pixel formats we accept or reject.
|
|
3. **[framebuffer.md](framebuffer.md) — the framebuffer.** What the linear
|
|
framebuffer the loader hands over actually is, and what **pitch** (stride)
|
|
means versus width — the detail you have to get right to avoid a skewed image.
|
|
4. **[memory-map.md](memory-map.md) — the memory map.** How the loader learns what
|
|
physical RAM exists and hands it to the kernel in danos's own neutral format,
|
|
rather than leaking UEFI's memory descriptors across the boundary.
|
|
5. **[frame-allocator.md](frame-allocator.md) — the physical frame allocator.** The
|
|
bitmap allocator that hands out and reclaims 4 KiB physical frames from that
|
|
map — the primitive page tables and the heap are built on.
|
|
6. **[interrupts.md](interrupts.md) — interrupts and exceptions.** The GDT, IDT and
|
|
TSS, the exception stubs, and the handler that reports a CPU fault in red instead
|
|
of letting it triple-fault into a silent reset.
|
|
7. **[paging.md](paging.md) — the kernel's page tables.** Building our own 4-level
|
|
page tables, identity-mapping the low 4 GiB, and switching CR3 off the firmware's
|
|
tables onto ours.
|
|
8. **[device-interrupts.md](device-interrupts.md) — device interrupts.** The Local
|
|
APIC and its timer — the kernel's first interrupt that is *handled and returned
|
|
from*, giving it a heartbeat.
|
|
9. **[heap.md](heap.md) — the kernel heap.** A growable free-list allocator built on
|
|
the VMM, exposed as a `std.mem.Allocator` so std containers work — dynamic
|
|
allocation for the kernel.
|
|
10. **[scheduling.md](scheduling.md) — the scheduler.** Fixed-priority preemptive
|
|
multitasking: kernel threads, the context switch, O(1) priority selection, and
|
|
blocking (sleep, wait queues) — the leap to a running system.
|
|
11. **[ipc.md](ipc.md) — inter-process communication.** Bounded blocking
|
|
message-passing channels, then synchronous call/reply between *processes* over
|
|
endpoints — the backbone the microkernel's isolated servers talk over.
|
|
12. **[syscall.md](syscall.md) — system calls.** How ring 3 asks the kernel for
|
|
something: the `syscall`/`sysret` fast path, the trap frame, and why the table is
|
|
deliberately tiny.
|
|
13. **[drivers.md](drivers.md) — writing a driver.** The payoff: a driver is an
|
|
ordinary ring-3 process that claims a device, maps its registers, and **sleeps
|
|
until its hardware interrupts it**. The claim is the capability; `irq_ack` is the
|
|
unmask.
|
|
14. **[driver-model.md](driver-model.md) — buses, classes and host controllers.** How
|
|
real driver stacks factor into three shapes, how families share code, and the
|
|
proposed ABI for the three primitives still missing (capability passing, DMA +
|
|
memory barriers, MSI).
|
|
15. **[halting.md](halting.md) — halting.** Why a kernel can't just "exit", and
|
|
how `while (true) hlt` parks the CPU safely once there's nothing left to do.
|
|
|
|
Start with the north star:
|
|
|
|
- **[vision.md](vision.md) — the vision.** danos is a **learning-by-doing**
|
|
microkernel: minimal kernel, drivers/services isolated in user space, chosen for
|
|
**resilience** (restartable components). Win condition: runs on the author's PC and
|
|
both Raspberry Pis, ideally with a GUI. Real-time is an option to explore, not a
|
|
requirement. The *why* that shapes everything below.
|
|
- **[resilience.md](resilience.md) — resilience.** A design note (not built yet) on
|
|
fault isolation + live restart — the reincarnation-server + capability model that
|
|
makes "if I break it, I can restart it" real. danos's core motivation.
|
|
|
|
Cutting across all of these:
|
|
|
|
- **[arch.md](arch.md) — the architecture split.** How CPU-specific code is kept
|
|
behind a build-time `arch` module so the generic kernel never names x86_64,
|
|
leaving room for other systems (e.g. an AArch64 Raspberry Pi) later.
|
|
- **[arm.md](arm.md) — ARM targets.** The Raspberry Pi landscape the arch split is
|
|
aiming at: `arm` (32-bit, Pi Zero W) vs `aarch64` (64-bit, Pi 3-5), UEFI vs
|
|
device-tree boot, and what each layer needs.
|
|
- **[discovery.md](discovery.md) — device discovery.** A design note on learning what
|
|
hardware exists via ACPI (x86) or device tree (ARM) behind one neutral device model —
|
|
when to build it, and how to keep it architecture-agnostic.
|
|
- **[acpi.md](acpi.md) — finding the ACPI tables.** The concrete x86 locator chain:
|
|
how the loader captures the **RSDP**, hands its physical address across in `BootInfo`,
|
|
and how the platform derives the **RSDT/XSDT** from it and walks the SDTs.
|
|
- **[smp.md](smp.md) — multiple cores.** A design/research note on how microkernels
|
|
(L4, seL4) handle SMP — big kernel lock vs per-CPU vs multikernel — and how the
|
|
right choice depends on whether danos is chasing real-time or resilience.
|
|
- **[coding-standards.md](coding-standards.md) — coding standards.** The naming rule the
|
|
tree follows: non-acronyms are spelled out in full (`message`, not `msg`), files are
|
|
`kebab-case`, code follows Zig's case conventions, and the handful of exceptions
|
|
(POSIX/C ABI names, `init`/`len`/`ptr`, acronyms).
|
|
- **[sysv.md](sysv.md) — the calling convention.** What "the kernel is SysV" means,
|
|
and why the loader→kernel boundary has to pin it (the RDI-vs-RCX handoff).
|
|
- **[testing.md](testing.md) — testing.** How the kernel is tested by booting it in
|
|
QEMU and asserting on its serial output — reproducibly, and structured so the
|
|
same tests run across architectures.
|
|
- **[logging.md](logging.md) — logging.** The multi-sink diagnostic log (serial,
|
|
0xE9 debugcon, file later) kept separate from the framebuffer display, plus the
|
|
robustness path: optional framebuffer, POST-code checkpoints, and a persistent
|
|
panic breadcrumb so the kernel survives — and can be diagnosed — with no output.
|
|
|
|
## How the pieces relate
|
|
|
|
The boot flow ties them together: UEFI runs the loader ([efi.md](efi.md)), which
|
|
queries the **GOP** to pick a graphics mode ([gop.md](gop.md)), hands the kernel a
|
|
**framebuffer** to draw into ([framebuffer.md](framebuffer.md)) and a **memory
|
|
map** of physical RAM ([memory-map.md](memory-map.md)); the kernel turns that map
|
|
into a **frame allocator** ([frame-allocator.md](frame-allocator.md)), installs
|
|
its **descriptor tables** so CPU faults are caught ([interrupts.md](interrupts.md)),
|
|
builds its own **page tables** and switches onto them ([paging.md](paging.md)),
|
|
brings up the **heap** for dynamic allocation ([heap.md](heap.md)), starts the
|
|
**scheduler** ([scheduling.md](scheduling.md)) and the **timer** that preempts it
|
|
([device-interrupts.md](device-interrupts.md)) — with tasks blocking, sleeping and
|
|
passing messages over **[IPC](ipc.md)** channels — runs, its CPU-specific bits
|
|
behind the [arch](arch.md) boundary, and when idle, or on a panic, it **halts**
|
|
([halting.md](halting.md)).
|
|
|
|
Above that line the microkernel proper begins: **discovery** ([discovery.md](discovery.md),
|
|
[acpi.md](acpi.md)) learns what hardware exists, ring-3 processes ask the kernel for
|
|
things through the small **[syscall](syscall.md)** table, isolated servers reach each
|
|
other over IPC **endpoints** ([ipc.md](ipc.md)), and a **[driver](drivers.md)** claims
|
|
a device, maps its registers, and sleeps until the hardware interrupts it — which is
|
|
the whole reason for the arrangement ([vision.md](vision.md)).
|
|
|
|
## Repository layout
|
|
|
|
danos is a **monorepo of sub-projects**. Each service or driver is a directory that is
|
|
its own Zig module — it can hold as many files as it needs, and other sub-projects
|
|
reach it *by module name*, never by a path into its files. The source tree deliberately
|
|
**mirrors the runtime FHS** ([danos-file-system-hierarchy-FSH.md](danos-file-system-hierarchy-FSH.md)):
|
|
what you see under `system/` in the source is what a running danos represents under
|
|
`/system`.
|
|
|
|
**A sub-project is addressed by its directory; its entry point repeats the directory's
|
|
name.** `system/services/init/` contains `init.zig` (its root), and produces a binary
|
|
addressed as **`system/services/init`** — the repeated leaf resolves away:
|
|
|
|
| Source (root file) | Addressed as (module / binary / FHS path) |
|
|
|----------------------------------------|--------------------------------------------|
|
|
| `system/services/init/init.zig` | `system/services/init` → `/system/services/init` |
|
|
| `system/drivers/hpet/hpet.zig` | `system/drivers/hpet` → `/system/drivers/hpet` |
|
|
| `library/runtime/runtime.zig` | `library/runtime` (the `runtime` module) |
|
|
|
|
In **source**, a sub-project is a directory so it can hold many files — the entry is
|
|
`init/init.zig`, beside it `vfs/vfs-test.zig`, `vfs/protocol.zig`, and so on. When
|
|
**addressed or installed**, that collapses to the single canonical path: the `init`
|
|
binary installs to `/system/services/init` (a file at that path), not
|
|
`/system/services/init/init`. The repeated leaf exists only in source; the directory is
|
|
the identity, the entry file is its implementation. (Same idea as a Go package being its
|
|
directory, or a macOS `.app` bundle addressed by the bundle, not the executable within.)
|
|
A sub-project's extra files are reached through the module, never as separate paths.
|
|
|
|
```
|
|
system/ → /system danos's own internals (the self-representation)
|
|
boot-handoff.zig the loader↔kernel contract (the `boot-handoff` module)
|
|
abi.zig the core kernel↔user ABI (the `abi` module)
|
|
parameters.zig initial-ramdisk.zig shared contracts
|
|
kernel/ IPC, memory, scheduling, the private syscall dispatch
|
|
architecture/x86_64/ the `architecture` module (never named by generic code)
|
|
devices/ the device model /system/devices reflects (+ aml/)
|
|
device-abi.zig the device wire types (the `device-abi` module)
|
|
drivers/ hpet/ bus/ one sub-project per driver → /system/drivers
|
|
services/ init/ vfs/ device-manager/ system servers → /system/services (vfs/ holds
|
|
vfs.zig, vfs-test.zig, protocol.zig)
|
|
library/ → /lib libraries, one sub-directory each
|
|
runtime/ the danos-native runtime — the stable application ABI
|
|
posix/ POSIX/C compatibility, layered over runtime
|
|
boot/ → /boot the loaders
|
|
tools/ test/ host-side build + QEMU test harness
|
|
```
|
|
|
|
A sub-project exposes its **public interface as a module**: `system/services/vfs/` owns
|
|
the VFS wire protocol (`protocol.zig`, the `vfs-protocol` module), which the POSIX
|
|
layer imports by name. `usb`/`block` drivers will expose their protocols the same way.
|
|
|
|
`library/posix/` is special: it is the **one place** POSIX/C spellings are allowed
|
|
verbatim (`stat`, `O_CREAT`, `fopen`, `errno`). Everywhere else follows the danos
|
|
naming rule with no exception — see [coding-standards.md](coding-standards.md). The
|
|
POSIX layer calls the runtime, never the kernel's system calls directly, so it never
|
|
appears in the private-ABI path.
|
|
|
|
## Source map
|
|
|
|
| Area | Code |
|
|
|------|------|
|
|
| Boot methods (one per way of booting the kernel) | `boot/` — `efi.zig` (UEFI) → `BOOTX64.efi` |
|
|
| Kernel entry, panic, bring-up | `system/kernel/kernel.zig` |
|
|
| Loader↔kernel handoff (`BootInfo`, `Framebuffer`, `MemoryMap`, VM layout) | `system/boot-handoff.zig` |
|
|
| Core kernel↔user ABI (`SystemCall`, mmap prot flags, `page_size`) | `system/abi.zig` |
|
|
| Device wire types (`DeviceDescriptor`, `DeviceClass`, …) | `system/devices/device-abi.zig` |
|
|
| Physical frame allocator | `system/kernel/pmm.zig` |
|
|
| Kernel heap (`std.mem.Allocator`) | `system/kernel/heap.zig` |
|
|
| Scheduler (fixed-priority preemptive; blocking, wait queues) | `system/kernel/scheduler.zig` |
|
|
| Big kernel lock + interrupt-safe critical sections | `system/kernel/sync.zig` |
|
|
| IPC channels between kernel threads (message passing) | `system/kernel/ipc.zig` |
|
|
| IPC endpoints: cross-address-space call/reply, handles, notifications | `system/kernel/ipc-synchronous.zig` |
|
|
| User processes: ELF loading, address spaces, the syscall table | `system/kernel/process.zig` |
|
|
| Device tree + claim capability + `device_register` containment | `system/kernel/devices-broker.zig` |
|
|
| IRQ-as-IPC: routing a device interrupt to a driver's endpoint | `system/kernel/irq.zig` |
|
|
| Hardware discovery (ACPI/device tree) behind one neutral device model | `system/devices/` |
|
|
| Framebuffer text console (mirrors to serial) | `system/kernel/console.zig` |
|
|
| In-kernel test cases | `system/kernel/tests.zig` |
|
|
| Arch-specific kernel code (`halt`, GDT/IDT/TSS, exception + interrupt stubs, page tables, APIC/IO-APIC/timer, serial, linker script) | `system/kernel/architecture/x86_64/` |
|
|
| danos-native runtime (`runtime`): syscall wrappers, heap, IPC, device access — the stable application ABI | `library/runtime/` |
|
|
| POSIX/C compatibility (`posix`): unistd, stdio — the one place POSIX names are allowed | `library/posix/` |
|
|
| System services (init, the VFS server + `protocol`, the device-manager) | `system/services/` |
|
|
| Device drivers, one sub-project each (`hpet` leaf driver, `bus` bus driver) | `system/drivers/` |
|
|
| Build + `run-x86-64` (QEMU/OVMF) | `build.zig` |
|
|
| QEMU integration test harness | `test/qemu_test.py` |
|