277 lines
20 KiB
Markdown
277 lines
20 KiB
Markdown
# danos documentation
|
|
|
|
Notes on how danos boots and draws, written to explain the *why* behind the code
|
|
rather than restate it. Roughly in the order things happen at runtime:
|
|
|
|
1. **[efi.md](efi.md) — EFI / the boot process.** How UEFI firmware finds and
|
|
runs the bootloader, what the loader gathers before `ExitBootServices`, how it
|
|
loads the kernel ELF, and the ABI contract for the jump into the kernel. Start
|
|
here.
|
|
2. **[gop.md](gop.md) — the Graphics Output Protocol.** How UEFI exposes graphics
|
|
modes (unlike fixed VGA modes), how we detect the monitor's native resolution
|
|
from EDID and switch to it, and the pixel formats we accept or reject.
|
|
3. **[framebuffer.md](framebuffer.md) — the framebuffer.** What the linear
|
|
framebuffer the loader hands over actually is, and what **pitch** (stride)
|
|
means versus width — the detail you have to get right to avoid a skewed image.
|
|
4. **[memory-map.md](memory-map.md) — the memory map.** How the loader learns what
|
|
physical RAM exists and hands it to the kernel in danos's own neutral format,
|
|
rather than leaking UEFI's memory descriptors across the boundary.
|
|
5. **[frame-allocator.md](frame-allocator.md) — the physical frame allocator.** The
|
|
bitmap allocator that hands out and reclaims 4 KiB physical frames from that
|
|
map — the primitive page tables and the heap are built on.
|
|
6. **[interrupts.md](interrupts.md) — interrupts and exceptions.** The GDT, IDT and
|
|
TSS, the exception stubs, and the handler that reports a CPU fault in red instead
|
|
of letting it triple-fault into a silent reset.
|
|
7. **[paging.md](paging.md) — the kernel's page tables.** Building our own 4-level
|
|
page tables, identity-mapping the low 4 GiB, and switching CR3 off the firmware's
|
|
tables onto ours.
|
|
8. **[device-interrupts.md](device-interrupts.md) — device interrupts.** The Local
|
|
APIC and its timer — the kernel's first interrupt that is *handled and returned
|
|
from*, giving it a heartbeat.
|
|
9. **[heap.md](heap.md) — the kernel heap.** A growable free-list allocator built on
|
|
the VMM, exposed as a `std.mem.Allocator` so std containers work — dynamic
|
|
allocation for the kernel.
|
|
10. **[scheduling.md](scheduling.md) — the scheduler.** Fixed-priority preemptive
|
|
multitasking: kernel threads, the context switch, O(1) priority selection, and
|
|
blocking (sleep, wait queues) — the leap to a running system.
|
|
11. **[ipc.md](ipc.md) — inter-process communication.** Bounded blocking
|
|
message-passing channels, then synchronous call/reply between *processes* over
|
|
endpoints — the backbone the microkernel's isolated servers talk over.
|
|
12. **[syscall.md](syscall.md) — system calls.** How ring 3 asks the kernel for
|
|
something: the `syscall`/`sysret` fast path, the trap frame, and why the table is
|
|
deliberately tiny. The numbers are a **private** ABI — [vdso.md](vdso.md) designs
|
|
the public boundary that will hide them.
|
|
13. **[vfs-protocol.md](vfs-protocol.md) — the VFS wire protocol.** The language-neutral
|
|
byte-level spec of the file protocol spoken over IPC: request/reply headers,
|
|
the operation table, mount routing, and the append-only evolution rules — the
|
|
first IPC protocol documented as public ABI.
|
|
14. **[drivers.md](drivers.md) — writing a driver.** The payoff: a driver is an
|
|
ordinary ring-3 process that claims a device, maps its registers, and **sleeps
|
|
until its hardware interrupts it**. The claim is the capability; `irq_ack` is the
|
|
unmask.
|
|
15. **[driver-model.md](driver-model.md) — buses, classes and host controllers.** How
|
|
real driver stacks factor into three shapes and how families share code. The
|
|
three primitives it proposed are long since built (M13 capability passing,
|
|
M14 DMA + barriers, M15 MSI), and the driver *contract* on top of them —
|
|
hello, supervision, restart — is built too (device-manager.md, M18).
|
|
16. **[process-management.md](process-management.md) — process management.** The
|
|
microkernel's `ps`/`kill`/SIGCHLD: enumerate as a table snapshot, the
|
|
supervision link as the kill authority, and child-exit notifications over the
|
|
same endpoints IRQs arrive on.
|
|
17. **[process-lifecycle.md](process-lifecycle.md) — the process lifecycle.** Built
|
|
(M17): signals over IPC as the one lifecycle vocabulary every process speaks — the
|
|
POSIX.1-1990 words with message delivery instead of stack hijack, the stable
|
|
`runtime.process` interface, exit reasons, published exit events any stateful
|
|
service can subscribe to (the VFS releasing dead clients' handles), and the two
|
|
iron rules (cleanup is the kernel's job; kill is not a signal).
|
|
18. **[device-manager.md](device-manager.md) — the device manager.** Built (M18,
|
|
through the app surface): the
|
|
tree, the matcher, and the supervisor. Tree structure lives in the manager,
|
|
authority stays in the kernel; bus drivers report what they see; drivers are
|
|
restarted through the lifecycle vocabulary — the plan that turns
|
|
[resilience.md](resilience.md)'s restart goal into increments.
|
|
19. **[input.md](input.md) — the input module.** Broadcasting input events (keyboard,
|
|
mouse, joystick): why a synchronous rendezvous can't fan out to many listeners, the
|
|
asynchronous `ipc_send` primitive built to fix it, and the per-device subscribe/publish
|
|
service layered on top.
|
|
20. **[display.md](display.md) — the display service.** The display half of the GUI
|
|
track: a user-space compositor that owns the framebuffer, composes a layer stack into
|
|
a double buffer, and presents it. Why GOP and the PCI display device are two views of
|
|
one controller, the device-node + write-combining handoff, and what flicker-free buys
|
|
that tear-free doesn't. Plan: [display-plan.md](display-plan.md). **v2** (complete) makes
|
|
scanout a pluggable backend — GOP floor + a native virtio-gpu driver, hot-attached, with
|
|
runtime mode-set, EDID, fenced vsync presents, and restart re-attach:
|
|
[display-v2.md](display-v2.md), plan [display-v2-plan.md](display-v2-plan.md). Looking
|
|
further out, two research snapshots survey what a *native* driver for real GPU silicon
|
|
would take as another `.scanout` backend: [nvidia-gpus.md](nvidia-gpus.md) (RTX 3060 /
|
|
Ampere) and [intel-igpu.md](intel-igpu.md) (Intel iGPU).
|
|
21. **[halting.md](halting.md) — halting.** Why a kernel can't just "exit", and
|
|
how `while (true) hlt` parks the CPU safely once there's nothing left to do.
|
|
|
|
Start with the north star:
|
|
|
|
- **[vision.md](vision.md) — the vision.** danos is a **learning-by-doing**
|
|
microkernel: minimal kernel, drivers/services isolated in user space, chosen for
|
|
**resilience** (restartable components). Win condition: runs on the author's PC and
|
|
both Raspberry Pis, ideally with a GUI. Real-time is an option to explore, not a
|
|
requirement. The *why* that shapes everything below.
|
|
- **[resilience.md](resilience.md) — resilience.** A design note (not built yet) on
|
|
fault isolation + live restart — the reincarnation-server + capability model that
|
|
makes "if I break it, I can restart it" real. danos's core motivation.
|
|
- **[zig-self-hosting.md](zig-self-hosting.md) — running Zig on danos.** A design note
|
|
(not built yet) on making danos a real Zig target (`-target x86_64-danos`) and
|
|
eventually running the compiler on it. The key realisation: Zig 0.16 reduces an OS
|
|
port to **one seam** (`std.os.danos`), so we build `runtime.os` (→ that seam) plus a
|
|
thin `runtime.fs`, retire the `posix` shim, and follow a phased path to
|
|
`zig build-exe hello.zig` running on danos — **not** Linux-ABI emulation.
|
|
- **[threading.md](threading.md) — threads, the std-shaped way.** A design note (not
|
|
built yet) on `runtime.Thread`: a type that mirrors `std.Thread`'s API (spawn/join,
|
|
Mutex/Condition) over a **private** thread ABI — several tasks sharing one address
|
|
space via a `thread_spawn` syscall, futex-backed blocking, aspace refcounting. Why
|
|
it's the native type and not literal `std.Thread` (the [private ABI](syscall.md)),
|
|
and why threads stay a narrow opt-in against the [resilience](resilience.md) default.
|
|
- **[vdso.md](vdso.md) — the vDSO, the public system-call boundary.** A design note
|
|
(not built yet) on keeping `abi.zig` genuinely private: a kernel-supplied, C-ABI
|
|
entry blob mapped into every process as the *only* way into the kernel — so the
|
|
syscall numbers can be renumbered or randomised at will, and Rust/C binaries get a
|
|
stable boundary without danos growing a dynamic linker. danos's public ABI = the
|
|
vDSO + the documented IPC wire protocols ([vfs-protocol.md](vfs-protocol.md) first).
|
|
|
|
Cutting across all of these:
|
|
|
|
- **[system-requirements.md](system-requirements.md) — system requirements.** The
|
|
hardware needed to run danos: minimum specs (UEFI x86-64, ACPI, PCIe ECAM,
|
|
xHCI, ~128 MiB RAM) grounded in what the boot path actually assumes, plus a
|
|
plain-language guide matching Intel/AMD CPU generations by name.
|
|
- **[release-iso.md](release-iso.md) — the release ISO.** The flashable boot
|
|
media: `zig build release-x86-64` wraps the FAT32 boot volume in a hybrid ISO
|
|
(MBR ESP partition + El Torito EFI entry, one embedded image) that Etcher/dd
|
|
flash to USB or a burner writes to disc — built by an in-repo pure-Python
|
|
tool, like the FAT image itself.
|
|
- **[arch.md](arch.md) — the architecture split.** How CPU-specific code is kept
|
|
behind a build-time `arch` module so the generic kernel never names x86_64,
|
|
leaving room for other systems (e.g. an AArch64 Raspberry Pi) later.
|
|
- **[arm.md](arm.md) — ARM targets.** The Raspberry Pi landscape the arch split is
|
|
aiming at: `arm` (32-bit, Pi Zero W) vs `aarch64` (64-bit, Pi 3-5), UEFI vs
|
|
device-tree boot, and what each layer needs.
|
|
- **[discovery.md](discovery.md) — device discovery.** A design note on learning what
|
|
hardware exists via ACPI (x86) or device tree (ARM) behind one neutral device model —
|
|
when to build it, and how to keep it architecture-agnostic.
|
|
- **[acpi.md](acpi.md) — finding the ACPI tables.** The concrete x86 locator chain:
|
|
how the loader captures the **RSDP**, hands its physical address across in `BootInfo`,
|
|
and how the platform derives the **RSDT/XSDT** from it and walks the SDTs — plus the
|
|
live event side (the SCI, the power button, GPE/Notify) the ring-3 acpi service runs.
|
|
- **[power.md](power.md) — the power service.** System power as a domain-named
|
|
service: button/lid/battery events published to subscribers, and init's orderly
|
|
shutdown composing the [lifecycle](process-lifecycle.md) stop sequence with an ACPI
|
|
S5 write. Firmware-neutral — a PSCI backend drops in on ARM.
|
|
- **[timers.md](timers.md) — timers and time.** The ring-3 surface for reading the
|
|
clock and waiting: why `now()` is a syscall rather than a service, and the one-shot
|
|
timer notification (`timer_bind`) that gives supervisors a timed wait — built on the
|
|
LAPIC heartbeat and calibrated TSC of [device-interrupts.md](device-interrupts.md).
|
|
- **[smp.md](smp.md) — multiple cores.** A design/research note on how microkernels
|
|
(L4, seL4) handle SMP — big kernel lock vs per-CPU vs multikernel — and how the
|
|
right choice depends on whether danos is chasing real-time or resilience.
|
|
- **[coding-standards.md](coding-standards.md) — coding standards.** The naming rule the
|
|
tree follows: non-acronyms are spelled out in full (`message`, not `msg`), files are
|
|
`kebab-case`, code follows Zig's case conventions, and the handful of exceptions
|
|
(POSIX/C ABI names, `init`/`len`/`ptr`, acronyms).
|
|
- **[sysv.md](sysv.md) — the calling convention.** What "the kernel is SysV" means,
|
|
and why the loader→kernel boundary has to pin it (the RDI-vs-RCX handoff).
|
|
- **[testing.md](testing.md) — testing.** How the kernel is tested by booting it in
|
|
QEMU and asserting on its serial output — reproducibly, and structured so the
|
|
same tests run across architectures.
|
|
- **[logging.md](logging.md) — logging.** The multi-sink diagnostic log (serial,
|
|
0xE9 debugcon, file later) kept separate from the framebuffer display, plus the
|
|
robustness path: optional framebuffer, POST-code checkpoints, and a persistent
|
|
panic breadcrumb so the kernel survives — and can be diagnosed — with no output.
|
|
|
|
## How the pieces relate
|
|
|
|
The boot flow ties them together: UEFI runs the loader ([efi.md](efi.md)), which
|
|
queries the **GOP** to pick a graphics mode ([gop.md](gop.md)), hands the kernel a
|
|
**framebuffer** to draw into ([framebuffer.md](framebuffer.md)) and a **memory
|
|
map** of physical RAM ([memory-map.md](memory-map.md)); the kernel turns that map
|
|
into a **frame allocator** ([frame-allocator.md](frame-allocator.md)), installs
|
|
its **descriptor tables** so CPU faults are caught ([interrupts.md](interrupts.md)),
|
|
builds its own **page tables** and switches onto them ([paging.md](paging.md)),
|
|
brings up the **heap** for dynamic allocation ([heap.md](heap.md)), starts the
|
|
**scheduler** ([scheduling.md](scheduling.md)) and the **timer** that preempts it
|
|
([device-interrupts.md](device-interrupts.md)) — with tasks blocking, sleeping and
|
|
passing messages over **[IPC](ipc.md)** channels — runs, its CPU-specific bits
|
|
behind the [arch](arch.md) boundary, and when idle, or on a panic, it **halts**
|
|
([halting.md](halting.md)).
|
|
|
|
Above that line the microkernel proper begins: **discovery** ([discovery.md](discovery.md),
|
|
[acpi.md](acpi.md)) learns what hardware exists, ring-3 processes ask the kernel for
|
|
things through the small **[syscall](syscall.md)** table, isolated servers reach each
|
|
other over IPC **endpoints** ([ipc.md](ipc.md)), and a **[driver](drivers.md)** claims
|
|
a device, maps its registers, and sleeps until the hardware interrupts it — which is
|
|
the whole reason for the arrangement ([vision.md](vision.md)).
|
|
|
|
## Repository layout
|
|
|
|
danos is a **monorepo of sub-projects**. Each service or driver is a directory that is
|
|
its own Zig module — it can hold as many files as it needs, and other sub-projects
|
|
reach it *by module name*, never by a path into its files. The source tree deliberately
|
|
**mirrors the runtime FHS** ([danos-file-system-hierarchy-FSH.md](danos-file-system-hierarchy-FSH.md)):
|
|
what you see under `system/` in the source is what a running danos represents under
|
|
`/system`.
|
|
|
|
**A sub-project is addressed by its directory; its entry point repeats the directory's
|
|
name.** `system/services/init/` contains `init.zig` (its root), and produces a binary
|
|
addressed as **`system/services/init`** — the repeated leaf resolves away:
|
|
|
|
| Source (root file) | Addressed as (module / binary / FHS path) |
|
|
|----------------------------------------|--------------------------------------------|
|
|
| `system/services/init/init.zig` | `system/services/init` → `/system/services/init` |
|
|
| `system/drivers/ps2-bus/ps2-bus.zig` | `system/drivers/ps2-bus` → `/system/drivers/ps2-bus` |
|
|
| `library/runtime/runtime.zig` | `library/runtime` (the `runtime` module) |
|
|
|
|
In **source**, a sub-project is a directory so it can hold many files — the entry is
|
|
`init/init.zig`, beside it `vfs/vfs-test.zig`, `vfs/protocol.zig`, and so on. When
|
|
**addressed or installed**, that collapses to the single canonical path: the `init`
|
|
binary installs to `/system/services/init` (a file at that path), not
|
|
`/system/services/init/init`. The repeated leaf exists only in source; the directory is
|
|
the identity, the entry file is its implementation. (Same idea as a Go package being its
|
|
directory, or a macOS `.app` bundle addressed by the bundle, not the executable within.)
|
|
A sub-project's extra files are reached through the module, never as separate paths.
|
|
|
|
```
|
|
system/ → /system danos's own internals (the self-representation)
|
|
boot-handoff.zig the loader↔kernel contract (the `boot-handoff` module)
|
|
abi.zig the private kernel↔runtime syscall ABI (the `abi` module)
|
|
parameters.zig initial-ramdisk.zig shared contracts
|
|
kernel/ IPC, memory, scheduling, the private syscall dispatch
|
|
architecture/x86_64/ the `architecture` module (never named by generic code)
|
|
devices/ the device model /system/devices reflects (+ aml/)
|
|
device-abi.zig the device wire types (the `device-abi` module)
|
|
drivers/ hpet/ bus/ one sub-project per driver → /system/drivers
|
|
services/ init/ vfs/ device-manager/ system servers → /system/services (vfs/ holds
|
|
vfs.zig, vfs-test.zig, protocol.zig)
|
|
library/ → /lib libraries, one sub-directory each
|
|
runtime/ the danos-native runtime + file API (fs) — the stable application ABI
|
|
boot/ → /boot the loaders
|
|
tools/ test/ host-side build + QEMU test harness
|
|
```
|
|
|
|
A sub-project exposes its **public interface as a module**: `system/services/vfs/` owns
|
|
the VFS wire protocol (`protocol.zig`, the `vfs-protocol` module), which the runtime's
|
|
file API (`runtime.fs`) imports by name. `usb`/`block` drivers expose their protocols the
|
|
same way.
|
|
|
|
There is **no POSIX/C compatibility layer today**: danos programs do file I/O through the
|
|
danos-native `runtime.fs` (open/read/write/list over the VFS). A hand-rolled POSIX shim
|
|
(`library/posix/`) was retired as premature — the real POSIX/C surface will come later
|
|
from the `std.os.danos` seam (and, eventually, musl) when danos becomes a Zig target (see
|
|
[zig-self-hosting.md](zig-self-hosting.md)). When it does, the foreign-ABI naming
|
|
exception in [coding-standards.md](coding-standards.md) applies to that seam.
|
|
|
|
## Source map
|
|
|
|
| Area | Code |
|
|
|------|------|
|
|
| Boot methods (one per way of booting the kernel) | `boot/` — `efi.zig` (UEFI) → `BOOTX64.efi` |
|
|
| Kernel entry, panic, bring-up | `system/kernel/kernel.zig` |
|
|
| Loader↔kernel handoff (`BootInfo`, `Framebuffer`, `MemoryMap`, VM layout) | `system/boot-handoff.zig` |
|
|
| Private kernel↔runtime syscall ABI (`SystemCall`, mmap prot flags, `page_size`) — the runtime speaks it, not apps | `system/abi.zig` |
|
|
| Device wire types (`DeviceDescriptor`, `DeviceClass`, …) | `system/devices/device-abi.zig` |
|
|
| Physical frame allocator | `system/kernel/pmm.zig` |
|
|
| Kernel heap (`std.mem.Allocator`) | `system/kernel/heap.zig` |
|
|
| Scheduler (fixed-priority preemptive; blocking, wait queues) | `system/kernel/scheduler.zig` |
|
|
| Big kernel lock + interrupt-safe critical sections | `system/kernel/sync.zig` |
|
|
| IPC channels between kernel threads (message passing) | `system/kernel/ipc.zig` |
|
|
| IPC endpoints: cross-address-space call/reply, handles, notifications | `system/kernel/ipc-synchronous.zig` |
|
|
| User processes: ELF loading, address spaces, the syscall table | `system/kernel/process.zig` |
|
|
| Device tree + claim capability + `device_register` containment | `system/kernel/devices-broker.zig` |
|
|
| IRQ-as-IPC: routing a device interrupt to a driver's endpoint | `system/kernel/irq.zig` |
|
|
| Hardware discovery (ACPI/device tree) behind one neutral device model | `system/devices/` |
|
|
| Framebuffer text console (mirrors to serial) | `system/kernel/console.zig` |
|
|
| In-kernel test cases | `system/kernel/tests.zig` |
|
|
| Arch-specific kernel code (`halt`, GDT/IDT/TSS, exception + interrupt stubs, page tables, APIC/IO-APIC/timer, serial, linker script) | `system/kernel/architecture/x86_64/` |
|
|
| danos-native runtime (`runtime`): syscall wrappers, heap, IPC, device access, the file API (`fs`) — the stable application ABI | `library/runtime/` |
|
|
| System services (init, the VFS server + `protocol`, the device-manager) | `system/services/` |
|
|
| Device drivers, one sub-project each (`pci-bus`, `ps2-bus`, `usb-xhci-bus` bus drivers) | `system/drivers/` |
|
|
| Build + `run-x86-64` (QEMU/OVMF) + `release-x86-64` (the flashable ISO) | `build.zig` |
|
|
| QEMU integration test harness | `test/qemu_test.py` |
|