danos/docs/testing.md

Testing

danos is a freestanding kernel — it can't be unit-tested like a normal library, because most of what it does only means anything on a booted CPU. So the main test strategy is boot it in QEMU and assert on what it does, reproducibly and without a human staring at the screen.

There are two layers:

• Host unit tests (zig build test) — for pure, platform-independent logic in the shared contracts (system/boot-handoff.zig, system/abi.zig, system/devices/device-abi.zig), which also compile-checks the three-way split stays self-consistent. These compile for the host and run natively.
• QEMU integration tests (python3 test/qemu_test.py) — boot the real kernel and check its behaviour. This is the interesting part.

The key enabler: serial output

The framebuffer console draws pixels, which a test can't read without screen-scraping. So the kernel also writes everything to a serial port (system/kernel/architecture/x86_64/serial.zig, a 16550 UART on COM1). Console.write mirrors every byte to it, so all kernel output — boot log, memory summary, exception reports — appears on serial as plain text.

QEMU captures that with -serial file:serial.log, giving a machine-readable transcript. Serial is per-architecture (x86 uses port I/O; an ARM board uses a memory-mapped UART), so it lives behind the arch boundary — and adding a new architecture's UART is what makes the same tests run there.

The serial log sink is compiled in only under -Dserial (off by default). A real machine often has no live legacy COM1 — writing to a dead one is slow — and the boot log is kept in a RAM buffer (klog) and flushed to disk instead, so serial is now purely a QEMU/dev aid. The harness (test/qemu_test.py) builds every case with -Dserial=true, and zig build run-x86-64 boots a serial-enabled image variant, so both get the transcript; a flashable zig build image leaves serial out. (Even with -Dserial, a loopback probe disables a dead port at boot, so a serial-enabled image is still safe on real hardware.)

In-kernel test cases

Building with -Dtest-case=<name> makes the kernel, after normal bring-up, run one self-test from system/kernel/tests.zig instead of idling. Each case writes structured markers to serial:

DANOS-TEST-BEGIN: smoke
[PASS] memory map reports usable RAM
[PASS] alloc returns distinct frames
...
DANOS-TEST-RESULT: PASS (6 passed, 0 failed)
DANOS-TEST-DONE

Current cases:

Case	What it checks	How the harness confirms it
smoke	memory map has usable RAM; frame alloc/free; paging active	DANOS-TEST-RESULT: PASS
timer	device interrupts fire and return (tick count advances)	DANOS-TEST-RESULT: PASS
clock	LAPIC + TSC calibrated; monotonic uptime advances; nanos() has sub-ms resolution	DANOS-TEST-RESULT: PASS
vmm	on-demand map works: a mapped page is writable and reads back	DANOS-TEST-RESULT: PASS
heap	kernel heap: alloc/free, block reuse, growth, and a std container on it	DANOS-TEST-RESULT: PASS
sched	preemption: three non-yielding tasks all make progress	DANOS-TEST-RESULT: PASS
priority	fixed-priority tasks run highest-first	DANOS-TEST-RESULT: PASS
sleep	a task blocks for ~50 ms (real block, not a busy-wait)	DANOS-TEST-RESULT: PASS
event	a task blocks on a wait queue and is woken (preempting)	DANOS-TEST-RESULT: PASS
ipc	producer/consumer pass 100 messages through a 4-slot channel intact	DANOS-TEST-RESULT: PASS
fault-ud	invalid-opcode exception is caught	serial shows invalid opcode (vector 6)
fault-pf	page fault caught with CR2	page fault (vector 14)
fault-df	double fault caught on IST1 (not a triple-fault reset)	double fault (vector 8)
fault-nx	executing a data page (NX) faults	page fault (vector 14)
fault-null	dereferencing the unmapped page 0 faults	page fault (vector 14)

The faulting cases don't print a result line — they deliberately raise a CPU exception, and the harness asserts on the exception report the handler prints (which also reaches serial). This reuses the real fault path as the test oracle: if the IDT/TSS weren't wired up, fault-df would triple-fault and the marker would never appear.

The harness

test/qemu_test.py ties it together. For each case it:

1. builds the kernel with -Dtest-case=<name>,
2. assembles a fresh EFI System Partition from the built binaries,
3. boots it headless in QEMU with serial captured to a file and -no-reboot (so a triple fault exits rather than looping),
4. polls the serial log until the case's expected regex appears (pass), a failure marker appears, or a timeout elapses (fail),
5. kills QEMU and moves on.

$ python3 test/qemu_test.py
danos qemu tests  arch=x86_64  cases=4

  smoke        ... PASS  (matched 'DANOS-TEST-RESULT: PASS')
  fault-ud     ... PASS  (matched 'invalid opcode \(vector 6\)')
  fault-pf     ... PASS  (matched 'page fault \(vector 14\)')
  fault-df     ... PASS  (matched 'double fault \(vector 8\)')

4/4 passed

It exits non-zero if any case fails, so it drops straight into CI. Run a subset with python3 test/qemu_test.py smoke fault-pf.

(It's a standalone script rather than a zig build step on purpose: a build step that shells out to a harness which itself runs zig build would contend on the build cache lock.)

Built for multiple architectures

The runner separates what is tested (the cases and their expected markers) from how a given CPU is built and booted (the ARCHES table: the QEMU binary, firmware, boot method, serial device). The cases are architecture-neutral — "a page fault is reported", not "this x86 encoding faults".

So bringing up a second architecture — an AArch64 Raspberry Pi is the motivating one — means:

1. implement system/kernel/arch/aarch64/ (CPU ops, its UART, exception vectors, page tables) behind the same arch interface,
2. add an aarch64 entry to ARCHES with its qemu-system-aarch64 invocation,

and the same smoke / fault-* cases run against it: python3 test/qemu_test.py --arch aarch64. A green suite on both is the definition of "it works across architectures".

Writing a new case

1. Add a function to system/kernel/tests.zig and dispatch it in run on its name.
2. Emit [PASS]/[FAIL] lines and a DANOS-TEST-RESULT: line (non-faulting cases), or trigger the condition and rely on the handler's output (faulting cases).
3. Add an entry to CASES in test/qemu_test.py with the regex that proves it.