danos/docs/testing.md

# Testing

danos is a freestanding kernel — it can't be unit-tested like a normal library,
because most of what it does only means anything on a booted CPU. So the main test
strategy is **boot it in QEMU and assert on what it does**, reproducibly and
without a human staring at the screen.

There are two layers:

- **Host unit tests** (`zig build test`) — for pure, platform-independent logic in
  the shared contracts (`system/boot-handoff.zig`, `system/abi.zig`,
  `system/devices/device-abi.zig`), which also compile-checks the three-way split
  stays self-consistent. These compile for the host and run natively.
- **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel
  and check its behaviour. This is the interesting part.

## The key enabler: serial output

The framebuffer console draws pixels, which a test can't read without
screen-scraping. So the kernel also writes everything to a **serial port**
(`system/kernel/architecture/x86_64/serial.zig`, a 16550 UART on COM1). `Console.write` mirrors every
byte to it, so all kernel output — boot log, memory summary, exception reports —
appears on serial as plain text.

QEMU captures that with `-serial file:serial.log`, giving a machine-readable
transcript. Serial is per-architecture (x86 uses port I/O; an ARM board uses a
memory-mapped UART), so it lives behind the [arch](arch.md) boundary — and adding
a new architecture's UART is what makes the same tests run there.

## In-kernel test cases

Building with `-Dtest-case=<name>` makes the kernel, after normal bring-up, run one
self-test from `system/kernel/tests.zig` instead of idling. Each case writes structured
markers to serial:

```
DANOS-TEST-BEGIN: smoke
[PASS] memory map reports usable RAM
[PASS] alloc returns distinct frames
...
DANOS-TEST-RESULT: PASS (6 passed, 0 failed)
DANOS-TEST-DONE
```

Current cases:

| Case | What it checks | How the harness confirms it |
|------|----------------|-----------------------------|
| `smoke` | memory map has usable RAM; frame alloc/free; paging active | `DANOS-TEST-RESULT: PASS` |
| `timer` | device interrupts fire and return (tick count advances) | `DANOS-TEST-RESULT: PASS` |
| `clock` | LAPIC + TSC calibrated; monotonic uptime advances; `nanos()` has sub-ms resolution | `DANOS-TEST-RESULT: PASS` |
| `vmm` | on-demand `map` works: a mapped page is writable and reads back | `DANOS-TEST-RESULT: PASS` |
| `heap` | kernel heap: alloc/free, block reuse, growth, and a std container on it | `DANOS-TEST-RESULT: PASS` |
| `sched` | preemption: three non-yielding tasks all make progress | `DANOS-TEST-RESULT: PASS` |
| `priority` | fixed-priority tasks run highest-first | `DANOS-TEST-RESULT: PASS` |
| `sleep` | a task blocks for ~50 ms (real block, not a busy-wait) | `DANOS-TEST-RESULT: PASS` |
| `event` | a task blocks on a wait queue and is woken (preempting) | `DANOS-TEST-RESULT: PASS` |
| `ipc` | producer/consumer pass 100 messages through a 4-slot channel intact | `DANOS-TEST-RESULT: PASS` |
| `fault-ud` | invalid-opcode exception is caught | serial shows `invalid opcode (vector 6)` |
| `fault-pf` | page fault caught with CR2 | `page fault (vector 14)` |
| `fault-df` | double fault caught on IST1 (not a triple-fault reset) | `double fault (vector 8)` |
| `fault-nx` | executing a data page (NX) faults | `page fault (vector 14)` |
| `fault-null` | dereferencing the unmapped page 0 faults | `page fault (vector 14)` |

The faulting cases don't print a result line — they deliberately raise a CPU
exception, and the harness asserts on the [exception report](interrupts.md) the
handler prints (which also reaches serial). This reuses the real fault path as the
test oracle: if the IDT/TSS weren't wired up, `fault-df` would triple-fault and the
marker would never appear.

## The harness

`test/qemu_test.py` ties it together. For each case it:

1. builds the kernel with `-Dtest-case=<name>`,
2. assembles a fresh EFI System Partition from the built binaries,
3. boots it headless in QEMU with serial captured to a file and `-no-reboot`
   (so a triple fault exits rather than looping),
4. polls the serial log until the case's expected regex appears (**pass**), a
   failure marker appears, or a timeout elapses (**fail**),
5. kills QEMU and moves on.

```
$ python3 test/qemu_test.py
danos qemu tests  arch=x86_64  cases=4

  smoke        ... PASS  (matched 'DANOS-TEST-RESULT: PASS')
  fault-ud     ... PASS  (matched 'invalid opcode \(vector 6\)')
  fault-pf     ... PASS  (matched 'page fault \(vector 14\)')
  fault-df     ... PASS  (matched 'double fault \(vector 8\)')

4/4 passed
```

It exits non-zero if any case fails, so it drops straight into CI. Run a subset
with `python3 test/qemu_test.py smoke fault-pf`.

(It's a standalone script rather than a `zig build` step on purpose: a build step
that shells out to a harness which itself runs `zig build` would contend on the
build cache lock.)

## Built for multiple architectures

The runner separates *what* is tested (the cases and their expected markers) from
*how a given CPU is built and booted* (the `ARCHES` table: the QEMU binary,
firmware, boot method, serial device). The cases are architecture-neutral —
"a page fault is reported", not "this x86 encoding faults".

So bringing up a second architecture — an AArch64 Raspberry Pi is the motivating
one — means:

1. implement `system/kernel/arch/aarch64/` (CPU ops, its UART, exception vectors, page
   tables) behind the same `arch` interface,
2. add an `aarch64` entry to `ARCHES` with its `qemu-system-aarch64` invocation,

and the *same* `smoke` / `fault-*` cases run against it: `python3 test/qemu_test.py
--arch aarch64`. A green suite on both is the definition of "it works across
architectures".

## Writing a new case

1. Add a function to `system/kernel/tests.zig` and dispatch it in `run` on its name.
2. Emit `[PASS]/[FAIL]` lines and a `DANOS-TEST-RESULT:` line (non-faulting cases),
   or trigger the condition and rely on the handler's output (faulting cases).
3. Add an entry to `CASES` in `test/qemu_test.py` with the regex that proves it.