danos/docs/testing.md

# Testing

danos is a freestanding kernel — it can't be unit-tested like a normal library,
because most of what it does only means anything on a booted CPU. So the main test
strategy is **boot it in QEMU and assert on what it does**, reproducibly and
without a human staring at the screen.

There are two layers:

- **Host unit tests** (`zig build test`) — for pure, platform-independent logic.
  What began as the three shared contracts (`system/boot-handoff.zig`,
  `system/abi.zig`, `library/device/model/device-abi.zig`) now spans ~26 modules:
  protocol and on-wire definitions (VFS, USB, virtio-gpu), the FAT engine, the
  display compositor, PS/2 and HID decoding, the kernel log ring, and the
  runtime's `time`/`thread` — the list is distributed across the library-domain
  and binary packages' own `test` steps, which the root `zig build test`
  aggregates (docs/build-packages-plan.md).
  These compile for the host and run natively.
- **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel
  and check its behaviour. This is the interesting part.

## The key enabler: serial output

The framebuffer console draws pixels, which a test can't read without
screen-scraping. So the kernel also writes everything to a **serial port**
(`system/kernel/architecture/x86_64/serial.zig`, a 16550 UART on COM1). The diagnostic log fans out
to registered sinks, and the serial UART is one of them, so all kernel output —
boot log, memory summary, exception reports — appears on serial as plain text.

QEMU captures that with `-serial file:serial.log`, giving a machine-readable
transcript. Serial is per-architecture (x86 uses port I/O; an ARM board uses a
memory-mapped UART), so it lives behind the [architecture](os-development/architecture.md) boundary — and adding
a new architecture's UART is what makes the same tests run there.

The serial log sink is **compiled in only under `-Dserial`** (off by default).
A real machine often has no live legacy COM1 — writing to a dead one is slow —
and the boot log is kept in a RAM buffer (`klog`) and flushed to disk instead,
so serial is now purely a QEMU/dev aid. The harness (`test/qemu_test.py`) builds
every case with `-Dserial=true`, and `zig build run-x86-64` boots a serial-enabled
image variant, so both get the transcript; a flashable `zig build` image leaves
serial out. (Even with `-Dserial`, a loopback probe disables a dead port at boot,
so a serial-enabled image is still safe on real hardware.)

## In-kernel test cases

Building with `-Dtest-case=<name>` makes the kernel, after normal bring-up, run one
self-test from `system/kernel/tests.zig` instead of idling. Each case writes structured
markers to serial:

```
DANOS-TEST-BEGIN: smoke
[PASS] memory map reports usable RAM
[PASS] alloc returns distinct frames
...
DANOS-TEST-RESULT: PASS (6 passed, 0 failed)
DANOS-TEST-DONE
```

The core kernel cases (the suite has since grown far beyond this table — SMP,
threads, processes, display, USB, FAT and more; the full list is the `CASES`
table in `test/qemu_test.py`):

| Case | What it checks | How the harness confirms it |
|------|----------------|-----------------------------|
| `smoke` | memory map has usable RAM; frame alloc/free; paging active | `DANOS-TEST-RESULT: PASS` |
| `discovery` | ACPI discovery populated the platform facts: MADT (LAPIC base, CPU count) and FADT (PM/reset registers) | `DANOS-TEST-RESULT: PASS` |
| `wx` | W^X audit: kernel code is executable; rodata, data, heap, and stack are NX | `DANOS-TEST-RESULT: PASS` |
| `timer` | device interrupts fire and return (tick count advances) | `DANOS-TEST-RESULT: PASS` |
| `clock` | LAPIC + TSC calibrated; monotonic uptime advances; `nanos()` has sub-ms resolution | `DANOS-TEST-RESULT: PASS` |
| `wall-clock` | the CMOS RTC read at boot yields a plausible current epoch | `DANOS-TEST-RESULT: PASS` |
| `vmm` | on-demand `map` works: a mapped page is writable and reads back | `DANOS-TEST-RESULT: PASS` |
| `heap` | kernel heap: alloc/free, block reuse, growth, and a std container on it | `DANOS-TEST-RESULT: PASS` |
| `sched` | preemption: three non-yielding tasks all make progress | `DANOS-TEST-RESULT: PASS` |
| `priority` | fixed-priority tasks run highest-first | `DANOS-TEST-RESULT: PASS` |
| `sleep` | a task blocks for ~50 ms (real block, not a busy-wait) | `DANOS-TEST-RESULT: PASS` |
| `event` | a task blocks on a wait queue and is woken (preempting) | `DANOS-TEST-RESULT: PASS` |
| `ipc` | producer/consumer pass 100 messages through a 4-slot channel intact | `DANOS-TEST-RESULT: PASS` |
| `ipc-call` | synchronous IPC: client and server ping-pong 100 calls through one endpoint (rendezvous, reply routing, cross-copy) | `DANOS-TEST-RESULT: PASS` |
| `ipc-cap` | capability passing: endpoints handed over in a call and its reply arrive as the same object, shared not moved | `DANOS-TEST-RESULT: PASS` |
| `dma` | DMA memory: contiguous frame allocation, below-4G cap, coherent mapping, reclaim on teardown | `DANOS-TEST-RESULT: PASS` |
| `msi` | an MSI vector is allocated and delivered as an endpoint notification (a self-IPI stands in for the device write) | `DANOS-TEST-RESULT: PASS` |
| `iommu` | the VT-d unit is found in the DMAR table and its registers read back (detection only; boots with an emulated IOMMU) | `DANOS-TEST-RESULT: PASS` |
| `ioport` | port I/O grants: an `io_port` resource admits in-range reads, refuses out-of-range/unclaimed | `DANOS-TEST-RESULT: PASS` |
| `fault-ud` | invalid-opcode exception is caught | serial shows `invalid opcode (vector 6)` |
| `fault-pf` | page fault caught with CR2 | `page fault (vector 14)` |
| `fault-df` | double fault caught on IST1 (not a triple-fault reset) | `double fault (vector 8)` |
| `fault-ap-df` | a double fault pinned to an application processor is caught by that core's own TSS/IST (boots with `-smp 4`) | `core N: double fault (vector 8)`, N ≥ 1 |
| `fault-nx` | executing a data page (NX) faults | `page fault (vector 14)` |
| `fault-null` | dereferencing the unmapped page 0 faults | `page fault (vector 14)` |
| `fault-recovery` | a ring-3 process that faults is killed and reaped while init keeps heartbeating — the OS survives | `DANOS-TEST-RESULT: PASS` |

The faulting cases don't print a result line — they deliberately raise a CPU
exception, and the harness asserts on the [exception report](os-development/interrupts.md) the
handler prints (which also reaches serial). This reuses the real fault path as the
test oracle: if the IDT/TSS weren't wired up, `fault-df` would triple-fault and the
marker would never appear.

## The harness

`test/qemu_test.py` ties it together. For each case it:

1. builds the kernel with `-Dtest-case=<name>` (the build produces the bootable
   FAT32 USB image, `zig-out/danos-usb.img`),
2. copies that image to a fresh per-run boot volume, so the guest's mutations
   don't dirty the build artifact,
3. boots it headless in QEMU with serial captured to a file and `-no-reboot`
   (so a triple fault exits rather than looping),
4. polls the serial log until the case's expected regex appears (**pass**), a
   failure marker appears, or a timeout elapses (**fail**),
5. kills QEMU and moves on.

```
$ python3 test/qemu_test.py
danos qemu tests  arch=x86_64  cases=4

  smoke        ... PASS  (matched 'DANOS-TEST-RESULT: PASS')
  fault-ud     ... PASS  (matched 'invalid opcode \(vector 6\)')
  fault-pf     ... PASS  (matched 'page fault \(vector 14\)')
  fault-df     ... PASS  (matched 'double fault \(vector 8\)')

4/4 passed
```

It exits non-zero if any case fails, so it drops straight into CI. Run a subset
with `python3 test/qemu_test.py smoke fault-pf`.

(It's a standalone script rather than a `zig build` step on purpose: a build step
that shells out to a harness which itself runs `zig build` would contend on the
build cache lock.)

## Built for multiple architectures

The runner separates *what* is tested (the cases and their expected markers) from
*how a given CPU is built and booted* (the `ARCHES` table: the QEMU binary,
firmware, boot method, serial device). The cases are architecture-neutral —
"a page fault is reported", not "this x86 encoding faults".

So bringing up a second architecture — an AArch64 Raspberry Pi is the motivating
one — means:

1. implement `system/kernel/architecture/aarch64/` (CPU ops, its UART, exception vectors, page
   tables) behind the same `architecture` interface,
2. add an `aarch64` entry to `ARCHES` with its `qemu-system-aarch64` invocation,

and the *same* `smoke` / `fault-*` cases run against it: `python3 test/qemu_test.py
--arch aarch64`. A green suite on both is the definition of "it works across
architectures".

## Writing a new case

1. Add a function to `system/kernel/tests.zig` and dispatch it in `run` on its name.
2. Emit `[PASS]/[FAIL]` lines and a `DANOS-TEST-RESULT:` line (non-faulting cases),
   or trigger the condition and rely on the handler's output (faulting cases).
3. Add an entry to `CASES` in `test/qemu_test.py` with the regex that proves it.