Cross-architecture test suite

docs/testing.md

@@ -0,0 +1,113 @@
+# Testing
+
+danos is a freestanding kernel — it can't be unit-tested like a normal library,
+because most of what it does only means anything on a booted CPU. So the main test
+strategy is **boot it in QEMU and assert on what it does**, reproducibly and
+without a human staring at the screen.
+
+There are two layers:
+
+- **Host unit tests** (`zig build test`) — for pure, platform-independent logic in
+  the shared `danos` module (the handoff layout in `src/root.zig`). These compile
+  for the host and run natively.
+- **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel
+  and check its behaviour. This is the interesting part.
+
+## The key enabler: serial output
+
+The framebuffer console draws pixels, which a test can't read without
+screen-scraping. So the kernel also writes everything to a **serial port**
+(`src/arch/x86_64/serial.zig`, a 16550 UART on COM1). `Console.write` mirrors every
+byte to it, so all kernel output — boot log, memory summary, exception reports —
+appears on serial as plain text.
+
+QEMU captures that with `-serial file:serial.log`, giving a machine-readable
+transcript. Serial is per-architecture (x86 uses port I/O; an ARM board uses a
+memory-mapped UART), so it lives behind the [arch](arch.md) boundary — and adding
+a new architecture's UART is what makes the same tests run there.
+
+## In-kernel test cases
+
+Building with `-Dtest-case=<name>` makes the kernel, after normal bring-up, run one
+self-test from `src/tests.zig` instead of idling. Each case writes structured
+markers to serial:
+
+```
+DANOS-TEST-BEGIN: smoke
+[PASS] memory map reports usable RAM
+[PASS] alloc returns distinct frames
+...
+DANOS-TEST-RESULT: PASS (6 passed, 0 failed)
+DANOS-TEST-DONE
+```
+
+Current cases:
+
+| Case | What it checks | How the harness confirms it |
+|------|----------------|-----------------------------|
+| `smoke` | memory map has usable RAM; frame alloc/free; paging active | `DANOS-TEST-RESULT: PASS` |
+| `fault-ud` | invalid-opcode exception is caught | serial shows `invalid opcode (vector 6)` |
+| `fault-pf` | page fault caught with CR2 | `page fault (vector 14)` |
+| `fault-df` | double fault caught on IST1 (not a triple-fault reset) | `double fault (vector 8)` |
+
+The faulting cases don't print a result line — they deliberately raise a CPU
+exception, and the harness asserts on the [exception report](interrupts.md) the
+handler prints (which also reaches serial). This reuses the real fault path as the
+test oracle: if the IDT/TSS weren't wired up, `fault-df` would triple-fault and the
+marker would never appear.
+
+## The harness
+
+`test/qemu_test.py` ties it together. For each case it:
+
+1. builds the kernel with `-Dtest-case=<name>`,
+2. assembles a fresh EFI System Partition from the built binaries,
+3. boots it headless in QEMU with serial captured to a file and `-no-reboot`
+   (so a triple fault exits rather than looping),
+4. polls the serial log until the case's expected regex appears (**pass**), a
+   failure marker appears, or a timeout elapses (**fail**),
+5. kills QEMU and moves on.
+
+```
+$ python3 test/qemu_test.py
+danos qemu tests  arch=x86_64  cases=4
+
+  smoke        ... PASS  (matched 'DANOS-TEST-RESULT: PASS')
+  fault-ud     ... PASS  (matched 'invalid opcode \(vector 6\)')
+  fault-pf     ... PASS  (matched 'page fault \(vector 14\)')
+  fault-df     ... PASS  (matched 'double fault \(vector 8\)')
+
+4/4 passed
+```
+
+It exits non-zero if any case fails, so it drops straight into CI. Run a subset
+with `python3 test/qemu_test.py smoke fault-pf`.
+
+(It's a standalone script rather than a `zig build` step on purpose: a build step
+that shells out to a harness which itself runs `zig build` would contend on the
+build cache lock.)
+
+## Built for multiple architectures
+
+The runner separates *what* is tested (the cases and their expected markers) from
+*how a given CPU is built and booted* (the `ARCHES` table: the QEMU binary,
+firmware, boot method, serial device). The cases are architecture-neutral —
+"a page fault is reported", not "this x86 encoding faults".
+
+So bringing up a second architecture — an AArch64 Raspberry Pi is the motivating
+one — means:
+
+1. implement `src/arch/aarch64/` (CPU ops, its UART, exception vectors, page
+   tables) behind the same `arch` interface,
+2. add an `aarch64` entry to `ARCHES` with its `qemu-system-aarch64` invocation,
+
+and the *same* `smoke` / `fault-*` cases run against it: `python3 test/qemu_test.py
+--arch aarch64`. A green suite on both is the definition of "it works across
+architectures".
+
+## Writing a new case
+
+1. Add a function to `src/tests.zig` and dispatch it in `run` on its name.
+2. Emit `[PASS]/[FAIL]` lines and a `DANOS-TEST-RESULT:` line (non-faulting cases),
+   or trigger the condition and rely on the handler's output (faulting cases).
+3. Add an entry to `CASES` in `test/qemu_test.py` with the regex that proves it.