Daniel Samson
8754d4e46a
The source layout now mirrors the runtime filesystem hierarchy
(docs/danos-file-system-hierarchy-FSH.md): what lives under system/ in the
source is what a running danos represents under /system. Each service and
driver is a sub-project directory that is its own Zig module — cross-project
references go by module name, never by a path into another project's files.
Moves (all git mv, history preserved):
- src/ -> system/ (danos internals; the self-representation)
root.zig -> danos.zig (the kernel<->user contract module)
kernel/arch/ -> kernel/architecture/ (arch -> architecture)
device/ -> devices/ (what /system/devices reflects)
boot/ -> /boot (the loaders, top level)
- sbin/ -> split by role:
init, vfs -> system/services/<name>/<name>.zig
hpetd, busd -> system/drivers/<name>/<name>.zig
vfs-test -> system/services/vfs/vfs-test.zig (inside the vfs project)
- lib/ -> library/runtime/ (room for other libraries beside runtime)
The VFS wire protocol becomes its own module, system/services/vfs/protocol.zig
("vfs-protocol"): the vfs sub-project exposes its interface, and the runtime's
file layer imports it by name. First instance of the "protocol module" pattern
(docs/driver-model.md); usb/block will expose theirs the same way.
Also: fix a naming-standard violation in the protocol — Op -> Operation (and
req -> request, _pad -> _padding). Docs updated: /system/services added to the
FHS doc, a repository-layout section added to the docs index, and stale source
paths swept across comments and docs.
Runtime boot paths are unchanged (the bootloader still loads /sbin/init);
aligning the runtime filesystem to the FHS is a separate follow-up. Suite 35/35
plus host tests green.
125 lines
5.9 KiB
Markdown
125 lines
5.9 KiB
Markdown
# Testing
|
|
|
|
danos is a freestanding kernel — it can't be unit-tested like a normal library,
|
|
because most of what it does only means anything on a booted CPU. So the main test
|
|
strategy is **boot it in QEMU and assert on what it does**, reproducibly and
|
|
without a human staring at the screen.
|
|
|
|
There are two layers:
|
|
|
|
- **Host unit tests** (`zig build test`) — for pure, platform-independent logic in
|
|
the shared `danos` module (the handoff layout in `system/danos.zig`). These compile
|
|
for the host and run natively.
|
|
- **QEMU integration tests** (`python3 test/qemu_test.py`) — boot the real kernel
|
|
and check its behaviour. This is the interesting part.
|
|
|
|
## The key enabler: serial output
|
|
|
|
The framebuffer console draws pixels, which a test can't read without
|
|
screen-scraping. So the kernel also writes everything to a **serial port**
|
|
(`system/kernel/architecture/x86_64/serial.zig`, a 16550 UART on COM1). `Console.write` mirrors every
|
|
byte to it, so all kernel output — boot log, memory summary, exception reports —
|
|
appears on serial as plain text.
|
|
|
|
QEMU captures that with `-serial file:serial.log`, giving a machine-readable
|
|
transcript. Serial is per-architecture (x86 uses port I/O; an ARM board uses a
|
|
memory-mapped UART), so it lives behind the [arch](arch.md) boundary — and adding
|
|
a new architecture's UART is what makes the same tests run there.
|
|
|
|
## In-kernel test cases
|
|
|
|
Building with `-Dtest-case=<name>` makes the kernel, after normal bring-up, run one
|
|
self-test from `system/kernel/tests.zig` instead of idling. Each case writes structured
|
|
markers to serial:
|
|
|
|
```
|
|
DANOS-TEST-BEGIN: smoke
|
|
[PASS] memory map reports usable RAM
|
|
[PASS] alloc returns distinct frames
|
|
...
|
|
DANOS-TEST-RESULT: PASS (6 passed, 0 failed)
|
|
DANOS-TEST-DONE
|
|
```
|
|
|
|
Current cases:
|
|
|
|
| Case | What it checks | How the harness confirms it |
|
|
|------|----------------|-----------------------------|
|
|
| `smoke` | memory map has usable RAM; frame alloc/free; paging active | `DANOS-TEST-RESULT: PASS` |
|
|
| `timer` | device interrupts fire and return (tick count advances) | `DANOS-TEST-RESULT: PASS` |
|
|
| `clock` | LAPIC + TSC calibrated; monotonic uptime advances; `nanos()` has sub-ms resolution | `DANOS-TEST-RESULT: PASS` |
|
|
| `vmm` | on-demand `map` works: a mapped page is writable and reads back | `DANOS-TEST-RESULT: PASS` |
|
|
| `heap` | kernel heap: alloc/free, block reuse, growth, and a std container on it | `DANOS-TEST-RESULT: PASS` |
|
|
| `sched` | preemption: three non-yielding tasks all make progress | `DANOS-TEST-RESULT: PASS` |
|
|
| `priority` | fixed-priority tasks run highest-first | `DANOS-TEST-RESULT: PASS` |
|
|
| `sleep` | a task blocks for ~50 ms (real block, not a busy-wait) | `DANOS-TEST-RESULT: PASS` |
|
|
| `event` | a task blocks on a wait queue and is woken (preempting) | `DANOS-TEST-RESULT: PASS` |
|
|
| `ipc` | producer/consumer pass 100 messages through a 4-slot channel intact | `DANOS-TEST-RESULT: PASS` |
|
|
| `fault-ud` | invalid-opcode exception is caught | serial shows `invalid opcode (vector 6)` |
|
|
| `fault-pf` | page fault caught with CR2 | `page fault (vector 14)` |
|
|
| `fault-df` | double fault caught on IST1 (not a triple-fault reset) | `double fault (vector 8)` |
|
|
| `fault-nx` | executing a data page (NX) faults | `page fault (vector 14)` |
|
|
| `fault-null` | dereferencing the unmapped page 0 faults | `page fault (vector 14)` |
|
|
|
|
The faulting cases don't print a result line — they deliberately raise a CPU
|
|
exception, and the harness asserts on the [exception report](interrupts.md) the
|
|
handler prints (which also reaches serial). This reuses the real fault path as the
|
|
test oracle: if the IDT/TSS weren't wired up, `fault-df` would triple-fault and the
|
|
marker would never appear.
|
|
|
|
## The harness
|
|
|
|
`test/qemu_test.py` ties it together. For each case it:
|
|
|
|
1. builds the kernel with `-Dtest-case=<name>`,
|
|
2. assembles a fresh EFI System Partition from the built binaries,
|
|
3. boots it headless in QEMU with serial captured to a file and `-no-reboot`
|
|
(so a triple fault exits rather than looping),
|
|
4. polls the serial log until the case's expected regex appears (**pass**), a
|
|
failure marker appears, or a timeout elapses (**fail**),
|
|
5. kills QEMU and moves on.
|
|
|
|
```
|
|
$ python3 test/qemu_test.py
|
|
danos qemu tests arch=x86_64 cases=4
|
|
|
|
smoke ... PASS (matched 'DANOS-TEST-RESULT: PASS')
|
|
fault-ud ... PASS (matched 'invalid opcode \(vector 6\)')
|
|
fault-pf ... PASS (matched 'page fault \(vector 14\)')
|
|
fault-df ... PASS (matched 'double fault \(vector 8\)')
|
|
|
|
4/4 passed
|
|
```
|
|
|
|
It exits non-zero if any case fails, so it drops straight into CI. Run a subset
|
|
with `python3 test/qemu_test.py smoke fault-pf`.
|
|
|
|
(It's a standalone script rather than a `zig build` step on purpose: a build step
|
|
that shells out to a harness which itself runs `zig build` would contend on the
|
|
build cache lock.)
|
|
|
|
## Built for multiple architectures
|
|
|
|
The runner separates *what* is tested (the cases and their expected markers) from
|
|
*how a given CPU is built and booted* (the `ARCHES` table: the QEMU binary,
|
|
firmware, boot method, serial device). The cases are architecture-neutral —
|
|
"a page fault is reported", not "this x86 encoding faults".
|
|
|
|
So bringing up a second architecture — an AArch64 Raspberry Pi is the motivating
|
|
one — means:
|
|
|
|
1. implement `system/kernel/arch/aarch64/` (CPU ops, its UART, exception vectors, page
|
|
tables) behind the same `arch` interface,
|
|
2. add an `aarch64` entry to `ARCHES` with its `qemu-system-aarch64` invocation,
|
|
|
|
and the *same* `smoke` / `fault-*` cases run against it: `python3 test/qemu_test.py
|
|
--arch aarch64`. A green suite on both is the definition of "it works across
|
|
architectures".
|
|
|
|
## Writing a new case
|
|
|
|
1. Add a function to `system/kernel/tests.zig` and dispatch it in `run` on its name.
|
|
2. Emit `[PASS]/[FAIL]` lines and a `DANOS-TEST-RESULT:` line (non-faulting cases),
|
|
or trigger the condition and rely on the handler's output (faulting cases).
|
|
3. Add an entry to `CASES` in `test/qemu_test.py` with the regex that proves it.
|