daniel/danos
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

M7: in-repo FAT32 image builder + boot the whole system off a USB stick

Browse Source 
The system now boots off a real FAT32 filesystem on a USB mass-storage device
instead of QEMU's synthesized VVFAT drive. A new in-repo image builder formats
that filesystem from the FHS boot tree, and both QEMU call sites (the run step
and the test harness) attach it as a usb-storage device on the xHCI bus, so
every boot exercises the full USB path OVMF -> BOOTX64.efi -> kernel.

- tools/make-fat-image.py: a Python 3 stdlib-only FAT32 formatter (mirrors
  tools/make-initial-ramdisk.py — no external host dependencies). It lays down
  the boot sector + BPB/EBPB32, FSInfo, backup boot sector, two FATs, and the
  root/subdir/file cluster chains, emitting long-name entries where a name is
  not 8.3. Packs the four boot inputs (EFI/BOOT/BOOTX64.efi, system/kernel,
  system/services/init, boot/initial-ramdisk.img) into their boot paths. A
  --verify subcommand re-checks the 0xAA55 signature, recomputes the cluster
  count -> FAT32, and resolves EFI/BOOT/BOOTX64.efi, all with no dependencies.

- build.zig: a mk_fat step builds zig-out/danos-usb.img from the four boot
  artifacts (so changing -Dtest-case rebuilds the image with that kernel), a
  check-fat-image step runs --verify, and run-x86-64 boots the image on a
  usb-storage device (if=none,id=bootusb + usb-storage,bus=xhci.0,bootindex=0),
  keeping usb-kbd/usb-mouse on the same controller.

- test/qemu_test.py: the default boot config now boots off danos-usb.img on a
  usb-storage device (xHCI + usb-kbd + usb-mouse + the boot stick). The seven
  per-case qemu_extra blocks that added their own qemu-xhci/usb-kbd/usb-mouse
  (or a VVFAT stick) collided on id=xhci and are removed — the default provides
  the bus and the boot device. usb-storage and fat-mount now exercise the real
  FAT32 boot image (usb-storage reads its 0x55AA boot sector; fat mounts it at
  /mnt/usb). A build_case override lets a case reuse another's kernel, used by a
  new usb-boot case: an explicit, named boot-from-USB regression guard.

Verified: zig build, zig build test, and zig build check-fat-image are green
(FAT32, 128992 clusters, BOOTX64.efi present); a broad sequential QEMU sweep
passes — smoke, init, vfs, input, device-manager, usb-report, usb-hid,
usb-storage, fat-mount, device-list, driver-restart, acpi-report, iommu,
orderly-shutdown, usb-boot, dma, msi, initial-ramdisk, args, process — proving
the boot switch holds across kernel tests, the full init tree, the USB stack,
the FAT mount, and orderly shutdown.
This commit is contained in:
Daniel Samson
2026-07-13 15:34:18 +01:00
parent a64a01a6a9
commit 77d2e22ed1
3 changed files with 433 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/qemu_test.py
+35 -34
View File
@@ -67,7 +67,14 @@ ARCHES = {
"-machine", "q35", "-m", "128M",
"-drive", f"if=pflash,format=raw,readonly=on,file={a['ovmf_code']}",
"-drive", f"if=pflash,format=raw,file={vars_fd}",
"-drive", f"format=raw,file=fat:rw:{boot_volume}",
# Boot off a FAT USB device: the boot volume is a mass-storage device on
# the xHCI bus (usb-kbd/usb-mouse ride the same controller). `boot_volume`
# is the FAT image the build produces. bootindex=0 steers OVMF to it.
"-device", "qemu-xhci,id=xhci",
"-device", "usb-kbd,bus=xhci.0",
"-device", "usb-mouse,bus=xhci.0",
"-drive", f"if=none,id=bootusb,format=raw,file={boot_volume}",
"-device", "usb-storage,bus=xhci.0,drive=bootusb,removable=on,bootindex=0",
"-net", "none",
"-vga", "none", "-device", "VGA,edid=on,xres=1280,yres=720",
"-display", "none",
@@ -275,9 +282,8 @@ CASES = [
{"name": "usb-report",
"smp": 4,
"timeout": 150,
"qemu_extra": ["-device", "qemu-xhci,id=xhci",
"-device", "usb-kbd,bus=xhci.0",
"-device", "usb-mouse,bus=xhci.0"],
# The xHCI bus + usb-kbd/usb-mouse come from the default boot config now
# (every case boots off a usb-storage device on that bus).
"expect": r"device-manager: child added[\s\S]*"
r"device-manager: child added[\s\S]*"
r"device-manager: test mode: killing the reporter[\s\S]*"
@@ -292,36 +298,37 @@ CASES = [
{"name": "usb-hid",
"smp": 4,
"timeout": 150,
"qemu_extra": ["-device", "qemu-xhci,id=xhci",
"-device", "usb-kbd,bus=xhci.0",
"-device", "usb-mouse,bus=xhci.0"],
# usb-kbd/usb-mouse ride the default boot xHCI bus (see qemu_args).
"expect": r"(?=[\s\S]*usb-hid/keyboard: ok)(?=[\s\S]*usb-hid/mouse: ok)",
"fail": r"DANOS-TEST-RESULT: FAIL"},
# USB mass storage end to end: attach a usb-storage device (a FAT volume via
# QEMU's VVFAT, so it has a real boot sector), boot the full tree, and let the
# manager spawn usb-storage, which opens the device, runs the Bulk-Only /
# SCSI bring-up, reads its capacity, and reads block 0 (the 0x55AA boot sig) —
# proof of the bulk transfer path + BOT + SCSI end to end.
# USB mass storage end to end: the boot usb-storage device (the FAT32 image,
# which has a real 0x55AA boot sector) is enough — the manager spawns
# usb-storage, which opens the device, runs the Bulk-Only / SCSI bring-up,
# reads its capacity, and reads block 0 (the 0x55AA boot sig). Proof of the
# bulk transfer path + BOT + SCSI end to end.
{"name": "usb-storage",
"smp": 4,
"timeout": 150,
"qemu_extra": ["-device", "qemu-xhci,id=xhci",
"-drive", "if=none,id=stick,format=raw,file=fat:rw:" + os.path.join(REPO, "zig-out"),
"-device", "usb-storage,drive=stick,bus=xhci.0"],
"expect": r"usb-storage: ready[\s\S]*usb-storage: block 0 signature 0x55aa",
"fail": r"DANOS-TEST-RESULT: FAIL"},
# FAT mount end to end: attach a usb-storage device (a FAT volume via VVFAT),
# boot the full tree, and let the fat server mount it into the VFS at /mnt/usb.
# A fat-test client then lists and reads through the mount — proof of the whole
# stack: block device -> FAT parse -> VFS routing -> file read.
# FAT mount end to end: the fat server mounts the boot usb-storage device (the
# FAT32 image) into the VFS at /mnt/usb. A fat-test client then lists and reads
# through the mount — proof of the whole stack: block device -> FAT parse ->
# VFS routing -> file read.
{"name": "fat-mount",
"smp": 4,
"timeout": 150,
"qemu_extra": ["-device", "qemu-xhci,id=xhci",
"-drive", "if=none,id=stick,format=raw,file=fat:rw:" + os.path.join(REPO, "zig-out"),
"-device", "usb-storage,drive=stick,bus=xhci.0"],
"expect": r"fat: mounted /mnt/usb[\s\S]*fat-test: ok",
"fail": r"DANOS-TEST-RESULT: FAIL"},
# Boot-from-USB smoke: the whole system now boots off the FAT32 image on a
# usb-storage device (OVMF -> \EFI\BOOT\BOOTX64.efi -> kernel), so the kernel
# reaching its PASS marker at all proves the USB boot path end to end. Reuses
# the smoke kernel build; the value is the explicit, named regression guard.
{"name": "usb-boot",
"build_case": "smoke",
"qmp_after": {"delay": 2, "command": "query-status"},
"expect": r"DANOS-TEST-RESULT: PASS",
"fail": r"DANOS-TEST-RESULT: FAIL"},
# M20.1: the ring-3 AML parse (the acpi service maps the blobs and parses
# them) finds exactly the Device count the kernel's own parse produced.
{"name": "acpi-parse",
@@ -367,9 +374,6 @@ CASES = [
{"name": "acpi-report",
"smp": 4,
"timeout": 150,
"qemu_extra": ["-device", "qemu-xhci,id=xhci",
"-device", "usb-kbd,bus=xhci.0",
"-device", "usb-mouse,bus=xhci.0"],
"expect": r"acpi: reported PNP0303 \(device \d+, 3 resources\)[\s\S]*"
r"acpi: reported PNP0F13 \(device \d+, 1 resources\)",
"fail": r"DANOS-TEST-RESULT: FAIL"},
@@ -386,9 +390,6 @@ CASES = [
{"name": "device-list",
"smp": 4,
"timeout": 150,
"qemu_extra": ["-device", "qemu-xhci,id=xhci",
"-device", "usb-kbd,bus=xhci.0",
"-device", "usb-mouse,bus=xhci.0"],
"expect": r"device-list: \d+ devices[\s\S]*"
r"device-list: subscribed[\s\S]*"
r"device-manager: test mode: killing the reporter[\s\S]*"
@@ -401,9 +402,6 @@ CASES = [
{"name": "driver-restart",
"smp": 4,
"timeout": 150,
"qemu_extra": ["-device", "qemu-xhci,id=xhci",
"-device", "usb-kbd,bus=xhci.0",
"-device", "usb-mouse,bus=xhci.0"],
"expect": r"usb-xhci-bus: hello acknowledged[\s\S]*"
r"device-manager: restarting crash-test[\s\S]*"
r"device-manager: crash-test is failing repeatedly",
@@ -509,12 +507,15 @@ def qmp_send(path, command):
def run_case(arch, case):
err = build(arch, case["name"])
# A case's kernel build defaults to its name; `build_case` decouples the two
# so a case can reuse another's kernel (e.g. usb-boot reuses smoke's).
err = build(arch, case.get("build_case", case["name"]))
if err:
return False, "build failed:\n" + err
# zig-out is the FHS boot volume; hand it to the guest as-is (see qemu_args).
boot_volume = os.path.join(REPO, "zig-out")
# The bootable FAT32 USB image the build produced (tools/make-fat-image.py),
# presented to the guest as a usb-storage device (see qemu_args).
boot_volume = os.path.join(REPO, "zig-out", "danos-usb.img")
vars_fd = os.path.join(WORK, "vars.fd")
shutil.copy(arch["ovmf_vars"], vars_fd)
serial = os.path.join(WORK, "serial.log")