display: resilient scanout — supervised driver, survive loss, re-attach (v2 V6)

The compositor now survives the virtio-gpu driver dying and re-attaches when device-manager
restarts it — the last piece of display v2.

Three parts:

- The driver hellos the device manager (role: bus). It never did, so the manager — which
  spawns it supervised and expects a hello — was stopping it at the 3s hello deadline every
  run (the gate markers just printed first). Now it is properly supervised: not stopped for
  silence, and restarted on death.

- A kernel IPC fix so a call to a dead service errors instead of hanging. An endpoint records
  its owner; when that task dies, its registered endpoints are marked dead (and any parked
  senders woken with -EPEER), so ipc_call returns -EPEER rather than blocking on a reply that
  will never come. Without this the compositor's first present after the driver died blocked
  forever. General robustness — any client of any service benefits.

- The compositor re-attaches. Its .scanout calls now fail cleanly (caught), freezing the last
  frame; when the restarted driver re-announces, attach_scanout detects the backend is already
  virtio and logs "scanout re-attached", mapping the fresh shared surface and re-looking-up
  .scanout. (The previous shm mapping leaks — no shm_unmap syscall yet — but its frames are the
  dead driver's, reclaimed on exit.)

- device-manager gains a "test-scanout-restart" mode (like test-usb-restart) that kills the
  virtio-gpu driver once after it hellos; the displayReattachTest kernel scenario drives it.

Gate: python3 test/qemu_test.py display-reattach — "scanout upgraded to virtio-gpu" then
"scanout re-attached", no CPU exception, passing 3/3. host tests, ipc/ipc-call/ipc-cap,
supervision, shm, display-service, display-demo, virtio-gpu, display-native, and
display-modeset all pass; default zig build clean. v2 (V1-V6) complete.

docs/README.md

@@ -73,8 +73,9 @@ rather than restate it. Roughly in the order things happen at runtime:
     track: a user-space compositor that owns the framebuffer, composes a layer stack into
     a double buffer, and presents it. Why GOP and the PCI display device are two views of
     one controller, the device-node + write-combining handoff, and what flicker-free buys
-    that tear-free doesn't. Plan: [display-plan.md](display-plan.md). **v2** makes scanout
-    a pluggable backend (GOP floor + a native virtio-gpu driver, hot-attached):
+    that tear-free doesn't. Plan: [display-plan.md](display-plan.md). **v2** (complete) makes
+    scanout a pluggable backend — GOP floor + a native virtio-gpu driver, hot-attached, with
+    runtime mode-set, EDID, fenced vsync presents, and restart re-attach:
     [display-v2.md](display-v2.md), plan [display-v2-plan.md](display-v2-plan.md).
 20. **[halting.md](halting.md) — halting.** Why a kernel can't just "exit", and
     how `while (true) hlt` parks the CPU safely once there's nothing left to do.

docs/display-v2-plan.md

@@ -141,16 +141,24 @@ change by reading the backend's geometry back (`display: mode set to {w}x{h}, ve
 fenced present path is exercised and confirmed (`display: vsync present ok`) — both from serial,
 passing 3/3. The driver also logs the EDID preferred mode (`virtio-gpu: EDID preferred mode …`).
 
-## V6 — Resilience (restart + re-attach) + tests + docs
+## V6 — Resilience (restart + re-attach) + tests + docs ✅
 
-- [ ] The virtio-gpu driver is supervised (device-manager / init) and restartable; on
-      driver loss the compositor freezes the last frame and **re-attaches** when the driver
-      re-announces. Only a permanent give-up (crash-loop cap) attempts GOP again.
-- [ ] `test/qemu_test.py` cases: `virtio-gpu`, hot-attach, `display-modeset`, and a
-      driver-kill/re-attach case. Update [display-v2.md](display-v2.md) status; README index.
+- [x] The virtio-gpu driver now **hellos** the device manager (role: bus) so it is properly
+      supervised — no longer stopped at the hello deadline — and is restarted on death. On
+      driver loss the compositor keeps the last frame (its `.scanout` calls now return
+      `-EPEER` instead of hanging — a kernel fix: an endpoint is marked dead when its owner
+      dies) and **re-attaches** when the restarted driver re-announces. A permanent give-up
+      (crash-loop cap) leaves the frozen frame; GOP is not re-taken.
+- [x] `test/qemu_test.py`: the `virtio-gpu`, `display-native` (hot-attach), `display-modeset`,
+      and `display-reattach` (driver-kill/re-attach) cases. display-v2.md status updated.
 
-**Gate:** kill the virtio-gpu driver mid-run; the compositor survives and re-attaches on
-restart (`display: scanout re-attached`); all v1 + v2 cases pass; default `zig build` clean.
+**Gate (met):** the `display-reattach` case — device-manager (in `test-scanout-restart` mode)
+kills the virtio-gpu driver once after it hellos; the restart policy respawns it, it
+re-announces, and the compositor logs `display: scanout re-attached` after the initial
+`display: scanout upgraded to virtio-gpu`, with no CPU exception / panic (the compositor
+survives) — passing 3/3. All v1 + v2 cases (host tests, `ipc`/`ipc-call`/`ipc-cap`,
+`supervision`, `shm`, `display-service`, `display-demo`, `virtio-gpu`, `display-native`,
+`display-modeset`) pass; default `zig build` is clean.
 
 ---
 

docs/display-v2.md

@@ -1,5 +1,10 @@
 # The display service v2: a pluggable scanout backend
 
+**Status: complete (V1–V6).** The compositor boots on the GOP framebuffer and, when a
+virtio-gpu driver announces itself, hot-attaches a native backend over the shared `shm`
+scanout surface — with runtime mode-setting, EDID, and fenced (vsync) presents, and it
+re-attaches across driver restarts. All serial-gated (see [display-v2-plan.md](display-v2-plan.md)).
+
 v1 ([display.md](display.md)) is a compositor that owns the **GOP framebuffer** — it
 composites a layer stack into a cacheable back buffer and streams damage to the linear
 framebuffer the firmware handed over. That path is portable and good: it drives any GPU,

system/drivers/virtio-gpu/virtio-gpu.zig

@@ -23,6 +23,7 @@ const system = runtime.system;
 const ipc = runtime.ipc;
 const dp = runtime.display_protocol;
 const sp = runtime.scanout_protocol;
+const dm = runtime.device_manager_protocol;
 const vp = @import("virtio-pci.zig");
 const vg = @import("virtio-gpu-protocol.zig");
 
@@ -400,6 +401,11 @@ fn initialise(endpoint: ipc.Handle) bool {
     }
     log("virtio-gpu: scanout {d}x{d} online\n", .{ current_width, current_height });
 
+    // Hello the device manager so it counts us as up (and does not stop us at the hello
+    // deadline). A restarted instance re-hellos here and re-announces below — the compositor
+    // re-attaches to the fresh scanout (V6).
+    helloManager();
+
     // Read the monitor's EDID (best-effort, when the device offers it) — the mode list a real
     // driver derives from it; we log the preferred mode and keep our fixed offered list.
     readEdid();
@@ -500,6 +506,31 @@ fn presentFull() bool {
     return true;
 }
 
+/// Hello the device manager (role: bus — we own a PCI function, though we report no children):
+/// the handshake that marks us up so the manager doesn't stop us at the hello deadline, and
+/// (as a supervised driver) restarts us if we die. Best-effort: without a manager we still run.
+fn helloManager() void {
+    var tries: u32 = 0;
+    const manager = while (tries < 100) : (tries += 1) {
+        if (ipc.lookup(.device_manager)) |h| break h;
+        system.sleep(20);
+    } else {
+        log("virtio-gpu: no device manager to hello\n", .{});
+        return;
+    };
+    const hello = dm.Hello{ .role = @intFromEnum(dm.Role.bus), .device_id = device_id };
+    var reply: [dm.reply_size]u8 = undefined;
+    const n = ipc.call(manager, std.mem.asBytes(&hello), &reply) catch {
+        log("virtio-gpu: hello call failed\n", .{});
+        return;
+    };
+    if (n < dm.reply_size or std.mem.bytesToValue(dm.HelloReply, reply[0..dm.reply_size]).status != 0) {
+        log("virtio-gpu: hello refused\n", .{});
+        return;
+    }
+    log("virtio-gpu: hello acknowledged\n", .{});
+}
+
 /// Announce the scanout to the display service so it upgrades off the GOP framebuffer: hand it
 /// the shared surface as a capability plus the geometry. Best-effort and non-fatal — without a
 /// display service (the standalone virtio-gpu bring-up test) the driver is still a valid

system/kernel/ipc-synchronous.zig

@@ -90,6 +90,10 @@ const user_half_end: u64 = 0x0000_8000_0000_0000;
 /// (per process) and/or by a registry slot, counted by `refcount`.
 pub const Endpoint = struct {
     refcount: u32 = 1,
+    // The task that created it. When that task dies, the endpoint is marked `dead` so a caller
+    // gets -EPEER instead of blocking forever on a service that will never reply again (V6).
+    owner: u32 = 0,
+    dead: bool = false,
     // Callers blocked in `call`, awaiting receive, in FIFO order (threaded via
     // Task.next; each such task is .blocked and in no scheduler queue).
     sender_head: ?*Task = null,
@@ -110,10 +114,30 @@ pub const Endpoint = struct {
 
 pub fn createIpcEndpoint() ?*Endpoint {
     const endpoint = heap.allocator().create(Endpoint) catch return null;
-    endpoint.* = .{};
+    endpoint.* = .{ .owner = scheduler.currentId() };
     return endpoint;
 }
 
+/// A task is dying: kill the endpoints it registered as services. Mark each `dead` (so a later
+/// `call` returns -EPEER rather than blocking on a reply that will never come), wake anyone
+/// already parked sending to it with that error, and vacate its registry slot. Only *registered*
+/// endpoints are reachable from here; unregistered ones drop with the task's handle table. The
+/// caller holds the big kernel lock (this runs on the death path). See docs/display-v2.md (V6).
+pub fn killOwnedEndpointsLocked(task_id: u32) void {
+    for (&registry) |*slot| {
+        const endpoint = slot.* orelse continue;
+        if (endpoint.owner != task_id) continue;
+        endpoint.dead = true;
+        while (dequeueSender(endpoint)) |sender| {
+            sender.ipc_status = -EPEER;
+            sender.ipc_received_cap = abi.no_cap;
+            scheduler.readyLocked(sender);
+        }
+        slot.* = null;
+        dropRef(endpoint);
+    }
+}
+
 /// Drop a reference; free the endpoint when the last one goes. (Frames are leaked
 /// today like other kernel objects — but the refcount bookkeeping lands now.)
 pub fn dropRef(endpoint: *Endpoint) void {
@@ -293,6 +317,7 @@ pub fn call(endpoint: *Endpoint, message_ptr: u64, message_len: u64, reply_ptr:
     if (message_len > MESSAGE_MAXIMUM or reply_cap > MESSAGE_MAXIMUM) return -E2BIG;
     const flags = sync.enter();
     defer sync.leave(flags);
+    if (endpoint.dead) return -EPEER; // the service that owned this endpoint is gone — don't block
 
     const me = scheduler.current();
     me.ipc_send_ptr = message_ptr;

system/kernel/process.zig

@@ -737,6 +737,7 @@ fn releaseTaskResourcesLocked(t: *scheduler.Task) void {
         scheduler.readyLocked(client); // its blocked `call` now returns the error
     }
     ipc.abandonSenderLocked(t);
+    ipc.killOwnedEndpointsLocked(t.id); // its registered services are gone: callers get -EPEER, not a hang
     scheduler.removeFromWaitQueueLocked(t);
     scheduler.forgetIpcClientLocked(t);
     ipc.closeHandles(t);

system/kernel/tests.zig

@@ -107,6 +107,8 @@ pub fn run(case: []const u8, boot_information: *const BootInformation) void {
         virtioGpuTest(boot_information);
     } else if (eql(case, "display-native")) {
         displayNativeTest(boot_information);
+    } else if (eql(case, "display-reattach")) {
+        displayReattachTest(boot_information);
     } else if (eql(case, "clock")) {
         clockTest();
     } else if (eql(case, "smp")) {
@@ -2504,6 +2506,50 @@ fn displayNativeTest(boot_information: *const BootInformation) void {
     while (true) scheduler.yield();
 }
 
+/// V6 — resilience: the compositor survives the virtio-gpu driver dying and re-attaches when
+/// device-manager restarts it (docs/display-v2.md). Same boot as display-native, but the
+/// manager runs in "test-scanout-restart" mode: a moment after the driver hellos, it kills it
+/// once; the normal restart policy respawns it, the restarted driver re-announces, and the
+/// compositor re-attaches to the fresh scanout — logging `display: scanout re-attached` after
+/// the initial `display: scanout upgraded to virtio-gpu`. The compositor must not crash.
+fn displayReattachTest(boot_information: *const BootInformation) void {
+    log("DANOS-TEST-BEGIN: display-reattach\n", .{});
+    if (boot_information.initial_ramdisk_len == 0) {
+        check("bootloader handed over an initial_ramdisk", false);
+        result();
+        return;
+    }
+    const image = @as([*]const u8, @ptrFromInt(boot_handoff.physicalToVirtual(boot_information.initial_ramdisk_base)))[0..boot_information.initial_ramdisk_len];
+    const rd = initial_ramdisk.Reader.init(image) orelse {
+        check("initial_ramdisk image is valid", false);
+        result();
+        return;
+    };
+
+    process.setInitialRamdisk(image);
+    var manager: u32 = 0;
+    var i: u32 = 0;
+    while (i < rd.count) : (i += 1) {
+        const item = rd.entry(i) orelse continue;
+        if (!eql(item.name, "device-manager")) continue;
+        manager = process.spawnProcessSupervised(item.blob, 4, &.{ "device-manager", "test-scanout-restart" }, scheduler.currentId(), null) catch 0;
+        break;
+    }
+    if (manager == 0) {
+        log("display-reattach: could not spawn device-manager\n", .{});
+        result();
+        return;
+    }
+    if (!spawnNamed(rd, "display")) {
+        log("display-reattach: could not spawn the display service\n", .{});
+        result();
+        return;
+    }
+    _ = spawnNamed(rd, "display-demo");
+    scheduler.setPriority(1); // below the compositor, the demo, and the driver, so they run
+    while (true) scheduler.yield();
+}
+
 /// Process arguments, end to end: spawn args-echo bare (its argv[0] is the
 /// initial-ramdisk name). Instance 1 sees argc == 1 and respawns itself through
 /// `system_spawn` with the extra arguments "alpha beta-42" — the syscall argument

system/services/device-manager/device-manager.zig

@@ -159,6 +159,8 @@ var test_restart_mode = false;
 var test_usb_restart_mode = false;
 var test_usb_killed = false;
 var test_pci_restart_mode = false;
+var test_scanout_restart_mode = false;
+var test_scanout_killed = false;
 var test_kill_pid: u32 = 0;
 var test_kill_due_ns: u64 = 0;
 
@@ -425,6 +427,14 @@ fn onMessage(message: []const u8, reply: []u8, sender: u32, capability: ?runtime
     } else if (driverByProcess(sender)) |driver| {
         driver.state = .running;
         writeLine("/system/services/device-manager: hello from {s} (device {d})\n", .{ driver.name(), hello.device_id });
+        // Resilience drill (V6): once, kill the virtio-gpu driver a moment after it hellos, so
+        // the normal restart policy respawns it — the compositor must survive and re-attach.
+        if (test_scanout_restart_mode and !test_scanout_killed and std.mem.eql(u8, driver.name(), "virtio-gpu")) {
+            test_scanout_killed = true;
+            test_kill_pid = sender;
+            test_kill_due_ns = system.clock() + 1_500_000_000;
+            _ = system.timerOnce(manager_endpoint, 1600);
+        }
     } else {
         status = -1;
         writeLine("/system/services/device-manager: hello from unknown process {d}\n", .{sender});
@@ -567,6 +577,7 @@ pub fn main(init: runtime.process.Init) void {
         test_restart_mode = std.mem.eql(u8, mode, "test-restart");
         test_usb_restart_mode = std.mem.eql(u8, mode, "test-usb-restart");
         test_pci_restart_mode = std.mem.eql(u8, mode, "test-pci-restart");
+        test_scanout_restart_mode = std.mem.eql(u8, mode, "test-scanout-restart");
     }
     runtime.service.run(protocol.message_maximum, .{
         .service = .device_manager,

system/services/display/display.zig

@@ -224,6 +224,13 @@ fn attachScanout(stride: u32, width: u32, height: u32, format: u32, capability:
     if (width == 0 or height == 0 or stride < width) return fail(reply);
     const mapped = runtime.shm.map(cap) orelse return fail(reply);
     const scanout = ipc.lookup(.scanout) orelse return fail(reply);
+    // A second announce means the driver died and was restarted (V6): re-attach to its fresh
+    // scanout. (The previous shared mapping leaks — there is no shm_unmap syscall yet — but the
+    // frames are the dead driver's, reclaimed on its exit; a handful across a crash is benign.)
+    const reattach = switch (backend) {
+        .virtio => true,
+        else => false,
+    };
 
     backend = .{ .virtio = .{
         .pixels = @ptrCast(@alignCast(mapped)),
@@ -236,9 +243,12 @@ fn attachScanout(stride: u32, width: u32, height: u32, format: u32, capability:
     background = protocol.pack(format, 0x20, 0x30, 0x48); // re-pack the wallpaper for the mode
     addDamage(screenRect()); // the whole new surface must be painted
     pending_native_verify = true;
-    pending_modeset_check = true;
+    if (!reattach) pending_modeset_check = true; // the mode-set self-check runs once, on first upgrade
     _ = system.timerOnce(service_endpoint, 50); // present once the driver is serving .scanout
-    _ = system.write("display: scanout upgraded to virtio-gpu\n");
+    _ = system.write(if (reattach)
+        "display: scanout re-attached\n"
+    else
+        "display: scanout upgraded to virtio-gpu\n");
     return ok(reply);
 }
 

test/qemu_test.py

@@ -221,6 +221,15 @@ CASES = [
      "mem": "512M",
      "expect": r"(?s)(?=.*display: mode set to \d+x\d+, verified)(?=.*display: vsync present ok)",
      "fail":   r"display: mode set FAILED|display: mode-set self-check: |display: native present FAILED|CPU EXCEPTION|KERNEL PANIC"},
+    # Resilience: driver restart + re-attach (v2 V6). device-manager (in test-scanout-restart
+    # mode) kills the virtio-gpu driver once after it hellos; the restart policy respawns it, it
+    # re-announces, and the compositor re-attaches — surviving the loss. Expect the initial
+    # upgrade AND the re-attach; any CPU exception / panic (the compositor crashing) is a fail.
+    {"name": "display-reattach",
+     "qemu_extra": ["-device", "virtio-gpu-pci"],
+     "mem": "512M",
+     "expect": r"(?s)(?=.*display: scanout upgraded to virtio-gpu)(?=.*display: scanout re-attached)",
+     "fail":   r"CPU EXCEPTION|KERNEL PANIC|display: could not"},
     # Monotonic clock (clock() syscall source): calibrated, advancing, never backwards.
     {"name": "clock",
      "expect": r"DANOS-TEST-RESULT: PASS",