volume-manager: rebuild a volume when its storage driver dies (S5)

The V4 review's open edge: a storage driver that crashes while its device
stays in the tree left fat wedged on a dead channel — device-presence
polling (a device-manager enumerate) still reported the device present,
so nothing reaped it. pollTick now also probes channelAlive(dev), a
geometry() on the block channel that fails fast on the dead endpoint; a
present device with a dead channel is reaped like a pull, and the adopt
loop re-adopts it on the restarted driver's fresh channel — the rebuild.
The manager's own liveness probe makes fat self-detection unnecessary:
it rebuilds regardless of the wedged filesystem's state.

The drill: the device manager gains a test-storage-restart mode that
kills usb-storage once, ~2s after its hello (post-mount); a new
volume-driver-restart kernel case boots a manual tree with it, and the
QEMU case asserts a SECOND mount of the same id-path after the reap —
the rebuild. A pre-S5 manager, checking only device presence, never
reaps, so the second mount never appears. The manually-spawned volume
manager needed kernel-supervisor protocol grants (bind its name, open
the device manager), as the other manual-tree services already have.

Full suite 133/133.

system/services/device-manager/device-manager.zig

@@ -170,6 +170,8 @@ var test_usb_killed = false;
 var test_pci_restart_mode = false;
 var test_scanout_restart_mode = false;
 var test_scanout_killed = false;
+var test_storage_restart_mode = false;
+var test_storage_killed = false;
 var test_kill_pid: u32 = 0;
 var test_kill_due_ns: u64 = 0;
 
@@ -600,6 +602,16 @@ fn onHello(_: void, invocation: Invocation(device_manager_protocol.Hello), _: An
         test_kill_due_ns = time.clock() + 1_500_000_000;
         _ = time.timerOnce(manager_endpoint, 1600);
     }
+    // Storage-driver-crash drill (S5): once, a moment after usb-storage hellos —
+    // long enough that its volume has mounted — kill it. The manager re-delegates
+    // the still-present device to a restarted driver on a fresh channel; the volume
+    // manager's channel-liveness probe must notice the dead channel and rebuild.
+    if (test_storage_restart_mode and !test_storage_killed and std.mem.eql(u8, driver.name(), "/system/drivers/usb-storage")) {
+        test_storage_killed = true;
+        test_kill_pid = invocation.sender;
+        test_kill_due_ns = time.clock() + 2_000_000_000; // after the ~0.6s mount
+        _ = time.timerOnce(manager_endpoint, 2100);
+    }
     return 0;
 }
 
@@ -763,6 +775,7 @@ pub fn main(init: process.Init) void {
         test_usb_restart_mode = std.mem.eql(u8, mode, "test-usb-restart");
         test_pci_restart_mode = std.mem.eql(u8, mode, "test-pci-restart");
         test_scanout_restart_mode = std.mem.eql(u8, mode, "test-scanout-restart");
+        test_storage_restart_mode = std.mem.eql(u8, mode, "test-storage-restart");
     }
     service.run(device_manager_protocol.message_maximum, .{
         .service = "device-manager",

system/services/volume-manager/volume-manager.zig

@@ -405,13 +405,25 @@ fn removeDevice(dev: *StorageDevice) void {
     dropDevice(dev);
 }
 
+/// Whether a device's block channel still answers — a geometry() probe. A storage
+/// driver that DIED while its device stays in the tree (it crashed; the device
+/// manager will re-delegate the device to a restarted driver on a FRESH channel)
+/// leaves a dead channel here, even though isDevicePresent still reports the device
+/// present. geometry() on the dead endpoint fails fast, so this catches the crash
+/// that presence-polling alone cannot — the V4 review's open edge.
+fn channelAlive(dev: *StorageDevice) bool {
+    return dev.channel.geometry() != null;
+}
+
 /// One poll tick. Device removal is reconciled FIRST and supersedes a pending
-/// restart: a volume whose device left is retired before its restart could fire,
-/// so nothing respawns against a dead channel. Then due restarts fire for present
-/// volumes; then, if no device is adopted, a present device is brought up.
+/// restart: a volume whose device left (a pull) OR whose driver died on a channel
+/// that no longer answers is retired before its restart could fire, so nothing
+/// respawns against a dead channel. Dropping the device frees its slot, so the
+/// adopt loop below re-adopts the still-present device on the restarted driver's
+/// fresh channel — the rebuild. Then due restarts fire for present volumes.
 fn pollTick() void {
     for (&devices) |*dev| {
-        if (dev.used and !isDevicePresent(dev.device_id)) removeDevice(dev);
+        if (dev.used and (!isDevicePresent(dev.device_id) or !channelAlive(dev))) removeDevice(dev);
     }
     for (&volumes) |*v| {
         if (v.used and v.restart_pending and time.clock() >= v.restart_due_ns) {