Daniel Samson
79d859a111
child_added/child_removed join the device-manager protocol. The driver maps its register BAR (resource 0 is the ECAM config space; the walk starts at 1), reads CAPLENGTH and HCSPARAMS1, and reads one PORTSC per port: the connect bit and speed class come straight from hardware, no rings needed to see the devices. The manager mirrors reported children keyed by (parent, port), remembers which instance reported each, and prunes a dead reporter's children before deciding the restart — the children describe protocol state that died with the process. The usb-report scenario drives the whole loop: two QEMU devices reported, reporter killed, children pruned, driver respawned with backoff, and the new instance re-claims, re-scans, and re-reports.
202 lines
11 KiB
Markdown
202 lines
11 KiB
Markdown
# M17–M18 execution plan: process lifecycle + device manager
|
||
|
||
The operational plan for building [process-lifecycle.md](process-lifecycle.md)
|
||
(M17) and [device-manager.md](device-manager.md) increments 5–7 (M18). Design is
|
||
settled in those documents; this file is the build order — one phase at a time,
|
||
each phase green before the next starts. Delete or archive this file when M18
|
||
lands.
|
||
|
||
**Definition of green, every phase:** `zig build` clean, `zig build test` clean,
|
||
`python3 test/qemu_test.py` passes (existing scenarios plus the phase's new one),
|
||
and the relevant design doc's "known gaps" / status lines updated. Commit per
|
||
green phase (no co-author trailers).
|
||
|
||
**Workflow (settled 2026-07-12):** work happens in a dedicated git worktree, on
|
||
feature branches cut from `main` — `feat/process-lifecycle` (M17.1–17.4),
|
||
`feat/device-manager` (M18.1), `feat/usb-xhci-bus` (M18.2–18.3). When a branch's
|
||
phases are all green it is **auto-merged into `main`**; branches are kept after
|
||
merge, not deleted. Merges and branches are pushed to origin. Phase 0 (once):
|
||
commit the design docs, merge the outstanding `feat/usb` work into `main`, and
|
||
run the existing QEMU suite green before any new work starts.
|
||
|
||
**Numbering note:** continues the milestone sequence (driver track ended at M16).
|
||
|
||
## Status
|
||
|
||
The loop marks a phase `[x]` in the same commit that lands it. A phase is marked
|
||
only when its definition of green holds.
|
||
|
||
- [x] **Phase 0** — baseline: docs committed, feat/usb merged to main, pushed;
|
||
`usb-xhci-libary.zig` renamed to `usb-xhci-library.zig`; existing QEMU
|
||
suite green from the worktree (48/48, 2026-07-12).
|
||
- [x] **M17.1** — kernel releases claims/MSI on death (claims: `releaseAllOwnedBy`
|
||
in the reap; MSI was already swept by `irq.releaseOwner`; `claim-release`
|
||
test; suite 49/49)
|
||
- [x] **M17.2** — exit reasons (`ExitReason` recorded at exit/fault/kill before
|
||
the notification; `process_exit_reason` supervisor-gated;
|
||
`runtime.process.exitReason`; kernel + ring-3 assertions; suite 49/49)
|
||
- [x] **M17.3** — published exit events + VFS subscriber (`process_subscribe`,
|
||
bounded ref-counted table, publish on every death;
|
||
`runtime.process.subscribeExits`; VFS handles carry owners and are swept on
|
||
the owner's death; `vfs-client-death` test; suite 50/50)
|
||
- [x] **M17.4** — signals, timer notifications, `runtime.process`, the service
|
||
harness (signal_bind/process_signal + coalescing pending mask; timer_bind
|
||
on the tick; bindSignals/signalsFrom/sendSignal/stop + timerOnce;
|
||
runtime.service.run with the zero-length ping; VFS converted; `signals`
|
||
scenario; suite 51/51)
|
||
- [x] **merge** `feat/process-lifecycle` → main, push (merged 2026-07-13)
|
||
- [x] **M18.1** — device-manager protocol: hello + restart policy
|
||
(device-manager-protocol module; the manager as a harness service:
|
||
supervised spawns, hello deadline via timer sweep, restart with
|
||
300/600/1200ms backoff, exit reasons deciding restart-vs-stopped,
|
||
crash-loop cap; usb-xhci-bus first conforming driver; crash-test fixture
|
||
re-proving claim release each respawn; `driver-restart` scenario;
|
||
maximum_tasks 16→32 — the sweep was overflowing the pool; suite 52/52)
|
||
- [x] **merge** `feat/device-manager` → main, push (merged 2026-07-13)
|
||
- [x] **M18.2** — xHCI port scan + tree reports (child_added/child_removed in
|
||
the protocol; the manager's child mirror with death-pruning; xHCI maps the
|
||
register BAR — resource 0 is ECAM — reads CAPLENGTH/HCSPARAMS1, scans
|
||
PORTSC, reports connected ports with speed-class identity; `usb-report`
|
||
scenario proves report → prune → respawn → re-report; suite 53/53)
|
||
- [ ] **M18.3** — app surface: enumerate/subscribe + device-list
|
||
- [ ] **merge** `feat/usb-xhci-bus` → main, push — **loop ends here**
|
||
|
||
---
|
||
|
||
## M17.1 — the kernel releases a dead process's claims
|
||
|
||
The cleanup half of iron rule 1; the prerequisite for every restart story.
|
||
|
||
- `system/kernel/devices-broker.zig`: `releaseAllOwnedBy(owner: u32)` — clear
|
||
every `claimed[]` slot holding this task id.
|
||
- `system/kernel/process.zig`: call it from the reap path, alongside the existing
|
||
IRQ-binding release (the ordering comment there says why IRQs go first — claims
|
||
slot in after them, before the exit notification).
|
||
- MSI vectors: find where `msi_bind` records per-device vectors (interrupts
|
||
module) and release those by owner in the same pass.
|
||
- Docs: remove the claims bullet from process-management.md "Known gaps".
|
||
|
||
**Test:** new QEMU scenario `claim-release` — a test child claims an unclaimed
|
||
device, is killed, is respawned, and claims the same device again successfully;
|
||
assert both claims in the serial log. Kernel-side unit coverage in
|
||
`system/kernel/tests.zig` for `releaseAllOwnedBy` (claim two devices as two owners,
|
||
release one owner, verify exactly its claims freed).
|
||
|
||
## M17.2 — exit reasons
|
||
|
||
- `system/abi.zig`: `ExitReason` (exited, aborted, segmentation_fault,
|
||
illegal_instruction, arithmetic_fault, killed).
|
||
- Kernel: record the reason at every death site — clean exit path, each fault
|
||
class in `onException`, the kill path. Bounded recent-exits table (ids are never
|
||
reused, so a small ring keyed by id is enough).
|
||
- New system call `process_exit_reason(id)` — supervisor-gated, like kill; returns
|
||
the recorded reason or `-ESRCH` once evicted.
|
||
- `library/runtime/process.zig`: `ExitReason` + `exitReason(id: u32)`.
|
||
- Docs: remove the no-exit-status bullet from process-management.md.
|
||
|
||
**Test:** extend the `supervision` scenario — three children: one exits cleanly,
|
||
one faults (the fault-recovery pattern), one is killed; the supervisor asserts all
|
||
three reasons.
|
||
|
||
## M17.3 — published exit events
|
||
|
||
- Kernel: bounded subscriber table (endpoints); new system call
|
||
`process_subscribe(endpoint)` (ungated, like `process_enumerate`); every death
|
||
posts `notify_exit_bit | id` to each subscriber — the same post the supervisor
|
||
path already uses.
|
||
- `library/runtime/process.zig`: `subscribeExits(endpoint)`.
|
||
- VFS becomes the first subscriber: on an exit event, release every handle keyed
|
||
by that task id (badges already are task ids). Log the release.
|
||
- Docs: note the convention in ipc.md (exit events reuse the exit-notification
|
||
badge encoding).
|
||
|
||
**Test:** new QEMU scenario `vfs-client-death` — a client opens a file and is
|
||
killed without closing; assert the VFS logs the handle release and its open-handle
|
||
count returns to baseline.
|
||
|
||
## M17.4 — signals and the service harness
|
||
|
||
- Kernel: per-task pending mask + bound endpoint; system calls
|
||
`signal_bind(endpoint)` and `process_signal(id, signal)` (supervisor-or-self
|
||
gated); delivery posts `notify_signal_bit | pending mask`, coalescing; pending
|
||
signals with no bound endpoint pend silently.
|
||
- `library/runtime/process.zig`: `Signal`, `SignalSet`, `bindSignals`,
|
||
`signalsFrom`, `sendSignal`, `stop(id, deadline_ms)` (terminate → wait for exit
|
||
notification → kill). Implement `terminate`, `reload`, `user_1`, `user_2`;
|
||
`interrupt`/`quit` are enum members with no sender yet; `alarm` stays unbuilt.
|
||
- Kernel: **one-shot timer notifications** — `timer_bind(endpoint, ms)` posts a
|
||
notification badge when the deadline lands (IRQ-as-IPC again, on the timer
|
||
wheel `sleep` already uses). This is the missing timed-wait primitive:
|
||
`replyWait` blocks forever and `sleep` blocks the whole process, but `stop()`'s
|
||
escalation, the device manager's `hello` deadline (M18.1), and restart backoff
|
||
all need a deadline while staying responsive. It is also the mechanism `alarm`
|
||
gets for free later.
|
||
- New `library/runtime/service.zig`: the harness — `run(callbacks)` owning the
|
||
replyWait loop, folding protocol messages, signals, and child-exit notifications
|
||
into `init` / `on_message` / `on_reload` / `on_terminate`; answers the common
|
||
`ping` automatically. Define the reserved `ping` request encoding here and
|
||
document it in ipc.md (one obvious encoding; smallest that cannot collide with
|
||
existing protocols).
|
||
- Convert one existing service (input-source or hpet) to the harness as proof it
|
||
subtracts code rather than adding it.
|
||
|
||
**Test:** extend `supervision` — a harness-built child: `sendSignal(reload)`
|
||
observed in its log, `ping` answered, `stop()` produces a clean exit with reason
|
||
`exited`; a second child that ignores signals (no bind) is killed by `stop()`'s
|
||
deadline with reason `killed`.
|
||
|
||
## M18.1 — device-manager protocol: hello + restart policy
|
||
|
||
- New `system/services/device-manager/device-manager-protocol.zig` module
|
||
(vfs-protocol pattern): `hello { version, role, device_id }`; version constant;
|
||
reserved fields.
|
||
- Device manager: register the `.device_manager` endpoint; spawn drivers with its
|
||
exit endpoint; enforce the hello deadline; restart policy — backoff, crash-loop
|
||
cap (three fast deaths → mark failed, log, stop), reasons from M17.2 deciding
|
||
restart vs not.
|
||
- usb-xhci-bus: adopt the harness + send hello. hpet/ps2-bus follow only if the
|
||
conversion is mechanical; otherwise they keep working unconverted (the manager
|
||
only enforces hello on drivers spawned with an assignment).
|
||
- build.zig: test-loop entry for the protocol module if it grows pure logic.
|
||
|
||
**Test:** new QEMU scenario `driver-restart` — the xHCI driver takes a test-only
|
||
argv flag to fault after hello on its first run; assert: fault, exit reason
|
||
recorded, manager respawns with backoff, second run claims the controller
|
||
(M17.1) and hellos clean. Assert the crash-loop cap by a driver that always
|
||
faults (a tiny test driver, not xhci).
|
||
|
||
## M18.2 — bus tree reports
|
||
|
||
- Protocol: `child_added { parent, identity, resources }` / `child_removed { id }`.
|
||
- usb-xhci-bus: bring-up to **port scan only** — map the MMIO window (claimed in
|
||
M16-era work), controller reset/start per xHCI spec, walk the port registers,
|
||
report one `child_added` per connected port with speed + port number as
|
||
identity. **No transfer rings, no descriptors** — reading device/interface
|
||
descriptors (and therefore USB class triples for matching) is the follow-on USB
|
||
track, not this plan.
|
||
- Device manager: mirror reports into its tree; prune the subtree (emitting
|
||
`child_removed`) when a bus driver dies; assert re-report on restart.
|
||
|
||
**Test:** QEMU already attaches usb-kbd + usb-mouse on xhci.0 — assert two
|
||
`child_added` events reach the manager and appear in its tree dump; kill the
|
||
driver, assert two `child_removed` then two fresh `child_added` after respawn.
|
||
|
||
## M18.3 — the application surface
|
||
|
||
- Protocol: `enumerate` (tree snapshot) + `subscribe` (published add/remove
|
||
events, input-service pattern).
|
||
- A small client (`device-list`, the `ps` analog) exercising both; the manager
|
||
becomes the one answer to "what devices exist" for user space.
|
||
`device_enumerate` stays for drivers/kernel seeding — its retreat is tied to the
|
||
discovery migration, out of this plan.
|
||
|
||
**Test:** QEMU scenario — `device-list` shows the tree including USB children;
|
||
during a driver restart the subscribing client logs remove + add events.
|
||
|
||
---
|
||
|
||
**Explicitly out of scope** (own tracks, after M18): discovery migration (pci-bus
|
||
driver, acpi service, retiring the kernel scan), USB control transfers +
|
||
descriptors + class-driver matching, the musl layer, `interrupt`/`quit` senders
|
||
(needs a console), job control.
|