M15: interrupts for PCI devices (ECAM config space + MSI)

Two parts, both blockers for real PCI drivers.

ECAM config space per function: enumeratePci now gives every pci_device its own 4 KiB
configuration window as resource 0. A claimed PCI driver mmio_maps that to reach its
command register, BARs, and — the point — its capability list (MSI/MSI-X, PCIe extended
caps), with no new syscall. Verified in the discovery test (QEMU q35's functions each
carry it).

MSI: msi_bind(device_id, endpoint) -> address (rax), data (rdx) allocates a per-device
edge-triggered vector, binds it to the endpoint, and returns the (address, data) the
driver programs into its own MSI capability. Unlike irq_bind there's no GSI, no I/O APIC
entry, no sharing, and no ack cycle — dispatch recognises an MSI vector (vector_gsi ==
none, msi_bound set), EOIs, and notifies. Owner-keyed release drops the binding on exit.
Legacy INTx (_PRT parsing + shared lines) is deliberately skipped; MSI is the real
answer.

QEMU's HPET has no MSI, so the new `msi` test proves the vector-routing path with a
self-IPI (new apic.selfIpi) standing in for the device's MSI write: bind a vector,
fire it, the bound endpoint is notified. The msi_bind syscall wraps irq.msiBind with
the claim check and lands its first real use with the first PCI driver. Suite 39/39
plus host tests.
This commit is contained in:
Daniel Samson 2026-07-10 19:59:09 +01:00
parent 125a3b4993
commit 4ef21fa083
No known key found for this signature in database
GPG Key ID: A9EB2589C60F7268
10 changed files with 189 additions and 5 deletions

View File

@ -145,6 +145,13 @@ If a class driver needs `mmio`, it has become an HCD and should be one.
`dma_below_4g` caps the address for legacy engines; `dma_write_combining` is accepted
but falls back to coherent until PAT is programmed. hpet is refactored onto `/lib/mmio`;
no DMA driver consumes `dma_alloc` yet.
- **M15** — interrupts for PCI devices, the MSI half. Discovery now gives every PCI
function its 4 KiB ECAM config space as resource 0 (unblocking the capability walk
with no new syscall), and `msi_bind(device_id, endpoint) -> address, data` allocates a
per-device edge-triggered vector, delivered as an IPC notification with no mask and no
ack cycle. Legacy INTx (`_PRT` parsing + shared lines) is deliberately skipped — MSI
is the real answer. QEMU's HPET has no MSI, so delivery is proven with a self-IPI; the
first PCI driver is the first real consumer.
- **`system_spawn`** — a user-space supervisor starts a driver: `system_spawn(name)`
loads a binary bundled in the initial-ramdisk as a fresh ring-3 process. This is what
turned the device manager from "log the match" into "run the driver": the kernel now
@ -271,7 +278,12 @@ condition. Build the abstraction while there is one caller to fix.
(Zig note: `@fence` was **removed in 0.16**. Use `@atomicRmw(..., .seq_cst)` for a full
barrier, or per-arch inline asm — which is what `library/mmio.zig` should hide.)
## M15 — interrupts for PCI devices
## M15 — interrupts for PCI devices ✅ done (MSI)
*Implemented the MSI half: ECAM config space per PCI function (resource 0) and
`msi_bind` (per-device edge-triggered vector, delivered as a notification). Legacy INTx
`_PRT` parsing is skipped on purpose. `msi_bind` returns (address, data) as two values
rather than an out-struct. The rest of this section is the original design note.*
**The blocker, and it's a hard one.** No PCI device can take an interrupt today.
[`addBars`](system/devices/acpi.zig) records `.memory` and `.io_port` BARs and never an

View File

@ -3,6 +3,7 @@
//! ownership of its hardware; the claim is the capability the kernel checks before
//! mapping registers or routing an IRQ.
const abi = @import("abi");
const device_abi = @import("device-abi");
const sc = @import("system-call.zig");
@ -65,3 +66,27 @@ pub fn irqBind(device_id: u64, resource_index: u64, endpoint: usize) bool {
pub fn irqAck(device_id: u64, resource_index: u64) bool {
return !failed(sc.systemCall2(.irq_ack, device_id, resource_index));
}
/// The Message-Signalled Interrupt address/data a driver programs into its device's
/// MSI capability. The device raises the interrupt by writing `data` to `address`.
pub const Msi = struct { address: u64, data: u32 };
/// Set up MSI for a claimed device: the kernel allocates a per-device edge-triggered
/// vector, binds it to `endpoint` (delivered like `irqBind`, but with no mask and no
/// `irqAck` cycle), and returns the (address, data) to write into the device's MSI
/// capability found by mmio_mapping the device's ECAM config space (resource 0) and
/// walking its capability list. Returns null on failure. Two return values (address in
/// rax, data in rdx), so a hand-written stub.
pub fn msiBind(device_id: u64, endpoint: usize) ?Msi {
var rax: usize = undefined;
var rdx: usize = undefined;
asm volatile ("syscall"
: [rax] "={rax}" (rax),
[rdx] "={rdx}" (rdx),
: [n] "{rax}" (@intFromEnum(abi.SystemCall.msi_bind)),
[a0] "{rdi}" (device_id),
[a1] "{rsi}" (endpoint),
: .{ .rcx = true, .r11 = true, .memory = true });
if (failed(rax)) return null;
return .{ .address = rax, .data = @intCast(rdx) };
}

View File

@ -39,9 +39,16 @@ pub const SystemCall = enum(u64) {
system_spawn = 17, // system_spawn(name_ptr, name_len) -> 0: start a named initial-ramdisk binary as a new ring-3 process
dma_alloc = 18, // dma_alloc(len, flags) -> vaddr (rax), paddr (rdx): contiguous, pinned, uncacheable DMA memory
dma_free = 19, // dma_free(vaddr, len) -> 0: release a prior dma_alloc
msi_bind = 20, // msi_bind(device_id, endpoint) -> address (rax), data (rdx): a per-device MSI vector for a claimed device
_,
};
/// The x86 MSI message address base (`0xFEE0_0000`): a device raises an MSI by writing
/// `data` to this address, which the Local APIC turns into an interrupt at the vector
/// in `data`. The kernel returns the concrete (address, data) from `msi_bind`; this is
/// the fixed prefix, exposed so a driver's config-space programming reads clearly.
pub const msi_address_base: u64 = 0xFEE0_0000;
/// `dma_alloc` flags. `coherent` (uncacheable) is the portable default; the others are
/// opt-in for specific hardware. `write_combining` needs PAT programming (not yet it
/// currently falls back to coherent); see docs/driver-model.md (M14).

View File

@ -564,6 +564,15 @@ fn enumeratePci(
bridge.name(), bus, device, function,
}) catch "pcidev";
const node = try device_tree.addChild(bridge, .pci_device, nm);
// Resource 0 is the function's own 4 KiB ECAM configuration space. A
// claimed PCI driver mmio_maps this to reach its command register,
// BARs, and the point its capability list (MSI/MSI-X, PCIe
// extended caps), without any new syscall. Physical address per the
// ECAM formula (same as pciConfigurationPtr).
const config_physical = alloc.base_address +
(@as(u64, @as(u8, @intCast(bus)) - alloc.start_bus) << 20) +
(@as(u64, device) << 15) + (@as(u64, function) << 12);
_ = node.addResource(.memory, config_physical, abi.page_size);
node.ids.pci_vendor = h.vendor_id;
node.ids.pci_device = h.device_id;
node.ids.pci_class = (@as(u24, h.class_code) << 16) |

View File

@ -160,6 +160,15 @@ fn waitIcrIdle() void {
while (read(register_icr_low) & icr_delivery_pending != 0) {}
}
/// Send this core a fixed interrupt at `vector` (the "self" destination shorthand). A
/// device raises an MSI by writing its (address, data) to the LAPIC; with no such
/// device on QEMU's HPET, a self-IPI is the stand-in that lets the MSI vector-routing
/// path be tested end to end. Shorthand self (bits 19:18 = 01) | assert (bit 14).
pub fn selfIpi(vector: u8) void {
write(register_icr_low, 0x4_4000 | @as(u32, vector));
waitIcrIdle();
}
/// The calibration window: we time everything against a 10 ms reference interval.
const calib_ms = 10;

View File

@ -424,6 +424,12 @@ pub fn irqEoi() void {
apic.eoi();
}
/// Send this core a fixed interrupt at `vector`. Stands in for a device's MSI write
/// so the MSI vector-routing path can be exercised without MSI-capable hardware.
pub fn selfIpi(vector: u8) void {
apic.selfIpi(vector);
}
/// Enable the Local APIC, calibrate its timer against the best available reference
/// (see apic.calibrate no longer the PIT by default), and start it firing at
/// `timer_hz` the kernel's real-time heartbeat. Interrupts still have to be

View File

@ -67,6 +67,13 @@ var vector_gsi: [256]u32 = .{no_gsi} ** 256;
/// Vector currently assigned to each GSI (0 = none), so a rebind reuses it.
var gsi_vector: [maximum_gsi]u8 = .{0} ** maximum_gsi;
/// MSI bindings, keyed by **vector** (not GSI): an MSI has no I/O APIC redirection
/// entry, it is a per-device edge-triggered vector the device raises by writing the
/// (address, data) pair `msi_bind` hands back. Read from the ISR and written from
/// syscalls, always under the big kernel lock, like `bound`.
var msi_bound: [256]?*ipc_sync.Endpoint = .{null} ** 256;
var msi_owner: [256]u32 = .{0} ** 256;
/// Set once the trampolines are installed.
var installed = false;
@ -75,9 +82,15 @@ var installed = false;
fn dispatch(vector: u8) void {
const gsi = vector_gsi[vector];
if (gsi == no_gsi) {
// Nothing is routed here. Acknowledge so the LAPIC doesn't wedge on an
// in-service bit that never clears, but touch no redirection entry.
// No GSI is routed here. It may be an MSI vector (edge-triggered, unshared, no
// I/O APIC entry): just EOI and notify there is no line to mask and no ack
// cycle. Or it's spurious, and the EOI alone keeps the LAPIC from wedging on an
// in-service bit. Same lock discipline as below: msi_bound[vector] is read and
// used under the acquisition `msiRelease` writes it under.
_ = sync.enter();
defer sync.leaveIsr();
architecture.irqEoi();
if (msi_bound[vector]) |endpoint| ipc_sync.notifyLocked(endpoint, vector);
return;
}
@ -126,9 +139,9 @@ pub fn init() void {
fn allocVector() ?u8 {
var v: u8 = architecture.irq_vector_base;
while (v < architecture.irq_vector_base + architecture.irq_vector_count) : (v += 1) {
var used = false;
var used = msi_bound[v] != null; // taken by an MSI binding
for (gsi_vector) |gv| {
if (gv == v) used = true;
if (gv == v) used = true; // taken by a GSI binding
}
if (!used) return v;
}
@ -162,6 +175,21 @@ pub fn bind(gsi: u32, endpoint: *ipc_sync.Endpoint, owner: u32) BindError!void {
architecture.irqUnmask(gsi);
}
pub const MsiError = error{NoVector};
/// Allocate an interrupt vector and bind it to `endpoint` for task `owner`, returning
/// the vector. The `msi_bind` mechanism: MSI is edge-triggered and unshared, so there
/// is no GSI, no I/O APIC redirection entry, no mask, and no ack cycle the device
/// raises the interrupt by *writing* the (address, data) pair the syscall derives from
/// this vector, and `dispatch` just EOIs and notifies. Caller holds the big kernel
/// lock and has verified `owner` claimed the device. See docs/driver-model.md (M15).
pub fn msiBind(endpoint: *ipc_sync.Endpoint, owner: u32) MsiError!u8 {
const vector = allocVector() orelse return error.NoVector;
msi_bound[vector] = endpoint;
msi_owner[vector] = owner;
return vector;
}
/// Re-arm `gsi` after the driver has quieted the device. Caller holds the big lock
/// and has verified ownership.
pub fn ack(gsi: u32) bool {
@ -187,4 +215,13 @@ pub fn releaseOwner(owner: u32) void {
gsi_vector[gsi] = 0;
}
}
// MSI bindings carry no I/O APIC entry to mask just drop them so a stale vector
// stops notifying a freed endpoint. A late edge on a released vector is spurious
// (msi_bound is null) and `dispatch` EOIs it harmlessly.
for (&msi_bound, 0..) |*slot, vector| {
if (slot.* != null and msi_owner[vector] == owner) {
slot.* = null;
msi_owner[vector] = 0;
}
}
}

View File

@ -159,6 +159,7 @@ fn system_call(state: *architecture.CpuState) void {
.system_spawn => systemSpawn(state),
.dma_alloc => systemDmaAlloc(state),
.dma_free => systemDmaFree(state),
.msi_bind => systemMsiBind(state),
_ => fail(state),
}
}
@ -419,6 +420,29 @@ fn systemIrqBind(state: *architecture.CpuState) void {
architecture.setSystemCallResult(state, 0);
}
/// msi_bind(device_id, endpoint_handle) -> address (rax), data (rdx): set up
/// Message-Signalled Interrupts for a claimed device. The kernel allocates a vector,
/// binds it to `endpoint`, and hands back the (address, data) the driver programs into
/// its own MSI capability (found via its ECAM config space, resource 0). Unlike
/// `irq_bind` there is no GSI, no sharing, and no ack MSI is edge-triggered. The
/// claim is the capability. See docs/driver-model.md (M15).
fn systemMsiBind(state: *architecture.CpuState) void {
const device_id = architecture.systemCallArg(state, 0);
const t = scheduler.current();
if (t.aspace == 0) return fail(state);
const owner = devices_broker.ownerOf(device_id) orelse return fail(state);
if (owner != t.id) return fail(state); // not claimed by this process
const endpoint = ipc.resolveHandle(t, architecture.systemCallArg(state, 1)) orelse return failErr(state, ipc.EBADF);
const flags = sync.enter();
defer sync.leave(flags);
const vector = irq.msiBind(endpoint, t.id) catch return fail(state);
// x86 MSI: the address routes to the BSP's LAPIC (physical destination, fixed
// delivery dest field 0); the data carries the vector.
architecture.setSystemCallResult(state, abi.msi_address_base);
architecture.setSystemCallResult2(state, vector);
}
/// irq_ack(device_id, resource_index) -> 0/-1: re-arm a bound IRQ. The ISR left the line
/// masked (it could not quiet the device that's this driver's job), so nothing
/// more arrives until the driver says it has serviced the hardware.

View File

@ -86,6 +86,8 @@ pub fn run(case: []const u8, boot_information: *const BootInformation) void {
capabilityTest();
} else if (eql(case, "dma")) {
dmaTest();
} else if (eql(case, "msi")) {
msiTest();
} else if (eql(case, "smp")) {
smpTest();
} else if (eql(case, "affinity")) {
@ -234,6 +236,23 @@ fn discoveryTest() void {
check("AML parsed completely (consumed == total)", am.total > 0 and am.consumed == am.total);
check("at least one CPU enumerated (MADT)", platform.cpus().len >= 1);
// M15: every PCI function now carries its own 4 KiB ECAM configuration space as
// resource 0 the window a driver mmio_maps to walk its capability list (MSI etc).
var buffer: [64]device_abi.DeviceDescriptor = undefined;
const n = @min(devices_broker.enumerate(&buffer), buffer.len);
var pci_functions: u32 = 0;
var pci_config_ok = true;
for (buffer[0..n]) |d| {
if (d.class != @intFromEnum(device_abi.DeviceClass.pci_device)) continue;
pci_functions += 1;
const has_config = d.resource_count >= 1 and
d.resources[0].kind == @intFromEnum(device_abi.ResourceKind.memory) and
d.resources[0].len == abi.page_size;
if (!has_config) pci_config_ok = false;
}
check("PCI functions were enumerated (MCFG/ECAM)", pci_functions >= 1);
check("each PCI function exposes its ECAM config space as resource 0", pci_config_ok);
result();
}
@ -984,6 +1003,37 @@ fn dmaTest() void {
result();
}
/// MSI (M15): a per-device, edge-triggered interrupt vector delivered as an IPC
/// notification. QEMU's HPET has no MSI, so this exercises the vector-routing path with
/// a self-IPI standing in for the device's MSI memory write proving the kernel
/// allocates a vector, `dispatch` recognises it as MSI (EOI + notify, no mask cycle),
/// and the bound endpoint is notified. The `msi_bind` syscall wraps `irq.msiBind` with
/// the device-claim check and lands its first real use with the first PCI driver.
fn msiTest() void {
log("DANOS-TEST-BEGIN: msi\n", .{});
const endpoint = ipcsync.createEndpoint().?;
const flags = sync.enter();
const vector = irq.msiBind(endpoint, 0) catch {
sync.leave(flags);
check("msiBind allocated a vector", false);
result();
return;
};
sync.leave(flags);
check("msiBind allocated a vector in the device window", vector >= architecture.irq_vector_base and
vector < architecture.irq_vector_base + architecture.irq_vector_count);
// Fire the vector the stand-in for the device writing its MSI (address, data).
const tail: *volatile u8 = &endpoint.notify_tail;
const before = tail.*;
architecture.selfIpi(vector);
var spins: u64 = 0;
while (tail.* == before and spins < 100_000_000) : (spins += 1) scheduler.yield();
check("self-IPI at the MSI vector notified the bound endpoint", tail.* != before);
result();
}
var proc_worker_run: bool = true;
var proc_worker_ran: bool = false;

View File

@ -135,6 +135,11 @@ CASES = [
{"name": "dma",
"expect": r"DANOS-TEST-RESULT: PASS",
"fail": r"DANOS-TEST-RESULT: FAIL"},
# MSI (M15): allocate a per-device vector and deliver it as a notification (a
# self-IPI stands in for the device's MSI write, since the HPET has no MSI).
{"name": "msi",
"expect": r"DANOS-TEST-RESULT: PASS",
"fail": r"DANOS-TEST-RESULT: FAIL"},
# Parallelism: needs more than one core, so this case boots with -smp 4.
{"name": "smp",
"smp": 4,