M14b: DMA memory (dma_alloc / dma_free)

An HCD programs a bus-master engine: it needs a descriptor ring that is physically
contiguous, at a physical address it knows, uncacheable, and pinned. mmap gives none
of those. Add dma_alloc(len, flags) -> vaddr (rax), paddr (rdx) and dma_free(vaddr,
len): grant contiguous, zeroed, pinned, strong-uncacheable memory in a per-process DMA
arena (PML4[228]) and hand back both addresses.

Pieces: pmm.allocContiguous(count, max_phys) finds a run of contiguous free frames
below a cap (dma_below_4g for 32-bit engines); mapUserDmaInto maps them uncacheable
(PCD|PWT) but WITHOUT device_grant, so unlike an MMIO grant these frames are real RAM
and freeSubtree returns them on teardown — a driver that dies leaks nothing. dma_free
is bounded to the DMA arena so it can never unmap the caller's stack/heap/MMIO.
dma_write_combining is accepted but falls back to coherent (WC needs PAT programming).

Runtime: runtime.dma.alloc/free (a two-return-value stub, like replyWait). New `dma`
kernel test drives the mechanism directly — contiguity, the below-4G cap, coherent
mapping, and reclaim-on-teardown (no leak). The thin syscall wrappers follow the tested
mmap/mmio_map shape and land their first real use with the first DMA driver. Suite
38/38 plus host tests.
This commit is contained in:
Daniel Samson
2026-07-10 19:43:59 +01:00
parent e7c7e7b94c
commit 125a3b4993
11 changed files with 252 additions and 1 deletions
+67
View File
@@ -62,6 +62,13 @@ pub const user_half_end: u64 = 0x0000_8000_0000_0000;
pub const device_arena_base: u64 = 0x0000_7100_0000_0000;
pub const device_arena_end: u64 = device_arena_base + (4 << 30);
/// The DMA arena: where `dma_alloc` places coherent DMA buffers, in PML4[228] — a
/// user-exclusive region distinct from the MMIO arena. Unlike MMIO grants these back
/// real RAM (contiguous frames), so they are reclaimed on teardown. Per-process cursor
/// in `Task.dma_map_next`.
pub const dma_arena_base: u64 = 0x0000_7200_0000_0000;
pub const dma_arena_end: u64 = dma_arena_base + (256 << 20); // 256 MiB per process
/// Largest single `mmap` grant, in pages (1 MiB). The user heap grows in small
/// chunks, so this bound is generous; it also caps the frame scratch array below.
const maximum_mmap_pages = 256;
@@ -150,6 +157,8 @@ fn system_call(state: *architecture.CpuState) void {
.irq_ack => systemIrqAck(state),
.device_register => systemDeviceRegister(state),
.system_spawn => systemSpawn(state),
.dma_alloc => systemDmaAlloc(state),
.dma_free => systemDmaFree(state),
_ => fail(state),
}
}
@@ -261,6 +270,64 @@ fn systemMmioMap(state: *architecture.CpuState) void {
architecture.setSystemCallResult(state, base_v + (r.start & (page_size - 1))); // register base
}
/// dma_alloc(len, flags) -> vaddr (rax), paddr (rdx): grant `len` bytes (rounded up to
/// whole pages) of DMA-capable memory — physically contiguous, zeroed, pinned, and
/// strong-uncacheable (coherent) — mapping it into the caller's DMA arena and handing
/// back both the virtual address to touch and the physical address to program into the
/// device. This is what `sysMmap` can't do: mmap frames are scattered, cacheable, and
/// their physical address is never disclosed. `dma_below_4g` caps the physical address
/// for legacy engines; `dma_write_combining` is accepted but falls back to coherent
/// until PAT is programmed. See docs/driver-model.md (M14).
fn systemDmaAlloc(state: *architecture.CpuState) void {
const len = architecture.systemCallArg(state, 0);
const flags = architecture.systemCallArg(state, 1);
const t = scheduler.current();
if (t.aspace == 0 or len == 0) return fail(state);
const pages: usize = @intCast((len + page_size - 1) / page_size);
const max_phys: u64 = if (flags & abi.dma_below_4g != 0) (@as(u64, 4) << 30) else ~@as(u64, 0);
const phys = pmm.allocContiguous(pages, max_phys) orelse return fail(state);
if (t.dma_map_next == 0) t.dma_map_next = dma_arena_base;
const base_v = t.dma_map_next;
if (base_v + pages * page_size > dma_arena_end) {
for (0..pages) |i| pmm.free(phys + i * page_size); // arena exhausted; give the frames back
return fail(state);
}
// Zero through the physmap (the frames aren't mapped in the caller yet), then map.
const kernel_view: [*]u8 = @ptrFromInt(boot_handoff.physicalToVirtual(phys));
@memset(kernel_view[0 .. pages * page_size], 0);
architecture.mapUserDmaInto(t.aspace, base_v, phys, pages * page_size);
t.dma_map_next = base_v + pages * page_size;
architecture.setSystemCallResult(state, base_v); // virtual address for the CPU
architecture.setSystemCallResult2(state, phys); // physical address for the device
}
/// dma_free(vaddr, len) -> 0: release a prior `dma_alloc`. Bounded to the DMA arena so
/// it can never unmap-and-free the caller's stack, heap, or an MMIO grant; only pages
/// actually mapped are freed (an unmapped hole is skipped). Teardown also reclaims any
/// DMA pages left mapped at exit (they carry no `device_grant`, so `freeSubtree` frees
/// them as ordinary RAM), so a driver that just dies leaks nothing.
fn systemDmaFree(state: *architecture.CpuState) void {
const base_v = architecture.systemCallArg(state, 0);
const len = architecture.systemCallArg(state, 1);
const t = scheduler.current();
if (t.aspace == 0) return fail(state);
const pages: usize = @intCast((len + page_size - 1) / page_size);
if (base_v < dma_arena_base or base_v + pages * page_size > dma_arena_end) return fail(state);
for (0..pages) |i| {
const va = base_v + i * page_size;
if (architecture.translate(t.aspace, va)) |phys| {
architecture.unmapUserPageInto(t.aspace, va);
pmm.free(phys);
}
}
architecture.setSystemCallResult(state, 0);
}
/// device_register(parent_id, descriptor_ptr) -> id: publish a child device below a device
/// this process has claimed. The bus-driver primitive: a process that owns a bus
/// enumerates it and hands each device it finds to the table, where a class driver