M14b: DMA memory (dma_alloc / dma_free)

An HCD programs a bus-master engine: it needs a descriptor ring that is physically
contiguous, at a physical address it knows, uncacheable, and pinned. mmap gives none
of those. Add dma_alloc(len, flags) -> vaddr (rax), paddr (rdx) and dma_free(vaddr,
len): grant contiguous, zeroed, pinned, strong-uncacheable memory in a per-process DMA
arena (PML4[228]) and hand back both addresses.

Pieces: pmm.allocContiguous(count, max_phys) finds a run of contiguous free frames
below a cap (dma_below_4g for 32-bit engines); mapUserDmaInto maps them uncacheable
(PCD|PWT) but WITHOUT device_grant, so unlike an MMIO grant these frames are real RAM
and freeSubtree returns them on teardown — a driver that dies leaks nothing. dma_free
is bounded to the DMA arena so it can never unmap the caller's stack/heap/MMIO.
dma_write_combining is accepted but falls back to coherent (WC needs PAT programming).

Runtime: runtime.dma.alloc/free (a two-return-value stub, like replyWait). New `dma`
kernel test drives the mechanism directly — contiguity, the below-4G cap, coherent
mapping, and reclaim-on-teardown (no leak). The thin syscall wrappers follow the tested
mmap/mmio_map shape and land their first real use with the first DMA driver. Suite
38/38 plus host tests.
This commit is contained in:
Daniel Samson
2026-07-10 19:43:59 +01:00
parent e7c7e7b94c
commit 125a3b4993
11 changed files with 252 additions and 1 deletions
@@ -168,6 +168,12 @@ pub fn mapUserDeviceInto(root: u64, virtual: u64, physical: u64, len: u64) void
paging.mapUserDeviceInto(root, virtual, physical, len);
}
/// Map coherent DMA RAM into address space `root`: strong-uncacheable, RW+NX, but
/// reclaimed on teardown (real RAM, not MMIO). For dma_alloc.
pub fn mapUserDmaInto(root: u64, virtual: u64, physical: u64, len: u64) void {
paging.mapUserDmaInto(root, virtual, physical, len);
}
/// Map a page into the kernel address space (non-executable). For the heap, etc.
pub fn mapPage(virtual: u64, physical: u64, writable: bool) void {
paging.map(virtual, physical, writable);
@@ -273,6 +273,30 @@ pub fn mapUserDeviceInto(pml4: u64, virtual: u64, physical: u64, len: u64) void
}
}
/// Map `[physical, physical+len)` into the user half rooted at `pml4` as **coherent
/// DMA memory**: strong-uncacheable (PCD|PWT — a device reads/writes this RAM without
/// snooping the CPU caches) but, unlike `mapUserDeviceInto`, **without** `device_grant`
/// — because these frames are real RAM from `pmm.allocContiguous`, so teardown
/// (`freeSubtree`) must return them to the allocator like any other user page. RW + NX.
/// The caller aligns `virtual`/`physical` and places `virtual` in the DMA arena.
pub fn mapUserDmaInto(pml4: u64, virtual: u64, physical: u64, len: u64) void {
const flags: u64 = present | user | writable | no_execute | pcd | pwt;
const first = physical & ~@as(u64, page_size - 1);
const last = (physical + (if (len == 0) 1 else len) - 1) & ~@as(u64, page_size - 1);
var off: u64 = 0;
while (first + off <= last) : (off += page_size) {
const v = virtual + off;
const pml4e = &tableAt(pml4)[(v >> 39) & 0x1FF];
const pdpt = descendUser(pml4e);
const pdpte = &tableAt(pdpt)[(v >> 30) & 0x1FF];
const pd = descendUser(pdpte);
const pde = &tableAt(pd)[(v >> 21) & 0x1FF];
const pt = descendUser(pde);
tableAt(pt)[(v >> 12) & 0x1FF] = ((first + off) & address_mask) | flags;
invalidate(v);
}
}
/// Create a new address space: a fresh PML4 with an empty user half and the
/// kernel's higher half shared in (copying PML4[256..512), whose entries point
/// at the kernel's PDPTs — pre-created at init and never restaled, so growth in