re-org docs

2026-07-23 00:25:34 +01:00
parent 52d6e372fd
commit 757c6f14c3
52 changed files with 156 additions and 156 deletions
@@ -0,0 +1,182 @@
+# The memory map
+
+Before a kernel can manage memory, it has to *know what memory exists*: which
+physical address ranges are real RAM it may use, and which are firmware, hardware
+registers, or already occupied. That inventory is the **memory map**, and the
+firmware is the only thing that knows it. This page covers how danos gets that map
+from the firmware and hands it to the kernel — deliberately without dragging UEFI
+into the kernel.
+
+## Why not just pass UEFI's map through?
+
+UEFI hands the loader a perfectly good memory map. The tempting shortcut is to
+forward it to the kernel as-is. We don't, for two reasons:
+
+1. **It would tie the kernel to UEFI.** The kernel would compare against UEFI's
+   memory-type numbers and walk the array using UEFI's variable descriptor stride.
+   That's UEFI vocabulary bleeding across the handoff — and danos wants to boot on
+   systems that have no UEFI at all (a Raspberry Pi describes its memory with a
+   *device tree* instead). See [architecture.md](architecture.md) for the same "keep the kernel
+   platform-agnostic" principle applied to CPU code.
+2. **We already established the better pattern.** The loader doesn't hand the
+   kernel a raw UEFI GOP either — [`queryFramebuffer`](gop.md) converts it to
+   danos's own `Framebuffer`. The memory map follows the same discipline.
+
+So the boundary is: **each boot path translates its native memory description into
+danos's own neutral format, and the kernel only ever sees that.**
+
+## The neutral format
+
+Defined in `system/boot-handoff.zig`, the shared loader↔kernel contract:
+
+```zig
+pub const MemoryKind = enum(u32) {
+    usable,       // free RAM the kernel may allocate
+    reserved,     // firmware / kernel image / boot stack — real RAM, never hand out
+    acpi_tables,  // parse, then reclaim
+    acpi_nvs,     // preserve across sleep
+    mmio,         // device registers / reserved address space — not RAM at all
+};
+
+pub const MemoryRegion = extern struct {
+    base: u64,    // physical start
+    pages: u64,   // length in page_size (4 KiB) units
+    kind: MemoryKind,
+    _pad: u32 = 0,
+};
+
+pub const MemoryMap = extern struct {
+    regions: usize, // pointer to a [len]MemoryRegion
+    len: usize,
+};
+```
+
+`MemoryKind` is danos's *own* vocabulary — not UEFI's ~15 types, just the
+distinctions the kernel actually acts on. And because danos defines `MemoryRegion`
+itself, `@sizeOf` is authoritative: the kernel walks a plain `[]MemoryRegion` with
+no variable-stride subtlety (that stride problem is a UEFI-ism, and it stays in the
+loader).
+
+`BootInformation` carries it alongside the framebuffer (trimmed here to the
+fields this page is about — the full struct has since grown the kernel's
+PT_LOAD segments, the ACPI RSDP, and the initial-ramdisk span):
+
+```zig
+pub const BootInformation = extern struct {
+    framebuffer: Framebuffer,
+    memory_map: MemoryMap,
+    // ...kernel_segments, acpi_rsdp, initial_ramdisk_base/len
+};
+```
+
+## The loader side (UEFI)
+
+Two functions in `boot/efi.zig`: `exitBootServices` calls `convertMemoryMap`,
+which runs `classify` on each descriptor:
+
+- **`classify`** maps each UEFI descriptor to a `MemoryKind`:
+  `conventional_memory` **and** `boot_services_code`/`boot_services_data → usable`;
+  `acpi_reclaim_memory → acpi_tables`; `acpi_memory_nvs → acpi_nvs`;
+  `memory_mapped_io`/`memory_mapped_io_port_space → mmio`; **everything
+  else → reserved** (the safe default). Our own `loader_data` — the kernel image and
+  these buffers — falls into `reserved`.
+
+  Folding boot-services memory into `usable` is deliberate: we've already called
+  ExitBootServices, so it's free RAM now, and doing the classification *here* (in
+  the loader) means the kernel never learns about a UEFI-specific "reclaimable"
+  state — it just sees usable RAM. The one catch is that our stack lives in
+  boot-services memory and the kernel starts out running on it, so
+  `convertMemoryMap` keeps the single region containing the current stack pointer
+  `reserved`. All the boot-protocol knowledge stays on the loader side of the
+  boundary; the kernel's frame allocator has no idea any of this happened.
+
+  One subtlety: **a region that isn't writeback-cacheable (the descriptor's `wb`
+  attribute) is classified `mmio` regardless of type.** UEFI overloads
+  `reserved_memory_type` for both reserved RAM *and* reserved address-space windows
+  (PCIe config space, device BARs); the cache attribute is what actually tells them
+  apart, since only real RAM is writeback-cacheable. Without this, a QEMU q35 guest
+  reports ~12 GiB of "reserved" that is really a PCIe address hole near the 1 TB
+  mark — not memory at all.
+- **`convertMemoryMap`** walks the UEFI descriptors (striding by
+  `descriptor_size`, *not* `@sizeOf`), classifies each, and writes danos
+  `MemoryRegion`s into an output buffer, coalescing adjacent same-kind regions.
+
+### The ordering that makes it correct
+
+This is the fiddly part, dictated by two UEFI rules: you can only allocate memory
+*before* `ExitBootServices`, and the memory map is only final *at* the moment you
+exit (its "key" proves you've seen the latest state). So `exitBootServices` does,
+per attempt:
+
+1. `getMemoryMapInfo` to size things, then `allocatePool` **two** LoaderData
+   buffers — one for the raw UEFI map, one for the converted regions. Allocating
+   now, before exit, is mandatory.
+2. `getMemoryMap` then `exitBootServices(key)`. If either fails (allocating can
+   perturb the map and invalidate the key), free both buffers and retry.
+3. **After** the exit succeeds, convert. Conversion is pure computation on memory
+   we already hold — no boot-services calls — so it's safe once services are gone.
+
+Both buffers are `LoaderData`, which survives `ExitBootServices`, so the converted
+array the kernel is pointed at stays valid. (The raw UEFI buffer is just scratch
+for the conversion.)
+
+## The kernel side
+
+The kernel receives a plain array and reads it with zero UEFI knowledge:
+
+```zig
+const mm = boot_information.memory_map;
+const regions = @as(
+    [*]const boot_handoff.MemoryRegion,
+    @ptrFromInt(boot_handoff.physicalToVirtual(mm.regions)),
+)[0..mm.len];
+for (regions) |r| {
+    if (r.kind == .usable) usable_pages += r.pages;
+}
+```
+
+(`mm.regions` is a physical address, so it's dereferenced through the physmap —
+`physicalToVirtual` — since the kernel no longer runs under the loader's
+identity map.)
+
+`kmain` summarises the map to prove the handoff works. Booted in QEMU with
+128 MiB, it reports:
+
+```
+/system/kernel: physical memory
+  total RAM  : 0.12 GiB (127 MiB) - RAM the firmware reported
+  usable     : 121 MiB - free RAM (incl. reclaimed boot-services memory)
+  reserved   : 6 MiB - kernel image, boot stack, ACPI, runtime services
+  regions    : 28 - entries in the firmware memory map
+```
+
+`usable` is ~121 of ~127 MiB because the loader already folded the boot-services
+memory into it — so the frame allocator gets it all with no special step. The ~6 MiB
+`reserved` is the kernel image, the boot stack's region, ACPI, and runtime services.
+`total` counts only writeback-cacheable RAM, so the ~12 GiB PCIe address hole is
+excluded (it's `mmio`), and the RAM categories summing back to the firmware's total
+is the sanity check that nothing was dropped.
+
+## How Raspberry Pi will fit
+
+No UEFI there, but the boundary is unchanged. The Pi's firmware jumps into the
+kernel with a **device-tree blob**; the AArch64 entry code will parse its
+`/memory` and `/reserved-memory` nodes and produce the *same* `MemoryRegion`
+array. The kernel's memory code — the frame allocator and everything above it —
+never knows the difference.
+
+## What's next
+
+This page is plumbing plus classification only. The map's first consumer, the
+**physical frame allocator**, is built directly on the `usable` regions here — which
+already include the reclaimed boot-services memory the loader folded in (see
+[frame-allocator.md](frame-allocator.md)). Of the two items once listed here, one is done:
+
+- Freeing the `reserved` `loader_data` (these boot-time buffers) once the kernel
+  is done reading the map — still open: the frame allocator's bitmap tracks those
+  frames so they can be freed, but nothing frees them yet.
+- Capturing the ACPI RSDP from the UEFI configuration table before exit (the same
+  "grab it before ExitBootServices" pattern) — done: the loader stows it in the
+  boot handoff, and ACPI parsing consumes it from there ([acpi.md](acpi.md)).
+
+See the roadmap in [efi.md](efi.md) for where this sits in the boot flow.