Built paging / the kernel's own page tables (with a TSS+IST)

docs/README.md

@@ -19,10 +19,13 @@ rather than restate it. Roughly in the order things happen at runtime:
 5. **[frame-allocator.md](frame-allocator.md) — the physical frame allocator.** The
    bitmap allocator that hands out and reclaims 4 KiB physical frames from that
    map — the primitive page tables and the heap will be built on.
-6. **[interrupts.md](interrupts.md) — interrupts and exceptions.** The GDT and IDT,
+6. **[interrupts.md](interrupts.md) — interrupts and exceptions.** The GDT, IDT and
-   the exception stubs, and the handler that reports a CPU fault in red instead of
+   TSS, the exception stubs, and the handler that reports a CPU fault in red instead
-   letting it triple-fault into a silent reset.
+   of letting it triple-fault into a silent reset.
-7. **[halting.md](halting.md) — halting.** Why a kernel can't just "exit", and
+7. **[paging.md](paging.md) — the kernel's page tables.** Building our own 4-level
+   page tables, identity-mapping the low 4 GiB, and switching CR3 off the firmware's
+   tables onto ours.
+8. **[halting.md](halting.md) — halting.** Why a kernel can't just "exit", and
    how `while (true) hlt` parks the CPU safely once there's nothing left to do.
 
 Cutting across all of these:
@@ -39,6 +42,7 @@ queries the **GOP** to pick a graphics mode ([gop.md](gop.md)), hands the kernel
 map** of physical RAM ([memory-map.md](memory-map.md)); the kernel turns that map
 into a **frame allocator** ([frame-allocator.md](frame-allocator.md)), installs
 its **descriptor tables** so CPU faults are caught ([interrupts.md](interrupts.md)),
+builds its own **page tables** and switches onto them ([paging.md](paging.md)),
 runs — its CPU-specific bits behind the [arch](arch.md) boundary — and when it has
 finished, or panics, it **halts** ([halting.md](halting.md)).
 
@@ -51,5 +55,5 @@ finished, or panics, it **halts** ([halting.md](halting.md)).
 | Shared loader↔kernel contract (`BootInfo`, `Framebuffer`, `MemoryMap`, ABI) | `src/root.zig` |
 | Physical frame allocator | `src/pmm.zig` |
 | Framebuffer text console | `src/console.zig` |
-| Arch-specific kernel code (`halt`, GDT/IDT, exception stubs, linker script) | `src/arch/x86_64/` |
+| Arch-specific kernel code (`halt`, GDT/IDT/TSS, exception stubs, page tables, linker script) | `src/arch/x86_64/` |
 | Build + `run-efi` (QEMU/OVMF) | `build.zig` |

docs/interrupts.md

@@ -37,6 +37,25 @@ means present, ring 0, 64-bit interrupt gate. `src/arch/x86_64/idt.zig` builds t
 table, points the first 32 vectors at their stubs, and loads it with `lidt`
 (`idt_flush`).
 
+## The TSS and the double-fault stack
+
+There's one more table, the **Task State Segment**. In long mode its main
+remaining job is the **Interrupt Stack Table (IST)**: an IDT gate can name an IST
+slot, and when that vector fires the CPU switches to the stack recorded there —
+*regardless* of what the interrupted stack looked like.
+
+This matters most for the **double fault** (#DF, vector 8). A #DF means the CPU
+hit a fault *while trying to deliver another fault* — very often because the
+current stack pointer is bad, so pushing the exception frame itself faulted. If
+the #DF handler then tried to push onto that same bad stack, it would fault a
+third time and **triple-fault** — an instant reset. So the #DF gate is pointed at
+**IST1**, a small dedicated stack (`src/arch/x86_64/tss.zig`) that's always valid.
+
+Bringing it up: fill in the TSS's IST1 pointer, publish the TSS through a
+descriptor in the GDT (`gdt.setTss`), and load it into the task register with
+`ltr`. The TSS descriptor is a 16-byte system descriptor spanning two GDT slots,
+which is why the GDT grew from three entries to five.
+
 ## The stubs and the trap frame
 
 On an exception the CPU pushes a small frame (SS, RSP, RFLAGS, CS, RIP) and, for
@@ -87,11 +106,13 @@ together confirm the whole path: the GDT is active (we're still executing), the
 IDT vectored to the right stub, the stub built a correct `CpuState`, and the Zig
 handler read it and reported instead of triple-faulting.
 
+Separately, pointing RSP at an unmapped address and faulting forced a **double
+fault** — reported cleanly (`double fault (vector 8)`) rather than triple-faulting
+into a reset, which only works because #DF ran on IST1. That's the proof the
+TSS/IST is wired up: the handler survived a completely broken stack.
+
 ## What's next (not done here)
 
-- **A TSS with an IST** (interrupt stack table) so the double-fault handler runs
-  on a known-good stack — important because a double fault often means the current
-  stack is unusable, and without an IST the handler would itself fault.
 - **Device interrupts**: program the local APIC and IO-APIC, wire a timer and the
   keyboard onto vectors ≥ 32, and (unlike exceptions) actually *return* from them
   with `iretq` — which `isr_common` already does.

docs/paging.md

@@ -0,0 +1,78 @@
+# Paging: the kernel's own page tables
+
+Every memory access the CPU makes goes through the **page tables**: hardware walks
+them to translate a virtual address into a physical one, and faults if there's no
+valid mapping. Up to now danos ran on the *firmware's* page tables — which live in
+memory we'd like to reclaim and which we don't control. This step builds the
+kernel's own tables and switches onto them.
+
+It's x86_64-specific (the 4-level table format is an Intel/AMD thing), so it lives
+behind the [arch](arch.md) boundary in `src/arch/x86_64/paging.zig`.
+
+## What we map, and why identity
+
+x86_64 uses **4 levels**: PML4 → PDPT → PD → PT, each a 512-entry table, with 9
+bits of the virtual address indexing each level. A leaf can be a 4 KiB page (at
+the PT level) or, with the "huge" bit, a 2 MiB page straight from the PD.
+
+For this first set of tables we **identity-map the low 4 GiB** — virtual address
+equals physical address. Identity mapping is the pragmatic bootstrap: it keeps
+everything that's already running valid across the CR3 switch without relocating a
+single thing. The kernel image (at 1 MiB), the stack, the frame allocator's
+bitmap, the framebuffer (at 2 GiB), and device MMIO all sit in the low 4 GiB, so
+one flat identity range covers the lot.
+
+Using 2 MiB pages makes the whole map tiny: 4 GiB ÷ 2 MiB = 2048 leaves, which is
+one PML4, one PDPT, and four page directories — six frames total, from the
+[frame allocator](frame-allocator.md).
+
+## Building and switching
+
+`paging.init` takes a frame allocator and:
+
+1. Allocates and zeroes a PML4. **Zeroing matters**: the frame is recycled memory,
+   and any stale non-zero entry would map a bogus region.
+2. Walks PML4 → PDPT → PD for each 2 MiB address, creating intermediate tables on
+   demand (`descend`) and writing the leaf with present + writable + huge.
+3. Loads the PML4's physical address into **CR3**, which both switches address
+   spaces and flushes the TLB in one instruction.
+
+This all works because the firmware's identity map is still active *while we
+build*, so a freshly allocated frame's physical address is usable directly as a
+pointer. After the CR3 load we're on our own tables — and since those page-table
+frames came from low RAM, they remain mapped (and thus editable) for later.
+
+## Verifying it
+
+Two temporary tests confirmed both that we switched tables and that faults are
+caught with the right detail:
+
+- **Touch an address above 4 GiB** (`0xdeadbeef000`). Under the firmware's tables
+  this *didn't* fault (they map a huge range); under ours it does:
+
+  ```
+  CPU EXCEPTION: page fault (vector 14)
+    error code : 0x2                     (write, page not present)
+    CR2 (addr) : 0x00000deadbeef000      (the faulting address)
+  ```
+
+  A #PF at exactly the unmapped address is proof we're on our own, deliberately
+  smaller, map — and that CR2 reporting works (see [interrupts.md](interrupts.md)).
+- The console keeps working *after* the switch, confirming the framebuffer, kernel
+  code, and stack are all still mapped.
+
+> Toolchain note: a volatile store to a *compile-time-constant* address tripped a
+> codegen bug in the Zig self-hosted x86_64 backend ("no encoding for mov moffs").
+> Computing the address in a runtime variable sidesteps it (register-relative
+> store). Worth remembering when poking fixed MMIO addresses.
+
+## What's next (not done here)
+
+- **A higher-half kernel**: map the kernel at a high virtual base (e.g.
+  `0xffffffff80000000`) so user address space can own the low half later.
+- **Real permissions**: today every page is writable and executable. Map code
+  read-execute, data read-write + no-execute (needs setting EFER.NXE and the NX
+  bit), and leave a guard page unmapped to catch null-ish dereferences.
+- **4 KiB pages / a general `map(virt, phys, flags)`** for fine-grained mappings,
+  and unmapping with TLB invalidation (`invlpg`).
+- **Per-address-space tables** once there are user processes.

src/arch/x86_64/cpu.zig

@@ -5,19 +5,36 @@
 //! (halt, the descriptor tables, later paging), and nothing generic.
 
 const gdt = @import("gdt.zig");
+const tss = @import("tss.zig");
 const idt = @import("idt.zig");
+const paging = @import("paging.zig");
 
 /// The saved register/trap frame passed to a fault handler.
 pub const CpuState = idt.CpuState;
 
-/// Set up the CPU's descriptor tables: our own GDT, then the IDT with exception
+/// Set up the CPU's descriptor tables: our own GDT, the TSS (with an interrupt
-/// handlers. After this a CPU fault is reported instead of triple-faulting.
+/// stack for double faults), then the IDT with exception handlers. After this a
-/// Install the fault handler (setFaultHandler) first so early faults are caught.
+/// CPU fault is reported instead of triple-faulting. Install the fault handler
+/// (setFaultHandler) first so early faults are caught.
 pub fn init() void {
     gdt.init();
+    tss.init();
     idt.init();
 }
 
+/// Build the kernel's own page tables and switch onto them. Needs a physical
+/// frame allocator; call once the frame allocator is up.
+pub fn enablePaging(allocFrame: *const fn () ?u64) void {
+    paging.init(allocFrame);
+}
+
+/// CR3 holds the physical address of the active top-level page table.
+pub fn readCr3() u64 {
+    return asm volatile ("mov %%cr3, %[out]"
+        : [out] "=r" (-> u64),
+    );
+}
+
 /// Route CPU exceptions to `handler`, which receives the trap frame and does not
 /// return. Until set, faults just halt the core.
 pub fn setFaultHandler(handler: *const fn (*const CpuState) noreturn) void {

src/arch/x86_64/gdt.zig

@@ -7,17 +7,33 @@
 /// Selectors into the table below (index * 8).
 pub const kernel_code = 0x08;
 pub const kernel_data = 0x10;
+pub const tss_selector = 0x18;
 
 /// Flat 64-bit descriptors. Base/limit are ignored in long mode; what matters is
 /// the access byte and, for code, the long-mode (L) flag.
 ///   code: present, ring 0, executable, readable, L=1  -> 0x00AF9A00_0000FFFF
 ///   data: present, ring 0, writable                   -> 0x00CF9200_0000FFFF
+/// The last two slots hold one 16-byte TSS descriptor, filled in by setTss.
 var table = [_]u64{
     0, // null descriptor (required)
-    0x00AF9A000000FFFF, // kernel code
+    0x00AF9A000000FFFF, // kernel code (0x08)
-    0x00CF92000000FFFF, // kernel data
+    0x00CF92000000FFFF, // kernel data (0x10)
+    0, // TSS descriptor low  (0x18)
+    0, // TSS descriptor high
 };
 
+/// Fill the 64-bit TSS system descriptor (two GDT slots) so the task register can
+/// point at our TSS. Type 0x89 = present, ring 0, available 64-bit TSS.
+pub fn setTss(base: u64, limit: u64) void {
+    table[3] = (limit & 0xFFFF) |
+        ((base & 0xFFFF) << 16) |
+        (((base >> 16) & 0xFF) << 32) |
+        (@as(u64, 0x89) << 40) |
+        (((limit >> 16) & 0xF) << 48) |
+        (((base >> 24) & 0xFF) << 56);
+    table[4] = (base >> 32) & 0xFFFFFFFF;
+}
+
 /// The operand `lgdt` wants: table byte-length minus one, then its address.
 const Descriptor = packed struct {
     limit: u16,

src/arch/x86_64/idt.zig

@@ -7,6 +7,7 @@
 //! interrupts (the APIC, timer, keyboard) come later.
 
 const gdt = @import("gdt.zig");
+const tss = @import("tss.zig");
 
 /// The register + trap frame the ISR stubs build on the stack, laid out so the
 /// lowest address (where RSP points when we call the handler) is the first field.
@@ -106,6 +107,9 @@ pub fn init() void {
         const stub = @extern(*const anyopaque, .{ .name = std.fmt.comptimePrint("isr{d}", .{vector}) });
         setGate(vector, @intFromPtr(stub));
     }
+    // Run the double-fault handler (vector 8) on IST1: a #DF usually means the
+    // current stack is unusable, so it needs a guaranteed-good one. See tss.zig.
+    idt[8].ist = tss.double_fault_ist;
     const descriptor = Descriptor{
         .limit = @sizeOf(@TypeOf(idt)) - 1,
         .base = @intFromPtr(&idt),

src/arch/x86_64/isr.s

@@ -33,6 +33,12 @@ idt_flush:
     lidt (%rdi)
     ret
 
+# load_tr(di = TSS selector): load the task register.
+.global load_tr
+load_tr:
+    ltr %di
+    ret
+
 # Stub for a vector the CPU does NOT push an error code for: push a dummy 0.
 .macro STUB_NOERR vec
 .global isr\vec

src/arch/x86_64/paging.zig

@@ -0,0 +1,70 @@
+//! The kernel's own 4-level page tables. Until now we've been running on the
+//! firmware's page tables, which live in memory we'd like to reclaim and which we
+//! don't control. This builds our own set, identity-mapping the low 4 GiB, and
+//! loads CR3 to switch onto them.
+//!
+//! "Identity map" means virtual address == physical address, which keeps
+//! everything already running — kernel image, stack, framebuffer, the frame
+//! allocator's bitmap, MMIO — valid across the switch without having to relocate
+//! anything. 4 GiB comfortably covers all of that (RAM low down, the framebuffer
+//! at 2 GiB, device MMIO below 4 GiB). Higher-half mapping and per-region
+//! permissions come later; this is the bootstrap.
+//!
+//! We use 2 MiB pages, so the whole map is cheap: a PML4, a PDPT, and four page
+//! directories.
+
+const KiB = 1024;
+const MiB = 1024 * KiB;
+const GiB = 1024 * MiB;
+
+const present: u64 = 1 << 0;
+const writable: u64 = 1 << 1;
+const huge: u64 = 1 << 7; // in a PD entry: this maps a 2 MiB page directly
+const addr_mask: u64 = 0x000F_FFFF_FFFF_F000; // physical address bits of an entry
+
+/// A page table is 512 64-bit entries. While building the tables the firmware's
+/// identity map is still active, so a physical frame address is usable directly.
+fn tableAt(phys: u64) *[512]u64 {
+    return @ptrFromInt(phys);
+}
+
+/// Allocate and zero a fresh page-table frame. Zeroing matters: the frame comes
+/// from previously-used memory, and any stale non-zero entry would map a bogus
+/// region.
+fn allocTable(allocFrame: *const fn () ?u64) u64 {
+    const frame = allocFrame() orelse @panic("paging: out of memory building page tables");
+    @memset(tableAt(frame)[0..], 0);
+    return frame;
+}
+
+/// Return the table an entry points at, creating it if the entry is empty.
+fn descend(entry: *u64, allocFrame: *const fn () ?u64) u64 {
+    if (entry.* & present != 0) return entry.* & addr_mask;
+    const frame = allocTable(allocFrame);
+    entry.* = frame | present | writable;
+    return frame;
+}
+
+/// Identity-map one 2 MiB page: walk PML4 -> PDPT -> PD and write the leaf.
+fn mapHugePage(pml4: u64, addr: u64, allocFrame: *const fn () ?u64) void {
+    const pml4e = &tableAt(pml4)[(addr >> 39) & 0x1FF];
+    const pdpt = descend(pml4e, allocFrame);
+    const pdpte = &tableAt(pdpt)[(addr >> 30) & 0x1FF];
+    const pd = descend(pdpte, allocFrame);
+    tableAt(pd)[(addr >> 21) & 0x1FF] = addr | present | writable | huge;
+}
+
+/// Build the tables, identity-map the low 4 GiB, and switch CR3 onto them.
+pub fn init(allocFrame: *const fn () ?u64) void {
+    const pml4 = allocTable(allocFrame);
+    var addr: u64 = 0;
+    while (addr < 4 * GiB) : (addr += 2 * MiB) {
+        mapHugePage(pml4, addr, allocFrame);
+    }
+    // Loading CR3 switches address spaces and flushes the TLB in one step.
+    asm volatile ("mov %[pml4], %%cr3"
+        :
+        : [pml4] "r" (pml4),
+        : .{ .memory = true }
+    );
+}

src/arch/x86_64/tss.zig

@@ -0,0 +1,48 @@
+//! Task State Segment and its interrupt stack. In long mode the TSS's main job
+//! is the Interrupt Stack Table: an IDT gate can name an IST entry, and the CPU
+//! switches to that stack when the exception fires — no matter how broken the
+//! interrupted stack was. We use IST1 for the double-fault handler, so a fault
+//! that happens *because* the current stack is unusable still lands on solid
+//! ground instead of triple-faulting.
+
+const gdt = @import("gdt.zig");
+
+/// x86_64 TSS. `packed` because several 64-bit fields sit at 4-byte-unaligned
+/// offsets (rsp0 at byte 4), which a normal struct would pad away.
+const Tss = packed struct {
+    reserved0: u32 = 0,
+    rsp0: u64 = 0,
+    rsp1: u64 = 0,
+    rsp2: u64 = 0,
+    reserved1: u64 = 0,
+    ist1: u64 = 0,
+    ist2: u64 = 0,
+    ist3: u64 = 0,
+    ist4: u64 = 0,
+    ist5: u64 = 0,
+    ist6: u64 = 0,
+    ist7: u64 = 0,
+    reserved2: u64 = 0,
+    reserved3: u16 = 0,
+    iomap_base: u16 = 0,
+};
+
+/// The IST slot (1-based, as the IDT gate encodes it) used for critical faults.
+pub const double_fault_ist = 1;
+
+var tss: Tss align(16) = .{};
+
+/// Dedicated stack for IST1. Static so it needs no allocator and is always valid.
+var ist1_stack: [16 * 1024]u8 align(16) = undefined;
+
+/// Loads the task register with the TSS selector. Defined in isr.s.
+extern fn load_tr(selector: u16) callconv(.c) void;
+
+/// Point IST1 at its stack, publish the TSS through the GDT, and load it into the
+/// task register. Requires the GDT to already be loaded (gdt.init first).
+pub fn init() void {
+    tss.ist1 = @intFromPtr(&ist1_stack) + ist1_stack.len; // stacks grow down
+    tss.iomap_base = @sizeOf(Tss); // == limit: no I/O permission bitmap
+    gdt.setTss(@intFromPtr(&tss), @sizeOf(Tss) - 1);
+    load_tr(gdt.tss_selector);
+}

src/main.zig

@@ -36,6 +36,7 @@ fn kmain(boot_info: *const BootInfo) noreturn {
     arch.init();
 
     con.write("danos: framebuffer console online\n");
+    con.write("danos: cpu tables online (GDT, IDT, TSS)\n");
     con.print("  resolution : {d}x{d}\n", .{ fb.width, fb.height });
     con.print("  pitch      : {d} bytes\n", .{fb.pitch});
     con.print("  format     : {s}\n", .{@tagName(fb.format)});
@@ -81,6 +82,11 @@ fn kmain(boot_info: *const BootInfo) noreturn {
     if (f2) |p| pmm.free(p);
     con.print("  after free : {d} frames free\n", .{pmm.stats().free_frames});
 
+    // Switch off the firmware's page tables onto our own.
+    arch.enablePaging(pmm.alloc);
+    con.print("\ndanos: paging enabled\n", .{});
+    con.print("  page tables: CR3 = 0x{x:0>16}\n", .{arch.readCr3()});
+
     con.write("\nkernel initialised; nothing left to do, halting.\n");
 
     arch.halt();