danos/docs/os-development/heap.md

The kernel heap

The frame allocator hands out fixed 4 KiB physical frames; the VMM maps pages into virtual addresses. The kernel heap sits on top of both to provide what the rest of the kernel actually wants: alloc(n) / free(p) for arbitrary byte sizes. It's the first real consumer of map(), and the thing that unlocks dynamic data structures — lists, hash maps, driver state, eventually a process table.

It's generic kernel code (system/kernel/heap.zig): the allocator logic is architecture-neutral, using architecture.mapPage and the frame allocator underneath.

A growable free-list allocator

The algorithm is a classic first-fit free list:

• The heap owns a virtual region. Free space is tracked as an address-ordered singly linked list of free blocks; each block begins with a 16-byte header (size, and a next link used while free).
• alloc(n) walks the list for the first block big enough. If the block is much larger it's split — the front becomes the allocation, the remainder stays free. If nothing fits, the heap grows (below) and the search retries.
• free(p) finds the block header just before p and inserts it back into the list, coalescing with the physically adjacent free blocks on either side so the space can be reused as one region rather than fragmenting away.

Allocations are 16-byte aligned; larger alignments aren't supported yet (the std.mem.Allocator alloc returns null for them).

Growing on demand

The heap lives in the higher half of the address space (virtual base 0xFFFF_8000_0000_0000) — unmapped, well clear of the low half, which belongs to user space (unmapped in the kernel's own tables; per-process user address spaces now map into it). (That base is x86_64-canonical; another architecture would pick its own.)

When the free list can't satisfy a request, grow extends the mapped region: it pulls fresh frames from the frame allocator and maps each onto the end of the heap, then adds the new span as a free block (coalescing with the current tail). So the heap starts at one page and expands page-by-page as demand requires, up to a cap. This is exactly what the VMM's on-demand map was built for.

A std.mem.Allocator

The heap is exposed as a std.mem.Allocator (heap.allocator()), Zig's standard allocator interface. That's a deliberate multiplier: it means the whole of Zig's standard library — ArrayList, AutoHashMap, std.fmt.allocPrint, and the rest — works directly on the kernel heap, no bespoke containers required.

Verifying it

The heap test (see testing.md) exercises the allocator end to end:

[PASS] alloc 4096 bytes
[PASS] heap memory is writable and reads back
[PASS] freed block is reused              <- free list + coalescing works
[PASS] many allocations (heap growth) stay valid   <- grow() maps fresh frames
[PASS] std.ArrayList on the kernel heap    <- std containers work on it

The "freed block is reused" check (free then re-alloc returns the same address) is the proof that free and the free list actually work, not just alloc; "heap growth" forces allocation past the initial page so grow/map runs; and the ArrayList check is the std-integration payoff.

What's next (largely still true)

• Thread/interrupt safety — overtaken by the big kernel lock: SMP arrived with a single kernel lock taken at every kernel entry, which serializes all heap access. The heap still has no lock of its own, and needs none unless the big lock is ever split.
• Larger alignments than 16 — still unsupported; page-aligned and DMA buffers come straight from the frame allocator instead.
• resize/remap in place — still not done; growing an ArrayList copies.
• Reclaiming empty tail pages — still not done; the heap only ever grows.