Daniel Samson
8754d4e46a
The source layout now mirrors the runtime filesystem hierarchy
(docs/danos-file-system-hierarchy-FSH.md): what lives under system/ in the
source is what a running danos represents under /system. Each service and
driver is a sub-project directory that is its own Zig module — cross-project
references go by module name, never by a path into another project's files.
Moves (all git mv, history preserved):
- src/ -> system/ (danos internals; the self-representation)
root.zig -> danos.zig (the kernel<->user contract module)
kernel/arch/ -> kernel/architecture/ (arch -> architecture)
device/ -> devices/ (what /system/devices reflects)
boot/ -> /boot (the loaders, top level)
- sbin/ -> split by role:
init, vfs -> system/services/<name>/<name>.zig
hpetd, busd -> system/drivers/<name>/<name>.zig
vfs-test -> system/services/vfs/vfs-test.zig (inside the vfs project)
- lib/ -> library/runtime/ (room for other libraries beside runtime)
The VFS wire protocol becomes its own module, system/services/vfs/protocol.zig
("vfs-protocol"): the vfs sub-project exposes its interface, and the runtime's
file layer imports it by name. First instance of the "protocol module" pattern
(docs/driver-model.md); usb/block will expose theirs the same way.
Also: fix a naming-standard violation in the protocol — Op -> Operation (and
req -> request, _pad -> _padding). Docs updated: /system/services added to the
FHS doc, a repository-layout section added to the docs index, and stale source
paths swept across comments and docs.
Runtime boot paths are unchanged (the bootloader still loads /sbin/init);
aligning the runtime filesystem to the FHS is a separate follow-up. Suite 35/35
plus host tests green.
76 lines
3.7 KiB
Markdown
76 lines
3.7 KiB
Markdown
# The kernel heap
|
|
|
|
The [frame allocator](frame-allocator.md) hands out fixed 4 KiB physical frames;
|
|
the [VMM](paging.md) maps pages into virtual addresses. The **kernel heap** sits on
|
|
top of both to provide what the rest of the kernel actually wants: `alloc(n)` /
|
|
`free(p)` for arbitrary byte sizes. It's the first real consumer of `map()`, and
|
|
the thing that unlocks dynamic data structures — lists, hash maps, driver state,
|
|
eventually a process table.
|
|
|
|
It's generic kernel code (`system/kernel/heap.zig`): the allocator logic is
|
|
architecture-neutral, using `arch.mapPage` and the frame allocator underneath.
|
|
|
|
## A growable free-list allocator
|
|
|
|
The algorithm is a classic **first-fit free list**:
|
|
|
|
- The heap owns a virtual region. Free space is tracked as an **address-ordered
|
|
singly linked list** of free blocks; each block begins with a 16-byte header
|
|
(`size`, and a `next` link used while free).
|
|
- **alloc(n)** walks the list for the first block big enough. If the block is much
|
|
larger it's **split** — the front becomes the allocation, the remainder stays
|
|
free. If nothing fits, the heap **grows** (below) and the search retries.
|
|
- **free(p)** finds the block header just before `p` and inserts it back into the
|
|
list, **coalescing** with the physically adjacent free blocks on either side so
|
|
the space can be reused as one region rather than fragmenting away.
|
|
|
|
Allocations are 16-byte aligned; larger alignments aren't supported yet (the
|
|
`std.mem.Allocator` `alloc` returns `null` for them).
|
|
|
|
## Growing on demand
|
|
|
|
The heap lives in the **higher half** of the address space (virtual base
|
|
`0xFFFF_8000_0000_0000`) — unmapped, well clear of the identity-mapped low half,
|
|
and leaving the low half free for a future user address space. (That base is
|
|
x86_64-canonical; another architecture would pick its own.)
|
|
|
|
When the free list can't satisfy a request, `grow` extends the mapped region: it
|
|
pulls fresh frames from the [frame allocator](frame-allocator.md) and `map`s each
|
|
onto the end of the heap, then adds the new span as a free block (coalescing with
|
|
the current tail). So the heap starts at one page and expands page-by-page as
|
|
demand requires, up to a cap. This is exactly what the VMM's on-demand `map` was
|
|
built for.
|
|
|
|
## A std.mem.Allocator
|
|
|
|
The heap is exposed as a **`std.mem.Allocator`** (`heap.allocator()`), Zig's
|
|
standard allocator interface. That's a deliberate multiplier: it means the whole of
|
|
Zig's standard library — `ArrayList`, `AutoHashMap`, `std.fmt.allocPrint`, and the
|
|
rest — works directly on the kernel heap, no bespoke containers required.
|
|
|
|
## Verifying it
|
|
|
|
The `heap` test (see [testing.md](testing.md)) exercises the allocator end to end:
|
|
|
|
```
|
|
[PASS] alloc 4096 bytes
|
|
[PASS] heap memory is writable and reads back
|
|
[PASS] freed block is reused <- free list + coalescing works
|
|
[PASS] many allocations (heap growth) stay valid <- grow() maps fresh frames
|
|
[PASS] std.ArrayList on the kernel heap <- std containers work on it
|
|
```
|
|
|
|
The "freed block is reused" check (free then re-alloc returns the same address) is
|
|
the proof that free and the free list actually work, not just alloc; "heap growth"
|
|
forces allocation past the initial page so `grow`/`map` runs; and the `ArrayList`
|
|
check is the std-integration payoff.
|
|
|
|
## What's next (not done here)
|
|
|
|
- **Thread/interrupt safety.** The heap assumes a single caller — no lock yet.
|
|
It's safe now (nothing allocates from interrupt handlers), but threads or an
|
|
allocating IRQ handler will need a lock (or `cli` around the critical section).
|
|
- **Larger alignments** than 16 (for page-aligned buffers, DMA regions).
|
|
- **`resize`/`remap` in place**, so growing an `ArrayList` needn't always copy.
|
|
- **Reclaiming empty tail pages** back to the frame allocator when the heap shrinks.
|