daniel
/
danos
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

threads: design doc + /loop build plan 

Add docs/threading.md (native runtime.Thread mirroring std.Thread over a
private thread ABI) and docs/threading-plan.md (6 milestones, each with a
serial-checkable gate + guardrail, plus an unattended /loop execution
contract). Index both in docs/README.md.
This commit is contained in:
Daniel Samson 2026-07-20 20:39:24 +01:00
parent d26515706e
commit 6e8b02d771
3 changed files with 538 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/README.md
View File

@ -104,6 +104,12 @@ Start with the north star:
port to **one seam** (`std.os.danos`), so we build `runtime.os` (→ that seam) plus a
thin `runtime.fs`, retire the `posix` shim, and follow a phased path to
`zig build-exe hello.zig` running on danos — **not** Linux-ABI emulation.
- **[threading.md](threading.md) — threads, the std-shaped way.** A design note (not
built yet) on `runtime.Thread`: a type that mirrors `std.Thread`'s API (spawn/join,
Mutex/Condition) over a **private** thread ABI — several tasks sharing one address
space via a `thread_spawn` syscall, futex-backed blocking, aspace refcounting. Why
it's the native type and not literal `std.Thread` (the [private ABI](syscall.md)),
and why threads stay a narrow opt-in against the [resilience](resilience.md) default.
- **[vdso.md](vdso.md) — the vDSO, the public system-call boundary.** A design note
(not built yet) on keeping `abi.zig` genuinely private: a kernel-supplied, C-ABI
entry blob mapped into every process as the *only* way into the kernel — so the

219
docs/threading-plan.md Normal file
View File

@ -0,0 +1,219 @@
# Threading — build plan (`runtime.Thread` over a private thread ABI)
The ordered, checkpointable build-out for [threading.md](threading.md). Each milestone
lands on its own and ends in a **verifiable gate** — shaped for a `/loop` run, like
[display-v2-plan.md](display-v2-plan.md). Read threading.md first for the *why*.
## Locked decisions (do not relitigate)
- **`runtime.Thread` mirrors `std.Thread`'s API; the implementation is danos-native.**
Not literal `std.Thread` — that would break the [private ABI](syscall.md).
- **Threads are a narrow, per-binary opt-in.** Default concurrency stays process + IPC
([resilience.md](resilience.md)); only a service that asks is built
`single_threaded = false`.
- **Blocking is futex-backed, never spin-backed** — waiters park in the kernel so an
idle core still halts ([halting.md](halting.md)).
- **New syscalls are private**: extend [abi.zig](../system/abi.zig) `SystemCall` after
`shm_physical = 36` (`thread_spawn = 37`, `thread_exit = 38`, `futex_wait = 39`,
`futex_wake = 40`) + a `library/runtime` wrapper; user code never names a number.
- **Restart granularity stays the process** — a faulting thread kills its process; the
supervisor restarts the process, which respawns its threads.
## Conventions
Follow [coding-standards.md](coding-standards.md): spell out non-acronym abbreviations,
kebab-case file names, no `Co-Authored-By` trailers. New user binaries go through
`addUserBinary` (with the new `threaded` flag where a binary spawns threads) and get
packed into the initial-ramdisk; new syscalls extend [abi.zig](../system/abi.zig)
`SystemCall` + a `library/runtime` wrapper; test services live beside the code they
exercise and register a `ServiceId` if they must be looked up.
## How to verify along the way
**Every gate is serial-checkable — no screenshots** (this plan runs unattended). A
thread proves it ran by writing to **shared memory** the parent reads back, and proves
parallelism by stamping the **core index** it ran on (like the `smp`/`affinity` cases).
- `zig build test` — host unit tests (closure packing, mutex state machine, futex
wrapper encodings).
- `python3 test/qemu_test.py <case>` — boots the kernel in QEMU; asserts on serial
markers. Thread cases set `smp: true` (real parallelism) and bump `mem` (they boot
the process/scheduler stack); each milestone **adds its case to `CASES`** so its gate
is runnable.
- **Guardrail every milestone:** the concurrency-sensitive existing cases stay green —
`smoke`, `sched`, `priority`, `smp`, `affinity`, `process`, `process-kill`,
`supervision`, `fault-recovery`, `vfs-client-death`, `ipc`/`ipc-cap`,
`display-service`. A threading change that regresses those is rejected.
## Unattended execution (the loop contract)
This plan runs to completion **without human input**. Every design choice is already
fixed in *Locked decisions*; the checkboxes are the only state. A loop iteration must:
1. **Resume** at the first milestone that still has an unchecked `- [ ]`. (All earlier
milestones are done — do not revisit them.)
2. **Work on a branch.** On the first iteration, branch off `main` (e.g. `threading`);
never commit threading work to `main`. All work stays local — **do not push**.
3. **Implement** every unchecked item in that milestone, including adding its
`-Dtest-case` to `CASES` in [test/qemu_test.py](../test/qemu_test.py) (with
`smp: true` / a `mem` bump where noted) so the gate is runnable.
4. **Run the gate**: `python3 test/qemu_test.py <case>`, then the full **guardrail
set**, then `zig build` (clean) and `zig build test` (green).
5. **Decide, do not ask:**
- **Green** = the milestone's case prints its stated marker(s) and reports `PASS`,
the whole guardrail set passes, `zig build` is clean, and host tests are green.
→ tick this milestone's boxes **and** its `**Gate:**`-referenced case, `git commit`
(`threads(M<n>): <summary>`, no `Co-Authored-By` trailer per
[coding-standards.md](coding-standards.md)), and continue to the next milestone in
the same iteration if budget remains; otherwise let the loop re-fire.
- **Red** = anything above fails. Diagnose from the captured serial log
(`zig-out/qemu-test/<case>-failed-serial.log`) and fix in place, then re-run — up to
**3 fix attempts** for that gate. A concurrency case that fails then passes on a
bare re-run is **flaky, not green**: re-run it **twice more** and treat green only
if it passes all; otherwise fix the race (a real threading bug), don't paper over
it.
6. **A genuinely ambiguous fork is not a stop.** Pick the option most consistent with
[threading.md](threading.md)'s *Locked decisions*, note the choice in the commit
message, and continue. Do not pause for confirmation on in-scope, reversible work —
this plan is that authorization.
**The only stop conditions:**
- **Done** — every milestone box is checked (M1–M6), `zig build` clean, whole
`thread-*` suite + guardrail green. Update threading.md's status line to "built" (that
is M6's own task) and stop.
- **Blocked** — a gate is still red after 3 fix attempts, or a step needs something
outside the repo (a toolchain change, new hardware, a decision no locked decision
covers). Append `> **BLOCKED (M<n>):** <what failed, what was tried, the serial
marker missing>` under that milestone, commit the WIP on the branch, and stop. Do not
thrash further and do not silently skip the milestone.
Nothing else warrants stopping — not "should I proceed?", not "is this right?". The
checkboxes + git history are the resumable record; the next iteration picks up from the
first unchecked box.
---
## M1 — Address-space refcount (kernel foundation, no API, no behaviour change)
The one invariant change threads require, landed and proven **before** anything shares
an address space. Today aspace is 1:1 with a task and teardown destroys it on any user
task's exit; make destruction happen on the **last** exit.
- [ ] A refcount keyed by the address-space root: `createAddressSpace`
([process.zig](../system/kernel/process.zig)) sets it to 1; a helper
`retainAspace`/`releaseAspace` adjusts it under the big kernel lock.
- [ ] Task teardown ([scheduler.zig](../system/kernel/scheduler.zig), the
`destroyAddressSpace(t.aspace)` path) decrements and only destroys at **zero**.
- [ ] `-Dtest-case=aspace-refcount`: spawn and exit several processes in sequence and
assert the frame allocator's free count returns to the pre-spawn **baseline**
(each aspace destroyed exactly once — no leak, no double-free).
**Gate:** `python3 test/qemu_test.py aspace-refcount` logs `aspace-refcount: frames
reclaimed to baseline ok`, and the full guardrail set passes unchanged (the reframing
is invisible until an aspace is actually shared).
## M2 — `thread_spawn` + `thread_exit`: a thread runs in the shared address space
Spawn only — no join yet. Prove a second task executes in the **caller's** address
space and exits cleanly.
- [ ] [abi.zig](../system/abi.zig): `thread_spawn = 37`, `thread_exit = 38`. Handlers
in process.zig; `thread_spawn` calls the `spawnUserLocked` path with the caller's
aspace (`retainAspace`), `entry`, `user_sp`; `thread_exit` marks the task dead,
hands back its user-stack range, and `releaseAspace`s.
- [ ] `library/runtime/thread.zig` (barrel-exported as `runtime.Thread`): `spawn` maps
a stack (`mmap`), heap-allocates the closure `{fn, args, done}`, writes the
closure pointer to the stack top, calls `thread_spawn(&threadTrampoline, top,
closure)`. `threadTrampoline` reads the closure, calls the function, calls
`thread_exit`.
- [ ] `addUserBinary` gains `threaded: bool = false` → `single_threaded = false`; a
`thread-test` service (threaded) is the first opt-in binary.
- [ ] `-Dtest-case=thread-spawn`: the parent spawns one thread that writes a sentinel
to a **shared** global and sets `done`; the parent bounded-polls `done`, then
asserts the shared global holds the sentinel (same address space) and frames
return to baseline (thread exit released nothing extra — refcount held).
**Gate:** `python3 test/qemu_test.py thread-spawn` logs `thread: child ran in shared
aspace ok`; guardrail set green.
## M3 — `join` + `detach` + real parallelism
- [ ] `join` over the existing exit-notification path
([process-lifecycle.md](process-lifecycle.md)): `spawn` passes a per-thread
`exit_endpoint`; `join` blocks in `ipc_reply_wait` until the child-exit notice for
that `tid`, then `munmap`s the stack. `detach` relinquishes the join right; the
reaper reclaims a detached thread's stack + slot on exit.
- [ ] `runtime.Thread.join` / `detach` / `getCurrentId` (id = kernel task id).
- [ ] `-Dtest-case=thread-join` (`smp: true`): the parent spawns N threads that each do
K `@atomicRmw`-increments on a shared counter and stamp the core index they ran
on; the parent joins all N and asserts `counter == N*K` **and** `distinct cores >
1` (genuine cross-core parallelism). A detached thread sub-check confirms no leak.
**Gate:** `python3 test/qemu_test.py thread-join` logs `thread: N joined, counter=N*K,
cores=<>1>`; guardrail set green (esp. `smp`, `affinity`, `process-kill`).
## M4 — Futex: the one blocking primitive
- [ ] [abi.zig](../system/abi.zig): `futex_wait = 39`, `futex_wake = 40`. Kernel
wait-queue keyed by `(aspace_root, vaddr)`; `futex_wait(addr, expected, timeout)`
parks the task iff `*addr == expected` (re-checked under the lock) and returns on
wake or timeout; `futex_wake(addr, count)` moves up to `count` waiters back to
ready. No spinning — a parked waiter leaves its core free to `hlt`.
- [ ] `runtime.Thread.Futex` (`wait` / `timedWait` / `wake`) over the wrappers.
- [ ] `-Dtest-case=thread-futex`: thread A prints `waiting`, `futex_wait`s on a word;
thread B sets the word and `futex_wake`s; A prints `woke`. Serial order
`waiting → waking → woke` proves the kernel handoff (not a spin). A second check
confirms `timedWait` returns `error.Timeout` when nobody wakes it.
**Gate:** `python3 test/qemu_test.py thread-futex` logs `thread: futex handoff ordered
ok` and `thread: futex timeout ok`; guardrail set green.
## M5 — `Mutex` + `Condition` + `Semaphore`
- [ ] `runtime.Thread.Mutex` (atomic fast path, `futex_wait`/`wake` slow path),
`Condition` (`wait`/`timedWait`/`signal`/`broadcast`), `Semaphore` — the same
state machines `std.Thread` uses, ported onto our `Futex`. Host unit tests for the
lock/unlock/wait state transitions.
- [ ] Migrate `join` to a futex **completion word** (the std shape) — drops the
per-thread endpoint from M3.
- [ ] `-Dtest-case=thread-mutex` (`smp: true`): a bounded producer/consumer over
`Mutex` + `Condition` moves K items across cores; assert the final tally is
exactly K with **no lost wakeups** (consumer never misses an item, producer never
overruns the bound), and the consumer reaches a `parked` marker (it blocked, it
didn't spin).
**Gate:** `python3 test/qemu_test.py thread-mutex` logs `thread: produced/consumed K,
no lost wakeups`; `zig build test` covers the mutex/condition state machines; guardrail
set green.
## M6 — TLS, `getCurrentId` polish, docs, and CI wiring
- [ ] Per-thread TLS: `thread_spawn` sets `fs.base` to a runtime-allocated per-thread
block; enough for `getCurrentId` and a `threadlocal` slot. Only expanded if a
consumer needs full `threadlocal`.
- [ ] `RwLock` / `WaitGroup` if a consumer wants them (else deferred).
- [ ] All `thread-*` cases wired into [test/qemu_test.py](../test/qemu_test.py);
threading.md status line updated to "built"; a short worked example in the doc.
- [ ] `-Dtest-case=thread-tls`: two threads read `getCurrentId` (distinct) and each
writes its own `threadlocal` slot without cross-talk.
**Gate:** `python3 test/qemu_test.py thread-tls` logs `thread: per-thread id + tls ok`;
the whole `thread-*` suite plus the full guardrail set pass; default `zig build` is
clean and `zig build test` is green.
---
## Deferred (explicitly not in this plan)
- **Cross-process shared-memory futex** — the `(aspace, vaddr)` key can become a
physical-address key so two processes share a futex through an [shm](display-v2.md)
region. Not needed for intra-process threads.
- **Per-thread priorities / affinity distinct from the process** — threads inherit the
process priority ([scheduling.md](scheduling.md)); revisit only if it earns its keep.
- **Per-thread signal delivery** — signals stay process-scoped
([process-lifecycle.md](process-lifecycle.md)).
- **A `pthread`/POSIX surface** — the API is `std.Thread`-shaped Zig, nothing more.
- **A real `std.Thread` backend** — arrives with self-hosting
([zig-self-hosting.md](zig-self-hosting.md)); it sits on these same primitives, so it
swaps the impl under `runtime.Thread`, not the call sites.

313
docs/threading.md Normal file
View File

@ -0,0 +1,313 @@
# Threading: `runtime.Thread`, a std-shaped API over a private thread ABI
A design note (not built yet) for giving danos **threads** — several tasks sharing
one address space — through a `runtime.Thread` type that mirrors the shape of Zig's
`std.Thread` while keeping every kernel entry behind the [runtime](../library/runtime).
It is forward-looking, like [zig-self-hosting.md](zig-self-hosting.md) and
[vision.md](vision.md): it fixes the decisions now so the code we write bends toward
them. The analysis is against **Zig 0.16** (the pinned toolchain); `std.Thread`'s
internals move between releases, so treat upstream shapes as "0.16.x."
## The win condition
A danos service can write
```zig
const t = try runtime.Thread.spawn(.{}, worker, .{ctx});
// ... do other work concurrently ...
t.join();
```
and get real parallelism across cores — with `runtime.Thread.Mutex`,
`runtime.Thread.Condition`, and `runtime.Thread.Semaphore` available for
coordination — **without any code path reaching the kernel except through the
runtime**. The call sites read exactly like `std.Thread`, so the day danos becomes a
real Zig target (see [self-hosting](#the-self-hosting-endgame)) we swap the
implementation underneath, not the API above.
## Locked decisions (do not relitigate)
- **We build `runtime.Thread`, not literal `std.Thread`.** It mirrors std's *API and
features*; the implementation underneath is danos-native. See
[Why not literal std.Thread](#why-not-literal-stdthread).
- **Threads are a narrow, opt-in capability — not the default concurrency tool.** The
default for resilience stays **process + IPC** ([resilience.md](resilience.md),
[ipc.md](ipc.md)). See [Where threads fit](#where-threads-fit-the-resilience-tension).
- **Blocking synchronization is futex-backed, never spin-backed.** Waiters sleep in
the kernel so an idle core still halts ([halting.md](halting.md)).
- **Per-binary opt-in to multi-threaded codegen.** Only a service that asks for
threads is built `single_threaded = false`; the rest stay lean and single-threaded.
- **The thread ABI is private.** New syscalls extend [abi.zig](../system/abi.zig)
`SystemCall` and are reached only through `library/runtime` wrappers, exactly like
every other danos syscall ([syscall.md](syscall.md)) — numbers stay renumberable.
## Why not literal `std.Thread`
danos's ABI invariant is that the **runtime is the sole holder of the syscall ABI**,
and that ABI is private and renumberable ([syscall.md](syscall.md) — "unstable
private ABI"). That is a security and evolvability asset: no compiled binary can
hardcode a syscall number, and the kernel can renumber freely because only the
runtime — rebuilt in lockstep — knows the mapping.
`std.Thread` is incompatible with that invariant on two counts:
1. **It selects its backend from `builtin.os.tag`, and issues syscalls directly.**
danos targets `.os_tag = .freestanding` ([build.zig](../build.zig)), for which
`std.Thread` resolves to an unsupported stub that `@compileError`s. Adding a real
backend would either bake danos syscall numbers into std (breaking ABI privacy and
renumbering) or fork std to route back through the runtime — a permanent rebase
cost that buys nothing the native type doesn't.
2. **Our user binaries are built `single_threaded = true`** ([build.zig](../build.zig)
`addUserBinary`), which compiles threading out entirely and makes atomics and TLS
single-threaded. Threads need this flipped per binary regardless.
So we take the *shape* of `std.Thread`, not the *type*. The cost of replicating the
surface (spawn/join/Mutex/Condition) is small; the cost of the std type is the ABI
invariant.
## Where threads fit: the resilience tension
Threads are in genuine tension with a resilience-first microkernel, and it is worth
being explicit so we do not reach for them by reflex.
The reason danos pays for a microkernel is **fault isolation**
([resilience.md](resilience.md)): a component corrupts its own address space, faults,
and is **restarted** without touching anyone else — because the boundary *is* the
address space. Threads deliberately remove that boundary *within* a process:
- Threads share one address space, so one thread's stray write corrupts them all —
there is no isolation **between** threads.
- Threads share fate: a fault in any thread, or a "kill the process" decision, takes
down **all** of them. Restartability lives at the process level, not the thread
level.
- Shared mutable state reintroduces data races — the failure class the
isolate-and-message model was chosen to avoid.
**Therefore:** the default answer to "make X concurrent" stays *another process over
IPC* (isolated, independently restartable) or a single event loop with several
message sources. Reach for a thread only inside **one** service that needs genuine
**shared-memory, low-latency parallelism** and can accept intra-service fate-sharing —
e.g. a compositor splitting tile compositing across cores, where per-tile IPC would be
too chatty. "Input on one thread, display on another" is *not* that case; it wants two
processes. The isolation boundary stays at process granularity.
## The API surface (mirrors `std.Thread`)
Lives in `library/runtime/thread.zig`, re-exported as `runtime.Thread`.
```zig
pub const Thread = struct {
pub const Id = u32; // the kernel task id
pub const SpawnConfig = struct {
stack_size: usize = default_stack_size,
allocator: ?std.mem.Allocator = null, // for the closure + stack bookkeeping
};
pub const SpawnError = error{ OutOfMemory, ThreadQuotaExceeded, SystemResources };
pub fn spawn(config: SpawnConfig, comptime function: anytype, args: anytype) SpawnError!Thread;
pub fn join(self: Thread) void; // block until the thread ends, reclaim its stack
pub fn detach(self: Thread) void; // give up the right to join; kernel reclaims on exit
pub fn getCurrentId() Id;
pub fn yield() void; // -> existing `yield` syscall
pub const Mutex = struct { pub fn lock(*Mutex) void; pub fn tryLock(*Mutex) bool; pub fn unlock(*Mutex) void; };
pub const Condition = struct { pub fn wait(*Condition, *Mutex) void; pub fn timedWait(*Condition, *Mutex, u64) error{Timeout}!void; pub fn signal(*Condition) void; pub fn broadcast(*Condition) void; };
pub const Semaphore = struct { pub fn wait(*Semaphore) void; pub fn post(*Semaphore) void; };
pub const Futex = struct { pub fn wait(*const atomic.Value(u32), u32) void; pub fn timedWait(...) error{Timeout}!void; pub fn wake(*const atomic.Value(u32), u32) void; };
// RwLock / ResetEvent / WaitGroup follow the same pattern, added as needed.
};
```
Deviations from `std.Thread`, called out honestly:
- **The thread function's return value is discarded** (as `std.Thread.join` returns
`void`). Return data through shared state or a `Semaphore`/`Condition`, not the
return.
- `getCpuCount()` maps to the existing SMP core count ([smp.md](smp.md)); a service
rarely needs it.
## Kernel primitives (new private syscalls)
Four new entries extend [abi.zig](../system/abi.zig) `SystemCall` after
`shm_physical = 36`, each with a `library/runtime` wrapper:
| Syscall | Signature | Purpose |
|---|---|---|
| `thread_spawn` | `(entry, stack_top, arg) -> tid` | create a task sharing the **caller's** address space |
| `thread_exit` | `(stack_base, stack_len)` | end the calling thread; hand back its stack range for reclaim |
| `futex_wait` | `(addr, expected, timeout_ns) -> status` | block if `*addr == expected`, until woken or timeout |
| `futex_wake` | `(addr, count) -> woken` | wake up to `count` waiters on `addr` |
Plus one invariant change with no new syscall: **address-space reference counting**.
## Mechanics
### Address-space reference counting
Today an address space is 1:1 with a task: `spawnUserLocked` records `aspace` on the
Task, and teardown does `destroyAddressSpace(t.aspace)` when **any** user task exits
([scheduler.zig](../system/kernel/scheduler.zig)). With threads, several tasks share
one `aspace`, so the first to exit would rip the address space out from under its
siblings.
Fix: a small refcount keyed by the address-space root (`createAddressSpace` in
[process.zig](../system/kernel/process.zig) sets it to 1). `thread_spawn` increments
it; task teardown decrements and only calls `destroyAddressSpace` at **zero**. All of
this is already under the big kernel lock, so no new locking. This is the one piece
that must land and be proven before anything shares an address space.
### `thread_spawn` and the trampoline
The scheduler already accepts an arbitrary `aspace` and does **not** smuggle values
through registers — `startUserTask` reads the entry/stack from the Task and
`jumpToUser`s ([scheduler.zig](../system/kernel/scheduler.zig)). That makes the thread
path clean:
1. The runtime's `spawn` `mmap`s a stack (syscall `4`), heap-allocates a closure —
`{ fn_ptr, args_tuple, completion }`, the std "Instance" pattern — and writes the
closure pointer to the **top word of the new stack**.
2. It calls `thread_spawn(entry = &threadTrampoline, stack_top, arg = closure_ptr)`.
The kernel calls the same `spawnUserLocked` path with the **caller's aspace**
(refcount++), `entry`, and `user_sp = stack_top`.
3. `threadTrampoline` (a small runtime shim) reads the closure off its stack, calls
the user function, then calls `thread_exit`. No new register ABI — the closure
pointer rides the stack the runtime set up, mirroring how `startUserTask` avoids
register smuggling.
Unlike a process start, there is **no** System V argc/argv/auxv block
([sysv.md](sysv.md)) — a thread stack carries only the closure pointer.
### Lifetime: exit, join, detach, stack reclaim
- **`thread_exit`** marks the task dead and hands the kernel the thread's user-stack
range. The kernel reaps the task on the scheduler (already running on a *kernel*
stack, so it can safely unmap the user stack), decrements the aspace refcount, and
frees the task slot.
- **`join` — Stage 1** reuses the existing exit-notification machinery
([process-lifecycle.md](process-lifecycle.md)): `spawn` passes a per-thread
`exit_endpoint`, and `join` blocks in `ipc_reply_wait` until the child-exit
notification for that `tid` arrives, then `munmap`s the stack. No futex needed to
land spawn/join.
- **`join` — Stage 2 refinement** migrates to the std shape: a `completion` word in
the closure that `thread_exit`'s trampoline `futex_wake`s and `join` `futex_wait`s
on — dropping the per-thread endpoint. Kept as a refinement so Stage 1 ships first.
- **`detach`** relinquishes the join right; the kernel reclaims the stack and slot on
`thread_exit` (a detached thread's stack range is unmapped by the reaper, since no
joiner will).
### Futex, and the sync primitives on top
`futex_wait`/`futex_wake` are the one blocking primitive; `Mutex`, `Condition`, and
`Semaphore` are ordinary user-space state machines over an `atomic.Value(u32)` that
call the futex wrappers on the slow path — the same construction `std.Thread` uses,
so the algorithms port directly.
Keying: threads share an address space, so a **virtual address within that aspace**
identifies a futex uniquely; the kernel keys its wait queue by `(aspace_root, vaddr)`.
Keying by the **physical** address instead (translate `vaddr -> paddr` on entry) is a
deliberate forward door: it lets two *processes* share a futex through an
[shm](display-v2.md) region later, without changing the API. We start with the
private-per-aspace key and note the physical-key upgrade.
No spinning: a contended lock parks the task in the kernel and the core is free to run
other work or `hlt` ([halting.md](halting.md)). This is why futex is a locked
decision, not a "maybe later."
### TLS and `getCurrentId`
danos sets up no `fs.base` TLS today (fine under `single_threaded`). Two scoped needs:
- **`getCurrentId`** returns the kernel task id — either a trivial syscall or, better,
a value the runtime stashes in a per-thread control block.
- **`threadlocal` variables** need a real per-thread TLS block and `fs.base` set per
thread. `thread_spawn` sets `fs.base` to a runtime-allocated per-thread block; full
`threadlocal` support is Stage 3, only if a consumer needs it. Nothing in the core
spawn/join/mutex path requires `threadlocal`.
### Build: multi-threaded codegen, opt-in
`addUserBinary` gains a `threaded: bool = false` parameter; when set it builds that
binary `single_threaded = false` so atomics and (later) TLS are real. Threads and
atomics are unsound in a `single_threaded` image, so a binary must opt in **before**
it may call `runtime.Thread.spawn`. Everyone else stays single-threaded and lean.
## Interaction with the rest of the kernel
- **Scheduler / SMP** ([scheduling.md](scheduling.md), [smp.md](smp.md)): a thread is
just another `Task` with an `aspace` shared with its siblings; the existing
per-core ready queues, priorities, and affinity apply unchanged. Threads of one
process can run on different cores simultaneously — that is the point.
- **Halting** ([halting.md](halting.md)): futex-parked waiters keep the "idle core
halts" property intact under lock contention — no busy-wait.
- **Lifecycle** ([process-lifecycle.md](process-lifecycle.md)): killing a process
must kill *all* its threads and only then drop the last aspace ref. The kill path
already targets a process; it fans out to every task on that aspace.
- **Resilience** ([resilience.md](resilience.md)): a faulting thread kills its whole
process (shared fate). The supervisor restarts the **process**, which respawns its
threads from a known-good state — restart granularity stays the process.
## Build-out plan (staged, each gate serial-checkable)
The ordered, `/loop`-runnable milestones live in
**[threading-plan.md](threading-plan.md)** (shaped like
[display-v2-plan.md](display-v2-plan.md)): every milestone lands on its own and ends in
a verifiable gate (`python3 test/qemu_test.py <case>`, asserting serial markers;
`zig build test` for host unit tests). The stages below are the shape it expands.
- **Stage 0 — address-space refcount.** Refcount on the aspace root; teardown destroys
at zero. No API yet; nothing shares an aspace, so refcount is 1 everywhere.
*Gate:* the full QEMU suite stays green (no regression) — proves the reframing is
invisible until used.
- **Stage 1 — spawn / join / detach.** `thread_spawn` + `thread_exit`, the trampoline,
stacks via `mmap`, join over the exit-endpoint, the `threaded` build flag.
*Gate:* `-Dtest-case=thread-spawn` — a threaded test service spawns N threads that
each `@atomicRmw`-increment a shared counter, the parent joins all N, and asserts
the total is exactly N × iterations. Runs `smp` (multi-core) to prove real
parallelism.
- **Stage 2 — blocking synchronization.** `futex_wait`/`futex_wake` + `Futex`,
`Mutex`, `Condition`, `Semaphore`; optionally migrate join to a futex completion
word. *Gate:* `-Dtest-case=thread-mutex` — a bounded producer/consumer over a
`Mutex` + `Condition` moves K items with no lost wakeups and no busy-wait (assert
the consumer blocked, e.g. via a low idle tick count).
- **Stage 3 — polish.** Per-thread TLS / `fs.base` and `threadlocal` (only if a
consumer needs it), `RwLock`/`WaitGroup` as demanded, and this doc's cases wired
into [test/qemu_test.py](../test/qemu_test.py).
## Conventions
Follow [coding-standards.md](coding-standards.md): spell out non-acronym
abbreviations, kebab-case file names, no `Co-Authored-By` trailers. New syscalls
extend [abi.zig](../system/abi.zig) `SystemCall` + a `library/runtime` wrapper
([syscall.md](syscall.md)). `runtime.Thread` is a first-class runtime module, the same
way `runtime.process` ([process-lifecycle.md](process-lifecycle.md)) and `runtime.ipc`
are — user code never names a syscall.
## Non-goals
- **No preemptive user-space signals delivered to a specific thread.** Signals stay
process-scoped ([process-lifecycle.md](process-lifecycle.md)).
- **No thread priorities distinct from the process.** Threads inherit the process
priority; per-thread priority is a later question if it ever earns its keep.
- **No cross-process shared-memory futex yet** — the physical-address key leaves the
door open, but the first cut is private-per-aspace.
- **No `pthread`/POSIX surface.** The API is `std.Thread`-shaped Zig, nothing more.
## The self-hosting endgame
When danos becomes a real Zig target and we (eventually) add a danos backend to std
([zig-self-hosting.md](zig-self-hosting.md)), `std.Thread` can sit *on top of* these
same kernel primitives — the danos `std.Thread.Impl` would call the very
`thread_spawn`/`futex_*` wrappers `runtime.Thread` already uses. Because
`runtime.Thread` was built API-compatible from day one, that transition swaps the
implementation, not a single call site. Designing to the std shape now is what makes
the later self-hosting lift cheap.
## Further reading
- [scheduling.md](scheduling.md), [smp.md](smp.md) — the task model these threads join.
- [resilience.md](resilience.md), [vision.md](vision.md) — why isolation is the default
and threads are the exception.
- [syscall.md](syscall.md), [ipc.md](ipc.md) — the private ABI and the messaging model
threads sit beside.
- [halting.md](halting.md) — the idle/halt property futex-backed blocking preserves.
- [zig-self-hosting.md](zig-self-hosting.md) — the target this bends toward.