documenting things to research

docs/vision.md

@@ -1,84 +1,108 @@
-# Vision: a real-time microkernel
+# Vision: a microkernel, built to learn
 
-danos is aiming to be a **real-time operating system built on a microkernel** —
-where drivers and services run isolated in user space for maximum stability, and
-scheduling gives real guarantees about timing. This page is the north star: the
-*why* that shapes every design decision below it. Read it before adding anything
-structural.
+danos exists first and foremost as a **learning-by-doing project**: the point is to
+build a real operating system, bump into the hard constraints for real, and research
+them from a position of having actually hit them. The docs in this folder are part of
+that — they're where a constraint gets understood once it's been met.
 
-## Microkernel
+That framing sets the priorities. danos is not chasing a spec or a product; it's
+chasing understanding, with a concrete, motivating **win condition** to aim at.
 
-The kernel stays **minimal** — only what genuinely must run in privileged mode:
+## The win condition
+
+danos is a "win" when it:
+
+- **boots and runs on real hardware** — the author's **PC** (x86-64) and **both
+  Raspberry Pis**: the **Zero 2 W** and the **Pi 5** (both `aarch64`, one backend —
+  see [arm.md](arm.md)),
+- **has a graphical user interface**, ideally — building on the framebuffer it
+  already draws to.
+
+Everything below serves that, or serves the curiosity that the project runs on.
+
+## Why a microkernel: resilience
+
+The kernel stays **minimal** — only what genuinely must run privileged:
 
 - scheduling,
 - inter-process communication (IPC),
 - memory management (address spaces, page tables),
 - low-level interrupt dispatch.
 
-Everything else — device drivers, filesystems, the network stack — runs as an
-**isolated user-space server**, each in its own address space with only the
+Everything else — device drivers, filesystems, the GUI, the network stack — runs as
+an **isolated user-space server**, each in its own address space with only the
 privileges it needs.
 
-The payoff is **stability through isolation**. A driver bug can't corrupt the
-kernel or another driver; a crashing service is contained and can be restarted,
-while the rest of the system keeps running. That's the opposite of a monolithic
-kernel, where a single driver fault can take everything down.
+The reason for this shape is **resilience**: the ability to **re-initialise parts of
+the OS while it runs**. A driver bug can't corrupt the kernel or another driver; a
+crashed or wedged component is contained, killed, and **restarted** — "if I break
+something, I can just fix it," without rebooting. Keeping the kernel tiny is part of
+that strategy: the one thing that *can't* be restarted is the trusted base, so the
+less code in it, the less that can take the whole system down. This is the project's
+real motivation, and it has its own design note: [resilience.md](resilience.md).
 
-The cost is that **IPC becomes the backbone**: whatever used to be a function call
-across a monolithic kernel is now a message between address spaces. In a
-microkernel, IPC performance essentially *is* system performance (the lesson of
-L4). So IPC must be fast, and it's a first-class concern, not an afterthought.
-Hardware interrupts, too, become IPC: the kernel turns an IRQ into a message to the
-driver task that owns that device.
+The cost is that **IPC becomes the backbone**: what used to be a function call inside
+a monolithic kernel is now a message between address spaces. In a microkernel, IPC
+performance essentially *is* system performance (the lesson of L4), so it's a
+first-class concern. Hardware interrupts become IPC too: the kernel turns an IRQ into
+a message to the driver that owns the device.
 
-## Real-time
+## On real-time: an option, not a commitment
 
-danos schedules **preemptively, with guarantees about quanta** — the system must
-be able to promise that a task runs when it's supposed to, within bounded time.
-That imposes concrete requirements:
+danos was originally framed as a hard **real-time** OS. That's now held as **one
+interesting constraint to explore, not a requirement** — because real-time is a
+*pervasive* invariant (every operation must be provably time-bounded, everywhere)
+that would slow every milestone, whereas resilience is a set of *structural* features
+that's lighter to build and is what the project actually wants. The trade-off is
+written up in [smp.md](smp.md#does-the-right-choice-depend-on-real-time-vs-resilience).
 
-- **Fixed-priority preemptive scheduling.** The highest-priority ready task always
-  runs; a higher-priority task that becomes ready preempts a lower one immediately.
-  Not round-robin (which is fair but not predictable).
-- **A calibrated, deterministic clock.** Guarantees measured in "quanta" are
-  meaningless on an arbitrary tick rate — real time requires a timer calibrated to
-  a known frequency.
-- **Bounded interrupt latency.** Interrupt-disabled sections must be short and
-  bounded, so a ready high-priority task is never delayed by an unbounded kernel
-  operation.
-- **Deterministic kernel operations.** Scheduling decisions should be O(1) (e.g. a
-  priority bitmap), not "walk a list of unknown length."
-- **Priority inheritance** (once there are locks/IPC), so a high-priority task
-  blocked on a resource held by a low-priority one can't be delayed indefinitely by
-  a middle-priority task — bounding priority inversion.
+What danos keeps from the real-time direction, because it's cheap and useful anyway:
 
-A consequence worth stating early: the current [kernel heap](heap.md) is a
-first-fit free list, which has **unbounded allocation time** and can fragment — it
-is *not* real-time safe. It's fine for one-time kernel setup, but real-time paths
-must pre-allocate or use a bounded (fixed-size pool) allocator. Don't allocate on a
-hot real-time path.
+- **Fixed-priority preemptive scheduling** — the highest-priority ready task runs, and
+  preemption lets a runaway component be interrupted and killed (which *serves
+  resilience*). Already built ([scheduling.md](scheduling.md)).
+- **A calibrated, deterministic clock** — already built ([device-interrupts.md](device-interrupts.md)).
 
-## What this means for the roadmap
+What danos does *not* owe anyone unless it deliberately chooses real-time later:
+timing *guarantees*, priority inheritance, bounded allocators, tickless timers, MCS
+scheduling contexts. Concretely, the current [heap](heap.md) is a first-fit free list
+with unbounded allocation time — fine here, and only a problem *if* a hard-real-time
+path is ever added. Note that **QNX is both** a real-time and a restartable
+microkernel, so choosing resilience now doesn't close the real-time door — it just
+doesn't pay the tax yet.
 
-The vision reorders the obvious hobby-kernel path. Notably, **drivers are not
-built into the kernel** — so an in-kernel keyboard driver would be throwaway work.
-Input devices arrive later, as the *first user-space drivers*, once the machinery
-to isolate them exists. The trajectory:
+## The roadmap — tracks, not a strict line
 
-1. **Calibrated timer / clock** — a known-frequency, deterministic tick. The
-   foundation real-time quanta rest on. *(next)*
-2. **Real-time scheduler** — fixed-priority preemptive, kernel threads first:
-   context switch, task struct, priority run-queue, timer-driven preemption.
-3. **User mode + address-space isolation** — higher-half kernel, ring 3, per-process
-   page tables. The substrate for isolated servers.
-4. **IPC** — fast message passing between address spaces. The microkernel's heart.
-5. **User-space drivers** — interrupts delivered as IPC, plus MMIO/port-access
-   grants. The keyboard becomes the first one, validating the whole model.
+Because the driver is curiosity plus the win condition, the roadmap is a set of
+**tracks** with dependencies, not a rigid sequence. Pick by interest; mind the
+prerequisites.
 
-## Where we are
+**Done:** UEFI boot, framebuffer + [serial](testing.md), [physical frames](frame-allocator.md)
+(with boot-services memory reclaimed), [paging](paging.md) with W^X, [exceptions and
+interrupts](interrupts.md), a [calibrated timer + ns clock](device-interrupts.md), a
+[heap](heap.md), a [fixed-priority preemptive scheduler](scheduling.md) with blocking,
+and in-kernel [IPC channels](ipc.md) — plus a [test harness](testing.md).
 
-The foundation is in place: UEFI boot, framebuffer + [serial](testing.md),
-[physical frames](frame-allocator.md), [paging](paging.md) with W^X, [exceptions
-and interrupts](interrupts.md), a [timer](device-interrupts.md), and a
-[heap](heap.md) — plus a [test harness](testing.md). The kernel boots and has its
-core services; the next milestones make it *schedule*, then *isolate*.
+- **Isolation track** — **user mode + address-space isolation** (higher-half kernel,
+  ring 3, per-process page tables). The substrate everything else needs. *Next, and a
+  prerequisite for the resilience and driver tracks.*
+- **Resilience track** — fault → kill → notify, a supervisor/reincarnation server,
+  resource cleanup on death, then a restartable driver as proof. Needs isolation.
+  See [resilience.md](resilience.md).
+- **ARM track** — the `aarch64` port so danos runs on the Zero 2 W and Pi 5. Largely
+  independent of the others (it's the [arch layer](arch.md)); directly serves the win
+  condition. Likely via aarch64-UEFI first (QEMU `virt` + AAVMF), then real boards.
+  See [arm.md](arm.md), and [discovery.md](discovery.md) for the device tree it needs.
+- **GUI track** — a framebuffer-based windowing/compositor, and the input + display
+  drivers under it. Builds on the neutral framebuffer (so it's arch-independent), and
+  on the driver model from the isolation/resilience tracks. The visible payoff.
+
+The natural spine is **isolation → (resilience + drivers) → GUI**, with the **ARM
+track** pursued alongside whenever the itch to see it boot on a Pi wins out.
+
+## How to use this page
+
+Read it before adding anything structural. When a design decision comes up, the
+question is: does it serve the **win condition** (runs on the three machines, with a
+GUI), or the **learning** (a constraint worth meeting)? If it serves neither — e.g.
+paying the full real-time tax with no payoff in sight — it can wait.