danos/docs/python-on-danos-milestones.md

# Python on danos: the milestone plan

The execution plan for [python-on-danos.md](python-on-danos.md). That note holds
the *why* and the design decisions; this one slices the work into milestones with
concrete deliverables, tests, and exit criteria. Milestones are numbered **P0–P5**
(track-local — the global M-series stays with the driver/lifecycle tracks).

Dependencies at a glance:

```
P0 toolchain + mini-libc ──┐
P1 streams + console + seam ─┴─→ P2 CPython minimal ─→ P3 terminal + REPL
                                        │                    │
                                        └─→ P4 danos module  │
                                             + Python service│
P5 process control + shell ←─────────────────────────────────┘
```

P0 and P1 are independent of each other and can proceed in parallel. P1 is shared
work — it is also Zig self-hosting Phase 1 and the first three slices of
[character-devices-and-tty.md](character-devices-and-tty.md).

## P0 — Toolchain + the C library compatibility layer

**Goal:** a C hello-world, cross-compiled on the host with `zig cc`, runs on danos.

Design and slicing live in
[c-library-compatibility.md](c-library-compatibility.md): the **libdanos-c**
sysroot (hand-written danos-native headers + `libc.a`) as a `library/c/` build
package — pure computation (string, libm, `strtod`, the printf/scanf engines)
lifted from a vendored, pinned musl subtree; the OS plumbing written in Zig over
the `runtime` surface (re-targeting `runtime.os` when the Zig track authors it);
`malloc` over danos `mmap`; a `crt0` bridging the danos entry shim to C `main`.
Driven by `zig cc -target x86_64-freestanding-none -isystem` (the triple becomes
`x86_64-danos` if the Zig fork lands first; nothing else changes).

Its five slices (sysroot-skeleton, fd-plumbing, malloc, stdio,
mathematics-and-time) carry their own tests — host-side oracle suites for the
computation layer, QEMU cases (`c-hello`, `c-file-io`, `c-stdio`, `c-time`) for
the plumbing.

**Exit:** `c-hello` and `c-file-io` green in the QEMU suite; host computation
tests green.

## P1 — Stream nodes, console, and the seam pieces

**Goal:** the shared Phase-1 surface exists: byte-stream stdio, cwd, environment,
entropy. Design and slicing live in
[character-devices-and-tty.md](character-devices-and-tty.md); this milestone is
its slices 1–3 plus three small seam pieces:

- **cwd/chdir** — per-process current directory used by path resolution (the
  kernel already anchors a VFS root per `fs_resolve`; the cwd is the same idea,
  process-scoped, with `getcwd`/`chdir` exposed through `runtime` and the libc).
- **Environment** — spawn carries an environment block; the SysV entry stack's
  `envp` slot ([sysv.md](os-development/sysv.md)) stops being empty; `getenv`
  reads it. An empty block stays valid.
- **Entropy** — a kernel `entropy` syscall (RDSEED/RDRAND with a jitter fallback,
  mirroring the TSC-reliability posture of not trusting one CPU feature blindly);
  the libc exposes `getentropy`.

- **Tests.** QEMU: the character-device tests from the tty note (offsetless
  read/write, blocking read, cooked/raw control round-trip), plus `cwd-basics`
  (chdir + relative open), `env-roundtrip` (spawn with env, child reads it),
  `entropy-sane` (nonzero, changing, correct length).

**Exit:** a C program reads a cooked line from fd 0 and echoes it to fd 1 —
injected key events in, bytes read back through the console's in-memory sink,
all under QEMU with no hardware involved — and `getcwd`/`getenv`/`getentropy`
return real answers.

## P2 — CPython, minimal configuration

**Goal:** `python -c 'print(2**100)'` runs on danos under QEMU.

- Pin **CPython 3.13.x**; vendor as `third-party/cpython/` or fetch via the build
  (decide with the build-packages conventions).
- Host build-Python of the same version (`--with-build-python`).
- `config.site` cache for the cross answers; `config.sub` patch so
  `x86_64-unknown-danos` parses; a small `configure`/`pyconfig` patch set kept as
  rebasable diffs, WASI-style.
- `--disable-shared`; static `Modules/Setup`: `posix errno _io _codecs _weakref
  time math _stat _collections itertools _functools _locale _sre` plus what the
  interpreter core insists on; threadless build (WASI precedent).
- `Lib/` on the FAT image under the hierarchy (e.g. `/system/python/lib`);
  `PYTHONHOME` set accordingly; `.pyc` written with **checked-hash
  invalidation** (FAT's 2-second mtime granularity makes mtime-based validation
  lie during fast edit-run cycles).
- `PYTHONHASHSEED` pinned only if P1's entropy slipped — otherwise real
  hash randomization from day one.
- **Tests.** QEMU: `python-expr` (the exit criterion), `python-file` (run a
  script from FAT, write a file, read it back), then a curated slice of CPython's
  own suite (`test_int`, `test_float`, `test_io`, `test_dict`) as a
  longer-running target — the suite is the porting harness.

**Exit:** the four QEMU cases green; the CPython test slice green or with a
short, documented skip list.

## P3 — Terminal + REPL: the first real application

**Goal:** an interactive `python` REPL in a graphical danos terminal — the
milestone demo for the OS.

- Depends on the display track's font rendering (its stated next step) — until
  that lands, the REPL is exercised end-to-end through the pseudo-device
  harness from P1+P2 (scripted input in, output read back), so P2's exit is
  never blocked on graphics; the graphical terminal is the *interactive* debut.
- The terminal application: draws with the UI toolkit / display client, consumes
  keyboard `InputEvent`s, and — per the tty note's load-bearing decision —
  **serves the VFS stream protocol itself** to its children, reusing the console's
  line-discipline library. Spawns `python` with its endpoints as fd 0/1/2.
- Raw mode + the control set give the REPL line editing; window-size control
  gives it wrapping.
- **Tests.** QEMU: scripted terminal session (inject key events, assert rendered
  or captured output). Real-hardware smoke on the Intel box joins the existing
  checklist.

**Exit:** typing `2+2` into the terminal on the QEMU GPU target prints `4`.

## P4 — The `danos` extension module + a Python service

**Goal:** Python can speak danos: IPC, capabilities, spawn.

- The `danos` module, **written in Zig against `Python.h`**, statically linked
  via `Modules/Setup`: endpoints (create/send/receive), capability passing,
  spawn + exit-notification, and the service bootstrap (announce, supervision
  handshake) — the same surface Zig services use, re-exposed.
- UI-toolkit bindings as a second module once the toolkit's API settles.
- Prototype **one real service in Python** — policy-shaped, not data-plane
  (candidates: hot-plug policy, a settings service) — speaking an existing wire
  protocol, supervised by the device manager like any service.
- **Tests.** QEMU: `python-ipc-echo` (Python service echoes over an endpoint, a
  Zig client asserts), plus the prototype service's own protocol test.

**Exit:** a Python process runs as a supervised danos service exchanging IPC
with Zig peers.

## P5 — Process control, then the shell

**Goal:** danos can spawn arbitrary programs with arguments and pipes; a small
Python shell uses it.

The kernel/VFS cluster a shell forces (any shell, any language):

- **exec-of-path** — spawn an arbitrary VFS path, not a named ramdisk binary;
- **argv/envp** — carried through spawn onto the child's entry stack (env from
  P1, argv new);
- **numeric exit status** — extend the exit record beyond the categorical
  `ExitReason` (the gotcha the Zig roadmap flagged: `WEXITSTATUS` must be real);
- **fd inheritance + pipes** — a kernel or service pipe (a character device by
  the tty note's definition) and spawn-time fd mapping.

Then, in order: `subprocess` enabled in CPython (maps onto spawn + the
exit-notification endpoint — no fork, Windows-style); a **small Python shell** (a
few hundred lines over `subprocess` + the console: prompt, argv parsing, pipes,
cwd) as the forcing function that reveals what job control actually needs.

**Explicitly deferred past P5:** the pthread subset over `thread_spawn`/futex,
signals-in-libc via M17, termios job control (Ctrl-C to foreground child), and
**xonsh** — which wants all three and is the arc's endpoint, not a milestone.

**Tests.** QEMU: `spawn-argv-exit` (child echoes argv, exits 42, parent sees
42), `pipe-through` (parent → child → parent), `python-subprocess`, and a
scripted shell session.

**Exit:** the Python shell runs `program | program` typed at the terminal and
reports the exit status.

## Post-P5 outlook

Two tracks continue past this plan, each with its own design doc rather than a
P-number here:

- **Dynamic libraries** ([dynamic-libraries.md](dynamic-libraries.md), D1–D4) —
  an application-layer facility (the OS stays static and lean): `dlopen` in the
  libc, then libffi + `ctypes` + loadable extension modules, then shared
  read-only mappings so N Python services hold one physical `libpython`.
- **The full C compatibility layer**
  ([c-library-compatibility.md](c-library-compatibility.md), stages 2–3) — the
  standing rule that every system capability ships with its C spelling, draining
  the absence table toward "portable C builds on danos"; `fork` is the one
  permanent exception.

## Related

- [python-on-danos.md](python-on-danos.md) — the design note this executes.
- [c-library-compatibility.md](c-library-compatibility.md) — P0's design.
- [character-devices-and-tty.md](character-devices-and-tty.md) — P1's design.
- [zig-self-hosting.md](zig-self-hosting.md) — shares P1; its fork makes P0's
  triple prettier but gates nothing here.
- [os-development/process-management.md](os-development/process-management.md) —
  the spawn/exit surface P5 extends.