danos/docs/python-on-danos-milestones.md

Python on danos: the milestone plan

The execution plan for python-on-danos.md. That note holds the why and the design decisions; this one slices the work into milestones with concrete deliverables, tests, and exit criteria. Milestones are numbered P0–P5 (track-local — the global M-series stays with the driver/lifecycle tracks).

Dependencies at a glance:

P0 toolchain + mini-libc ──┐
P1 streams + console + seam ─┴─→ P2 CPython minimal ─→ P3 terminal + REPL
                                        │                    │
                                        └─→ P4 danos module  │
                                             + Python service│
P5 process control + shell ←─────────────────────────────────┘

P0 and P1 are independent of each other and can proceed in parallel. P1 is shared work — it is also Zig self-hosting Phase 1 and the first three slices of character-devices-and-tty.md.

P0 — Toolchain + the C library compatibility layer

Goal: a C hello-world, cross-compiled on the host with zig cc, runs on danos.

Design and slicing live in c-library-compatibility.md: the libdanos-c sysroot (hand-written danos-native headers + libc.a) as a library/c/ build package — pure computation (string, libm, strtod, the printf/scanf engines) lifted from a vendored, pinned musl subtree; the OS plumbing written in Zig over the runtime surface (re-targeting runtime.os when the Zig track authors it); malloc over danos mmap; a crt0 bridging the danos entry shim to C main. Driven by zig cc -target x86_64-freestanding-none -isystem (the triple becomes x86_64-danos if the Zig fork lands first; nothing else changes).

Its five slices (sysroot-skeleton, fd-plumbing, malloc, stdio, mathematics-and-time) carry their own tests — host-side oracle suites for the computation layer, QEMU cases (c-hello, c-file-io, c-stdio, c-time) for the plumbing.

Exit: c-hello and c-file-io green in the QEMU suite; host computation tests green.

P1 — Stream nodes, console, and the seam pieces

Goal: the shared Phase-1 surface exists: byte-stream stdio, cwd, environment, entropy. Design and slicing live in character-devices-and-tty.md; this milestone is its slices 1–3 plus three small seam pieces:

• cwd/chdir — per-process current directory used by path resolution (the kernel already anchors a VFS root per fs_resolve; the cwd is the same idea, process-scoped, with getcwd/chdir exposed through runtime and the libc).

• Environment — spawn carries an environment block; the SysV entry stack's envp slot (sysv.md) stops being empty; getenv reads it. An empty block stays valid.

• Entropy — a kernel entropy syscall (RDSEED/RDRAND with a jitter fallback, mirroring the TSC-reliability posture of not trusting one CPU feature blindly); the libc exposes getentropy.

• Tests. QEMU: the character-device tests from the tty note (offsetless read/write, blocking read, cooked/raw control round-trip), plus cwd-basics (chdir + relative open), env-roundtrip (spawn with env, child reads it), entropy-sane (nonzero, changing, correct length).

Exit: a C program reads a cooked line from fd 0 and echoes it to fd 1 — injected key events in, bytes read back through the console's in-memory sink, all under QEMU with no hardware involved — and getcwd/getenv/getentropy return real answers.

P2 — CPython, minimal configuration

Goal: python -c 'print(2**100)' runs on danos under QEMU.

• Pin CPython 3.13.x; vendor as third-party/cpython/ or fetch via the build (decide with the build-packages conventions).
• Host build-Python of the same version (--with-build-python).
• config.site cache for the cross answers; config.sub patch so x86_64-unknown-danos parses; a small configure/pyconfig patch set kept as rebasable diffs, WASI-style.
• --disable-shared; static Modules/Setup: posix errno _io _codecs _weakref time math _stat _collections itertools _functools _locale _sre plus what the interpreter core insists on; threadless build (WASI precedent).
• Lib/ on the FAT image under the hierarchy (e.g. /system/python/lib); PYTHONHOME set accordingly; .pyc written with checked-hash invalidation (FAT's 2-second mtime granularity makes mtime-based validation lie during fast edit-run cycles).
• PYTHONHASHSEED pinned only if P1's entropy slipped — otherwise real hash randomization from day one.
• Tests. QEMU: python-expr (the exit criterion), python-file (run a script from FAT, write a file, read it back), then a curated slice of CPython's own suite (test_int, test_float, test_io, test_dict) as a longer-running target — the suite is the porting harness.

Exit: the four QEMU cases green; the CPython test slice green or with a short, documented skip list.

P3 — Terminal + REPL: the first real application

Goal: an interactive python REPL in a graphical danos terminal — the milestone demo for the OS.

• Depends on the display track's font rendering (its stated next step) — until that lands, the REPL is exercised end-to-end through the pseudo-device harness from P1+P2 (scripted input in, output read back), so P2's exit is never blocked on graphics; the graphical terminal is the interactive debut.
• The terminal application: draws with the UI toolkit / display client, consumes keyboard InputEvents, and — per the tty note's load-bearing decision — serves the VFS stream protocol itself to its children, reusing the console's line-discipline library. Spawns python with its endpoints as fd 0/1/2.
• Raw mode + the control set give the REPL line editing; window-size control gives it wrapping.
• Tests. QEMU: scripted terminal session (inject key events, assert rendered or captured output). Real-hardware smoke on the Intel box joins the existing checklist.

Exit: typing 2+2 into the terminal on the QEMU GPU target prints 4.

P4 — The danos extension module + a Python service

Goal: Python can speak danos: IPC, capabilities, spawn.

• The danos module, written in Zig against Python.h, statically linked via Modules/Setup: endpoints (create/send/receive), capability passing, spawn + exit-notification, and the service bootstrap (announce, supervision handshake) — the same surface Zig services use, re-exposed.
• UI-toolkit bindings as a second module once the toolkit's API settles.
• Prototype one real service in Python — policy-shaped, not data-plane (candidates: hot-plug policy, a settings service) — speaking an existing wire protocol, supervised by the device manager like any service.
• Tests. QEMU: python-ipc-echo (Python service echoes over an endpoint, a Zig client asserts), plus the prototype service's own protocol test.

Exit: a Python process runs as a supervised danos service exchanging IPC with Zig peers.

P5 — Process control, then the shell

Goal: danos can spawn arbitrary programs with arguments and pipes; a small Python shell uses it.

The kernel/VFS cluster a shell forces (any shell, any language):

• exec-of-path — spawn an arbitrary VFS path, not a named ramdisk binary;
• argv/envp — carried through spawn onto the child's entry stack (env from P1, argv new);
• numeric exit status — extend the exit record beyond the categorical ExitReason (the gotcha the Zig roadmap flagged: WEXITSTATUS must be real);
• fd inheritance + pipes — a kernel or service pipe (a character device by the tty note's definition) and spawn-time fd mapping.

Then, in order: subprocess enabled in CPython (maps onto spawn + the exit-notification endpoint — no fork, Windows-style); a small Python shell (a few hundred lines over subprocess + the console: prompt, argv parsing, pipes, cwd) as the forcing function that reveals what job control actually needs.

Explicitly deferred past P5: the pthread subset over thread_spawn/futex, signals-in-libc via M17, termios job control (Ctrl-C to foreground child), and xonsh — which wants all three and is the arc's endpoint, not a milestone.

Tests. QEMU: spawn-argv-exit (child echoes argv, exits 42, parent sees 42), pipe-through (parent → child → parent), python-subprocess, and a scripted shell session.

Exit: the Python shell runs program | program typed at the terminal and reports the exit status.

Post-P5 outlook

Two tracks continue past this plan, each with its own design doc rather than a P-number here:

• Dynamic libraries (dynamic-libraries.md, D1–D4) — an application-layer facility (the OS stays static and lean): dlopen in the libc, then libffi + ctypes + loadable extension modules, then shared read-only mappings so N Python services hold one physical libpython.
• The full C compatibility layer (c-library-compatibility.md, stages 2–3) — the standing rule that every system capability ships with its C spelling, draining the absence table toward "portable C builds on danos"; fork is the one permanent exception.

Related

• python-on-danos.md — the design note this executes.
• c-library-compatibility.md — P0's design.
• character-devices-and-tty.md — P1's design.
• zig-self-hosting.md — shares P1; its fork makes P0's triple prettier but gates nothing here.
• os-development/process-management.md — the spawn/exit surface P5 extends.