docs: the Python track — CPython via zig cc, the C layer, streams, dynamic libraries

Five design-and-milestone notes for making danos programmable before Zig
self-hosts:

- python-on-danos: why CPython, the WASI precedent, static-only interop
- python-on-danos-milestones: P0–P5 (libc → seam → CPython → terminal+REPL →
  danos module → process control + shell) with tests and exit criteria
- c-library-compatibility: the libdanos-c sysroot — musl computation lifted,
  Zig plumbing over runtime; staged road to full coverage (fork never comes)
- character-devices-and-tty: stream nodes over the existing VFS; first device
  is an in-memory loopback (COM1 off the critical path); no pty object — the
  terminal serves its children directly
- dynamic-libraries: post-P5 D1–D4, application-layer only

The size doctrine runs through all of them: the OS stays lean (kernel in
kilobytes, services small and static, never linking the libc); applications
have their own budget.

docs/dynamic-libraries.md

@@ -0,0 +1,124 @@
+# Dynamic libraries on danos
+
+A design note and milestone plan for shared objects: building them, loading them
+with `dlopen`, and — the part that needs kernel work — actually *sharing* them
+between processes. Directional, post-P5 of
+[python-on-danos-milestones.md](python-on-danos-milestones.md); nothing on the
+CPython bring-up path depends on it.
+
+## Reconciling the earlier "rejected"
+
+Dynamic libraries were evaluated once before and rejected — but as an answer to a
+*different question*: whether they could claw back ReleaseSafe's measured ~2×
+code size. They cannot (the safety checks inline at every call site; no library
+scheme dedups them), and that verdict stands for that question. The reasons to
+build them now are the ones that investigation never weighed:
+
+- **`ctypes` and runtime FFI** — Python calling into a danos library without
+  rebuilding the interpreter. This is the piece that makes Python prototyping
+  self-serve: drop a `.so` on the image, `ctypes.CDLL` it, iterate.
+- **Loadable CPython extension modules** — today every C extension means
+  relinking the interpreter (`Modules/Setup`); with `dlopen`, an extension is a
+  file.
+- **One interpreter image, many Python services** — a statically-linked CPython
+  is tens of megabytes *per process*. A shared `libpython` mapped read-only once
+  (milestone D3 below) makes Python services cheap enough to be the default way
+  to prototype one.
+- **Plugin-shaped applications** — the UI toolkit and the terminal will want
+  them eventually.
+
+The scoping that dissolves the apparent contradiction is the **size doctrine**:
+leanness is an *operating-system* property — the kernel and system services stay
+small and statically linked, and none of them ever link the loader — while
+*applications* have their own budget and may be big. Dynamic libraries are an
+**application-layer facility**, full stop.
+
+What also does **not** change: the public ABI stays the vDSO + the IPC
+protocols. Shared objects are artifacts *within* one system image, versioned by
+the build — not a new stable ABI surface for the OS.
+
+## Design
+
+- **Format and codegen are free.** ELF shared objects with position-independent
+  code; `zig cc -fPIC -shared` against the [libdanos-c](c-library-compatibility.md)
+  sysroot already emits them. The work is entirely on the loading side.
+- **The loader lives in userspace, inside the libc.** `dlopen` reads the `.so`
+  through the VFS, maps its segments, applies relocations, resolves symbols
+  against the process and the `DT_NEEDED` dependency graph, runs constructors,
+  returns a handle. No kernel loader changes in v1 — segments land in anonymous
+  `mmap` as private copies.
+- **Bind-now, always.** All relocations resolved at `dlopen` time
+  (`RTLD_NOW` semantics only). Lazy PLT binding buys startup latency danos does
+  not care about, at the price of a writable GOT dance and a much subtler
+  loader. Not worth it; keep it out permanently.
+- **W^X from day one.** Map, relocate, then flip text pages read-execute —
+  which requires memory-protection change (`mprotect`-shaped) in the danos
+  `mmap` surface if it is not already there. No page is ever writable and
+  executable at once.
+- **TLS in shared objects is deferred.** Thread-local storage models
+  (initial-exec vs. general-dynamic) are the deep end of every dynamic linker.
+  v1 refuses a `.so` with a TLS segment; revisit alongside the post-P5 pthread
+  subset, which is when it could matter.
+- **Executables stay static until D4.** v1 is "a static binary that can
+  `dlopen`" — no `PT_INTERP`, no program interpreter, no dynamically-linked
+  `main` binaries. That keeps process startup untouched.
+
+## Milestones
+
+1. **D1 — dlopen in-process.** The `.so` build target; the loader in libdanos-c:
+   map, relocate (`RELATIVE`/`GLOB_DAT`/`JUMP_SLOT`), resolve, constructors;
+   `dlopen`/`dlsym`/`dlerror`/`dlclose`; private anonymous mappings; no TLS.
+   *Test:* QEMU `dlopen-hello` — load a `.so`, call a symbol, unload, reload.
+2. **D2 — the FFI payoff.** `DT_NEEDED` dependency graphs; a **libffi port**
+   (x86-64 SysV assembly is upstream; the port is its closure-allocation paths,
+   which must respect W^X); CPython's `ctypes` enabled; extension modules
+   loadable from file. *Test:* QEMU — a Python script `ctypes.CDLL`s a danos
+   `.so` and round-trips a call; a `.so` extension module imports.
+3. **D3 — actual sharing (the kernel milestone).** Shared read-only file-backed
+   mappings — a page-cache-shaped facility so N processes mapping `libpython`
+   hold one physical copy. This is the memory-win milestone and the only one
+   touching the kernel; design it with the existing shm machinery in view
+   (the shared-fate walks already locked the relevant paths). *Test:* N Python
+   services up; measure physical pages against N× the static baseline.
+4. **D4 — dynamically-linked executables** (optional, evaluate after D3):
+   `PT_INTERP`, a danos program interpreter, and the spawn path teaching the
+   loader about it. Only worth it if the image-size or update story demands it.
+
+## Risks and gotchas
+
+- **Scope creep is the failure mode.** Every dynamic linker grows toward glibc.
+  The fences: bind-now only, no lazy binding ever, no TLS until pthreads demand
+  it, no dlopen-from-memory, no versioned symbols. Each fence removed is a
+  design discussion, not a patch.
+- **Code loading is a security event.** `dlopen` turns file bytes into executable
+  code, so W^X discipline is table stakes and *what may be dlopened* is a
+  capability question — the natural danos answer is that loadability follows VFS
+  readability of the `.so`, and services' images are supervised like any other
+  artifact. Revisit explicitly at D3 when mappings become shared.
+- **`dlclose` is where loaders go to die.** Constructors/destructors,
+  dangling function pointers, re-open identity. Keep v1 semantics honest and
+  simple: `dlclose` runs destructors and unmaps; holding pointers past it is
+  undefined; no reference-counted deferral cleverness.
+- **The ReleaseSafe fact still applies to `.so`s** — a ReleaseSafe shared object
+  carries its inlined checks like any static code; D3's sharing saves *copies*,
+  not check overhead. Size expectations should be set accordingly.
+
+## Decisions needing sign-off
+
+- Dynamic libraries join the roadmap at all (this note exists because the
+  earlier size-motivated rejection was re-opened for ABI/sharing reasons).
+- **Bind-now only; no lazy binding, permanently.**
+- **Loader in userspace libc; kernel involvement only at D3** (shared read-only
+  mappings).
+- **Static executables until D4**, and D4 only on demonstrated need.
+
+## Related
+
+- [c-library-compatibility.md](c-library-compatibility.md) — the sysroot the
+  loader ships in; its absence table gains `dlfcn.h` at D1.
+- [python-on-danos.md](python-on-danos.md) — the `ctypes` story this unlocks.
+- [python-on-danos-milestones.md](python-on-danos-milestones.md) — sequencing;
+  this work is post-P5.
+- [os-development/memory-map.md](os-development/memory-map.md) /
+  [os-development/paging.md](os-development/paging.md) — where W^X and shared
+  mappings land.