danos/docs/c-library-compatibility.md

# The C library compatibility layer

A design note and milestone plan for **libdanos-c** — the mini C library that lets
`zig cc` cross-compile C programs for danos. It is milestone **P0** of
[python-on-danos-milestones.md](python-on-danos-milestones.md), expanded here the
way [character-devices-and-tty.md](character-devices-and-tty.md) expands P1.
CPython is the driving consumer, but the layer is general: any portable C program
within its surface should build.

## What it is — and the three things it is not

The deliverable is a **sysroot**: a set of C headers plus a static `libdanos-c.a`,
handed to `zig cc -target x86_64-freestanding-none` via `-isystem` and linked into
every C binary. Three explicit non-goals keep it small:

- **Not a musl port.** Whole-musl assumes Linux syscall semantics at its bottom
  (the door the Zig roadmap deferred, twice now). We *lift* musl's pure-computation
  source files and *write* a danos-native bottom — see the layer split below.
- **Not full POSIX — *yet*.** Stage 1's surface is "what CPython's minimal
  configuration and ordinary portable C need" — roughly 100–150 functions — and
  at that stage absence is a *feature*: configure scripts probe and adapt, and a
  linker error is honest. But the end state is a **full C compatibility layer**
  (see "The road to full coverage" below); the absence table is a schedule of
  arrivals, not a wall.
- **Not a second runtime.** The library is a thin C-ABI re-spelling of the same
  danos-native surface `runtime` already provides. It contains no policy of its
  own; when the Zig track's `runtime.os` seam is authored, the libc bottom
  re-targets it near-mechanically — the fourth appearance of the roadmap's "same
  surface" symmetry.

One scoping rule sits above all three — the **size doctrine**: this layer serves
**applications only**. The kernel and the system services never link libdanos-c;
they stay danos-native Zig over `runtime`, small and static, because leanness is
an operating-system property. Applications have their own budget and may be as
big as they need to be. The libc is how big software *lands on* danos, never how
danos itself is built.

## The layer split: lift the mathematics, write the plumbing

The realization that makes 100–150 functions tractable: a libc is two very
different kinds of code, and the hard kind is portable.

| Layer | Contents | Source |
|-------|----------|--------|
| **Pure computation** | `string.h`/`memcpy` family, all of libm, `strtod`/`dtoa`, `strtol`, `qsort`, `ctype` tables, `gmtime` calendar math, the `printf`/`scanf` engines, `setjmp` (a dozen instructions of x86-64 asm) | **Lift from musl**, vendored under `library/c/third-party/musl/` (MIT; files compile standalone) |
| **OS plumbing** | fds (`open`/`read`/`write`/`close`/`lseek`/`stat`/`getcwd`/`chdir`/`isatty`), `mmap`/`munmap`, clocks, `exit`, `getenv`, `getentropy` | **Write in Zig**, exporting C ABI over the `runtime` syscall + VFS client surface |
| **The middle** | `malloc` over danos `mmap` (simple free-list; CPython's arenas sit above), `FILE*` buffering, `errno` | **Write in Zig** (small, danos-shaped) |
| **Entry** | `crt0`: the existing danos entry shim ([sysv.md](os-development/sysv.md)) bridged to C `main(argc, argv, envp)`, `environ` initialised, `exit` flushing stdio | **Write** |

Two liftings deserve their own line because getting them wrong is silent
corruption rather than a linker error:

- **`strtod`/float formatting.** Python's float `repr` guarantees shortest
  round-trip; that property lives entirely in these routines. musl's are correct;
  an improvised one would be subtly wrong for years. Lift, never write.
- **The stdio engines.** musl's `vfprintf`/`vfscanf` are self-contained around
  its `FILE` abstraction (function-pointer read/write slots), so the whole
  formatted-I/O engine lifts too — we implement only the fd-backed slots
  (`__stdio_write`-shaped) and the buffering glue.

## Header policy

Hand-write the headers as danos's own minimal set rather than importing musl's
(musl's are entangled with Linux ABI details), borrowing declarations freely.
Freestanding compiler headers (`stdint.h`, `stddef.h`, `stdarg.h`, `stdbool.h`,
`float.h`, `limits.h`) come from clang via `zig cc` — do not duplicate them.
`errno.h` values are the danos errno enum re-spelled with POSIX names; there is no
Linux numbering to be compatible with, so the enum is the truth.

Deliberate absences, and their planned arrivals — this table is the
compatibility matrix, and "the road to full coverage" below is the schedule
that empties it:

| Absent | Arrives with |
|--------|--------------|
| `pthread.h` | the post-P5 pthread subset over `thread_spawn`/futex — but see the risk below |
| real `signal.h` (beyond no-op `signal()`/`raise` stubs) | M17 signals-over-IPC in the libc |
| `dlfcn.h` | [dynamic-libraries.md](dynamic-libraries.md) D1 |
| `fork`/`exec*`/`wait*` | P5 exposes danos spawn as `posix_spawn`; `fork` itself never (see below) |
| `socket.h` | a future networking track |
| locale beyond `"C"` | stage 3 evaluation (CPython is UTF-8-mode happy without it) |
| pipes (`pipe()`) | P5 process-control cluster |

## The road to full coverage

The layer grows in three stages; only stage 1 is a current milestone (P0), but
the stages exist so stage-1 decisions never have to be unmade:

- **Stage 1 — CPython-minimal** (P0, the slicing below): ~100–150 functions,
  static-only, absences honest.
- **Stage 2 — the danos-complete layer**: the full hosted C11 standard library,
  plus every POSIX facility danos semantics support, landing as its enabling
  milestone lands — pipes and `posix_spawn` at P5, real signals at M17, the
  pthread subset after P5, `dlfcn.h` at
  [dynamic-libraries](dynamic-libraries.md) D1, sockets with networking. Stage 2
  is not one milestone but the standing rule that **every system capability
  gets its C spelling when it ships**, so the matrix above drains as the OS
  grows.
- **Stage 3 — ecosystem grade**: the point where "portable C program" generally
  means "builds on danos" (autotools-style probing included). Reaching it is
  mostly stage 2 compounding, plus the long tail (locale, wide-char,
  `fnmatch`/`glob`/`regex` — the last three lift from musl like the rest). At
  this stage, re-evaluate hand-grown-vs-musl-port once with real data; the
  standing recommendation remains danos-native — musl's bottom assumes Linux
  syscall semantics, and by stage 3 the danos bottom exists and is tested —
  with musl continuing as the quarry for computation code.

Two boundaries are permanent and worth stating at every stage: **`fork` never
comes** — danos is a spawn-shaped OS, and `fork`'s address-space-duplication
semantics are hostile to everything from capabilities to threads; software that
hard-requires `fork` (not `posix_spawn`) stays off the platform. And the
**public ABI stays the vDSO + IPC protocols** — a full libc is a compatibility
*layer*, not a second stable system ABI.

## Milestone slicing

1. **sysroot-skeleton** — layout under `library/c/` (a build package:
   `include/`, Zig sources, vendored musl subtree); `crt0`; string/mem +
   `ctype` lifted; a `build.zig` step making C binaries first-class targets.
   *Test:* a C program using only computation links and runs in QEMU
   (`c-hello` printing via a raw `write` extern to `debug_write`).
2. **fd-plumbing** — `errno`; open/read/write/close/lseek/stat/unlink/mkdir/
   rename over the `runtime` VFS client; `getcwd`/`chdir`/`getenv`/
   `getentropy` arriving as P1 lands them (stubbed truthfully until then:
   `getenv` empty, `getentropy` `ENOSYS`). *Test:* QEMU `c-file-io` — create,
   write, reopen, read back, stat size + mtime through FAT.
3. **malloc** — free-list allocator over danos `mmap`; `calloc`/`realloc`/
   `free`; alignment guarantees documented. *Test:* host + QEMU allocator
   torture (interleaved sizes, realloc growth, alignment asserts).
4. **stdio** — `FILE*`, buffering modes, the lifted printf/scanf engines wired
   to the fd slots; `snprintf` family; stdin/stdout/stderr over fd 0/1/2.
   *Test:* host round-trip suite for format engines (especially `%.17g`
   float round-trip); QEMU `c-stdio` cooked-line echo once P1's console exists.
5. **mathematics-and-time** — libm lifted wholesale; `strtod`/`strtol`;
   `clock_gettime` (monotonic + realtime over `clock`/`wall_clock`);
   `gmtime`/`mktime`/`strftime` (UTC only — no timezone database);
   `setjmp`/`longjmp`; `qsort`/`bsearch`; `abort`/`assert`. *Test:* host
   `strtod`/`dtoa` vectors against known-hard cases; QEMU `c-time` sanity
   against the wall clock.

Slices 1, 3, 4-host, and 5-host have **no dependency on P1** and can start
immediately; slice 2 and the QEMU halves interleave with P1 as it lands.

**Exit for the layer as a whole** (= P0's exit): `c-hello` and `c-file-io` green
in the QEMU suite, and the host-side computation tests green — at which point P2
(CPython configure) becomes the layer's real integration test.

## Testing strategy: two targets, on purpose

The computation layer is target-independent, so it is unit-tested **on the host**
(built for the host triple, compared against the host libc's answers —
thousands of cheap oracle checks for `strtod`, `printf`, libm edge cases). The
plumbing layer only means anything **on danos**, so it is tested in the QEMU
suite like every other subsystem. Keeping the split explicit stops the slow-QEMU
suite from absorbing tests that a host `zig test` runs in milliseconds.

## Risks and gotchas

- **CPython's configure may insist on pthreads.** WASI-class targets build
  threadless, but verify this *first* in P2 bring-up; the fallback is a
  truthfully-single-threaded `pthread.h` stub set (create returns `EAGAIN`,
  mutexes are no-ops — valid when only one thread can exist). Decide from
  evidence, not assumption.
- **`long double` is x87 80-bit on x86-64.** musl's libm handles it, but keep
  CPython away from it (`configure` uses `double` throughout by default);
  don't hand-write anything touching x87.
- **errno is a contract, not a convention.** The Zig plumbing must map every
  `runtime` error to a POSIX name consistently — CPython turns errno into
  exception types (`FileNotFoundError` is `ENOENT`). One table, tested.
- **`malloc` alignment**: 16-byte minimum on x86-64 (SSE spills in
  compiled C). The free-list must guarantee it from day one; retrofitting
  alignment bugs out of an allocator is misery.
- **Vendoring discipline.** The musl subtree is lift-only — never edited in
  place (patches live beside it if ever needed), pinned to one musl release,
  with the file list documented so a version bump is a re-copy, not an
  archaeology dig.
- **stdio buffering vs. crashes.** Buffered stdout + a crashing program eats
  output — the classic debugging trap. `stderr` stays unbuffered (per C
  standard) and `exit`/`abort` flush; document that `_exit` does not.

## Decisions needing sign-off

- **Lift-from-musl for all pure computation** (vendored, pinned, unedited) rather
  than writing or porting whole-musl.
- **Hand-written danos-native headers**; danos errno values are the numbering.
- **`library/c/` as a build package** producing both the sysroot and the
  first-class C-binary build step.
- The **deliberate-absence table** as the living compatibility matrix, drained
  by the three-stage road above — with exactly one permanent "never": `fork`.
- **Full coverage as the end state** (stage 3), reached by the standing rule
  that every system capability ships with its C spelling — not by a musl port.

## Related

- [python-on-danos-milestones.md](python-on-danos-milestones.md) — this is P0.
- [dynamic-libraries.md](dynamic-libraries.md) — ships in this sysroot
  (`dlfcn.h` + the loader) once its D1 lands.
- [python-on-danos.md](python-on-danos.md) — the design note that scoped the
  layer.
- [character-devices-and-tty.md](character-devices-and-tty.md) — P1; supplies
  the console that makes stdio interactive.
- [zig-self-hosting.md](zig-self-hosting.md) — the `runtime.os` seam the
  plumbing layer will re-target when it exists.
- [os-development/sysv.md](os-development/sysv.md) — the entry stack `crt0`
  bridges.