daniel
/
danos
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects 1 Releases Wiki Activity
danos/docs/c-library-compatibility.md

12 KiB
Raw Blame History

The C library compatibility layer

A design note and milestone plan for libdanos-c — the mini C library that lets zig cc cross-compile C programs for danos. It is milestone P0 of python-on-danos-milestones.md, expanded here the way character-devices-and-tty.md expands P1. CPython is the driving consumer, but the layer is general: any portable C program within its surface should build.

What it is — and the three things it is not

The deliverable is a sysroot: a set of C headers plus a static libdanos-c.a, handed to zig cc -target x86_64-freestanding-none via -isystem and linked into every C binary. Three explicit non-goals keep it small:

  • Not a musl port. Whole-musl assumes Linux syscall semantics at its bottom (the door the Zig roadmap deferred, twice now). We lift musl's pure-computation source files and write a danos-native bottom — see the layer split below.
  • Not full POSIX — yet. Stage 1's surface is "what CPython's minimal configuration and ordinary portable C need" — roughly 100–150 functions — and at that stage absence is a feature: configure scripts probe and adapt, and a linker error is honest. But the end state is a full C compatibility layer (see "The road to full coverage" below); the absence table is a schedule of arrivals, not a wall.
  • Not a second runtime. The library is a thin C-ABI re-spelling of the same danos-native surface runtime already provides. It contains no policy of its own; when the Zig track's runtime.os seam is authored, the libc bottom re-targets it near-mechanically — the fourth appearance of the roadmap's "same surface" symmetry.

One scoping rule sits above all three — the size doctrine: this layer serves applications only. The kernel and the system services never link libdanos-c; they stay danos-native Zig over runtime, small and static, because leanness is an operating-system property. Applications have their own budget and may be as big as they need to be. The libc is how big software lands on danos, never how danos itself is built.

The layer split: lift the mathematics, write the plumbing

The realization that makes 100–150 functions tractable: a libc is two very different kinds of code, and the hard kind is portable.

Layer Contents Source
Pure computation string.h/memcpy family, all of libm, strtod/dtoa, strtol, qsort, ctype tables, gmtime calendar math, the printf/scanf engines, setjmp (a dozen instructions of x86-64 asm) Lift from musl, vendored under library/c/third-party/musl/ (MIT; files compile standalone)
OS plumbing fds (open/read/write/close/lseek/stat/getcwd/chdir/isatty), mmap/munmap, clocks, exit, getenv, getentropy Write in Zig, exporting C ABI over the runtime syscall + VFS client surface
The middle malloc over danos mmap (simple free-list; CPython's arenas sit above), FILE* buffering, errno Write in Zig (small, danos-shaped)
Entry crt0: the existing danos entry shim (sysv.md) bridged to C main(argc, argv, envp), environ initialised, exit flushing stdio Write

Two liftings deserve their own line because getting them wrong is silent corruption rather than a linker error:

  • strtod/float formatting. Python's float repr guarantees shortest round-trip; that property lives entirely in these routines. musl's are correct; an improvised one would be subtly wrong for years. Lift, never write.
  • The stdio engines. musl's vfprintf/vfscanf are self-contained around its FILE abstraction (function-pointer read/write slots), so the whole formatted-I/O engine lifts too — we implement only the fd-backed slots (__stdio_write-shaped) and the buffering glue.

Header policy

Hand-write the headers as danos's own minimal set rather than importing musl's (musl's are entangled with Linux ABI details), borrowing declarations freely. Freestanding compiler headers (stdint.h, stddef.h, stdarg.h, stdbool.h, float.h, limits.h) come from clang via zig cc — do not duplicate them. errno.h values are the danos errno enum re-spelled with POSIX names; there is no Linux numbering to be compatible with, so the enum is the truth.

Deliberate absences, and their planned arrivals — this table is the compatibility matrix, and "the road to full coverage" below is the schedule that empties it:

Absent Arrives with
pthread.h the post-P5 pthread subset over thread_spawn/futex — but see the risk below
real signal.h (beyond no-op signal()/raise stubs) M17 signals-over-IPC in the libc
dlfcn.h dynamic-libraries.md D1
fork/exec*/wait* P5 exposes danos spawn as posix_spawn; fork itself never (see below)
socket.h a future networking track
locale beyond "C" stage 3 evaluation (CPython is UTF-8-mode happy without it)
pipes (pipe()) P5 process-control cluster

The road to full coverage

The layer grows in three stages; only stage 1 is a current milestone (P0), but the stages exist so stage-1 decisions never have to be unmade:

  • Stage 1 — CPython-minimal (P0, the slicing below): ~100–150 functions, static-only, absences honest.
  • Stage 2 — the danos-complete layer: the full hosted C11 standard library, plus every POSIX facility danos semantics support, landing as its enabling milestone lands — pipes and posix_spawn at P5, real signals at M17, the pthread subset after P5, dlfcn.h at dynamic-libraries D1, sockets with networking. Stage 2 is not one milestone but the standing rule that every system capability gets its C spelling when it ships, so the matrix above drains as the OS grows.
  • Stage 3 — ecosystem grade: the point where "portable C program" generally means "builds on danos" (autotools-style probing included). Reaching it is mostly stage 2 compounding, plus the long tail (locale, wide-char, fnmatch/glob/regex — the last three lift from musl like the rest). At this stage, re-evaluate hand-grown-vs-musl-port once with real data; the standing recommendation remains danos-native — musl's bottom assumes Linux syscall semantics, and by stage 3 the danos bottom exists and is tested — with musl continuing as the quarry for computation code.

Two boundaries are permanent and worth stating at every stage: fork never comes — danos is a spawn-shaped OS, and fork's address-space-duplication semantics are hostile to everything from capabilities to threads; software that hard-requires fork (not posix_spawn) stays off the platform. And the public ABI stays the vDSO + IPC protocols — a full libc is a compatibility layer, not a second stable system ABI.

Milestone slicing

  1. sysroot-skeleton — layout under library/c/ (a build package: include/, Zig sources, vendored musl subtree); crt0; string/mem + ctype lifted; a build.zig step making C binaries first-class targets. Test: a C program using only computation links and runs in QEMU (c-hello printing via a raw write extern to debug_write).
  2. fd-plumbing — errno; open/read/write/close/lseek/stat/unlink/mkdir/ rename over the runtime VFS client; getcwd/chdir/getenv/ getentropy arriving as P1 lands them (stubbed truthfully until then: getenv empty, getentropy ENOSYS). Test: QEMU c-file-io — create, write, reopen, read back, stat size + mtime through FAT.
  3. malloc — free-list allocator over danos mmap; calloc/realloc/ free; alignment guarantees documented. Test: host + QEMU allocator torture (interleaved sizes, realloc growth, alignment asserts).
  4. stdio — FILE*, buffering modes, the lifted printf/scanf engines wired to the fd slots; snprintf family; stdin/stdout/stderr over fd 0/1/2. Test: host round-trip suite for format engines (especially %.17g float round-trip); QEMU c-stdio cooked-line echo once P1's console exists.
  5. mathematics-and-time — libm lifted wholesale; strtod/strtol; clock_gettime (monotonic + realtime over clock/wall_clock); gmtime/mktime/strftime (UTC only — no timezone database); setjmp/longjmp; qsort/bsearch; abort/assert. Test: host strtod/dtoa vectors against known-hard cases; QEMU c-time sanity against the wall clock.

Slices 1, 3, 4-host, and 5-host have no dependency on P1 and can start immediately; slice 2 and the QEMU halves interleave with P1 as it lands.

Exit for the layer as a whole (= P0's exit): c-hello and c-file-io green in the QEMU suite, and the host-side computation tests green — at which point P2 (CPython configure) becomes the layer's real integration test.

Testing strategy: two targets, on purpose

The computation layer is target-independent, so it is unit-tested on the host (built for the host triple, compared against the host libc's answers — thousands of cheap oracle checks for strtod, printf, libm edge cases). The plumbing layer only means anything on danos, so it is tested in the QEMU suite like every other subsystem. Keeping the split explicit stops the slow-QEMU suite from absorbing tests that a host zig test runs in milliseconds.

Risks and gotchas

  • CPython's configure may insist on pthreads. WASI-class targets build threadless, but verify this first in P2 bring-up; the fallback is a truthfully-single-threaded pthread.h stub set (create returns EAGAIN, mutexes are no-ops — valid when only one thread can exist). Decide from evidence, not assumption.
  • long double is x87 80-bit on x86-64. musl's libm handles it, but keep CPython away from it (configure uses double throughout by default); don't hand-write anything touching x87.
  • errno is a contract, not a convention. The Zig plumbing must map every runtime error to a POSIX name consistently — CPython turns errno into exception types (FileNotFoundError is ENOENT). One table, tested.
  • malloc alignment: 16-byte minimum on x86-64 (SSE spills in compiled C). The free-list must guarantee it from day one; retrofitting alignment bugs out of an allocator is misery.
  • Vendoring discipline. The musl subtree is lift-only — never edited in place (patches live beside it if ever needed), pinned to one musl release, with the file list documented so a version bump is a re-copy, not an archaeology dig.
  • stdio buffering vs. crashes. Buffered stdout + a crashing program eats output — the classic debugging trap. stderr stays unbuffered (per C standard) and exit/abort flush; document that _exit does not.

Decisions needing sign-off

  • Lift-from-musl for all pure computation (vendored, pinned, unedited) rather than writing or porting whole-musl.
  • Hand-written danos-native headers; danos errno values are the numbering.
  • library/c/ as a build package producing both the sysroot and the first-class C-binary build step.
  • The deliberate-absence table as the living compatibility matrix, drained by the three-stage road above — with exactly one permanent "never": fork.
  • Full coverage as the end state (stage 3), reached by the standing rule that every system capability ships with its C spelling — not by a musl port.

Related