daniel/danos
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

re-org docs

Browse Source
This commit is contained in:
Daniel Samson
2026-07-23 00:24:01 +01:00
parent f023f1cfd6
commit 52d6e372fd
53 changed files with 271 additions and 389 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/file-system-development/danos-file-system-hierarchy-FSH.md
+128
View File
@@ -0,0 +1,128 @@
# DanOS Filesystem Hierarchy Standard (DFHS)
Most modern Unix and Unix-like operating systems follow the FHS. DanOS has its own FHS structure which extends the unix FHS. Root path resolution is provided by the kernel-resident VFS root (`fs_resolve`, `system/kernel/vfs.zig`); mounted filesystem servers serve the subtrees they own.
## Directory structure
| Path | Description |
|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| / | Primary hierarchy root and root directory of the entire file system hierarchy. |
| /bin | Essential command binaries that need to be available in single-user mode, including to bring up the system or repair it, for all users (e.g., cat, ls, cp). |
| /boot | Boot loader files (e.g., EFI, initial-ramdisk.img ). |
| /dev | POSIX Device files (e.g., /dev/null, /dev/disk0, /dev/tty, /dev/random). |
| /etc | Host-specific system-wide configuration files. |
| /home | Users' home directories, containing saved files, personal settings, etc. |
| /lib | Libraries essential for the binaries in /bin and /sbin. eg realtime, system, ipc etc. |
| /sbin | Essential system binaries (e.g init) |
| /srv | Site-specific data served by this system, such as data and scripts for web servers, data offered by FTP servers, and repositories for version control systems |
| /system | DanOS operating system files (similar idea to C:\Windows). A true representation of danos — its layout mirrors the source tree, so `/system` is what danos *is*. |
| /system/devices | danos virtual device tree e.g. similar to /sys on linux but with danos device tree conventions (the structures in the devices module) |
| /system/drivers | driver binaries, one sub-project each (e.g. /system/drivers/pci-bus, /system/drivers/ps2-bus) |
| /system/services | system-service binaries — init, the FAT server, and other user-mode servers (e.g. /system/services/init, /system/services/fat) |
| /system/kernel | the kernel image |
| /test | Test fixtures for the QEMU integration suite. Read-only and initrd-backed like /system, and its layout likewise mirrors the source tree (the repo's test/ directory). Present on development and test images; a volume without it still boots. |
| /test/system/services | test-fixture binaries (e.g. /test/system/services/vfs-test, /test/system/services/thread-test) — the same path in the repo source tree and on the boot volume |
| /tmp | Directory for temporary files (see also /var/tmp). Often not preserved between system reboots and may be severely size-restricted. |
| /usr | Secondary hierarchy for read-only user data; contains the majority of (multi-)user utilities and applications. Should be shareable and read-only. |
| /var | Variable files: files whose content is expected to continually change during normal operation of the system, such as logs, spool files, and temporary e-mail files. |
## File types
POSIX specifies the long format of the ls command to represent the Unix file type as the first letter for an entry.
| type | symbol | Description |
|-------------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| regular | - | An ordinary file holding an uninterpreted byte stream. Reads and writes are positional, and the file grows on demand (e.g., a binary in /bin, a config file in /etc). |
| directory | d | A container mapping names to other files. It may only be modified through directory operations, never written to directly. |
| symbolic link | l | A file whose contents are a path that is resolved in its place. The target need not exist, and may cross mount points. |
| FIFO special | p | A named pipe: an in-order byte stream between processes, where writers block until a reader opens the other end. |
| block special | b | A device node addressed in fixed-size blocks with the kernel free to buffer and reorder access (e.g., /dev/disk0). |
| character special | c | A device node addressed as an unbuffered byte stream, delivered to the driver in order (e.g., /dev/tty, /dev/null). |
| socket | s | A named endpoint for bidirectional message-passing between processes, bound to a path rather than an address. |
## /dev
`/dev` holds the names through which processes reach devices. It is deliberately not
the device tree: the tree — every node discovered by ACPI or PCI enumeration, with its
resources and its parent — lives under [/system/devices](#directory-structure) and is
addressed by device id. `/dev` is the much smaller set of devices that have a driver
willing to serve them, addressed by name.
A device node is not a file the VFS can read. The bytes live in a driver process
([drivers.md](../device-driver-development-guide/drivers.md)), so opening a `/dev` name has to resolve to that driver's
IPC endpoint, and subsequent reads and writes are calls against it. Resolve-to-endpoint
is exactly what the kernel's `fs_resolve` already does for any mounted backend, and
`FileStatus.kind` is the field that marks a device node; **what is not implemented today
is `/dev` itself** — no service mounts it. (The flat eight-node ramfs this section once
described is retired: the kernel-resident VFS root in `system/kernel/vfs.zig` serves a
read-only initrd mount per top-level tree — `/system`, and `/test` on images that carry
the fixtures — with real directories and node kinds, and filesystem
backends such as the FAT server mount the rest.) The three sections below describe the
intended shape, and are honest about which parts the kernel can already support.
### Character devices
A character device is a byte stream with no addressable position: bytes are delivered
to the driver in the order written, and a read consumes what is there. Terminals,
serial lines, keyboards and mice are all of this shape. These are the natural first
device nodes in danos, because a character driver needs nothing the kernel doesn't
already provide — it claims its device, maps its registers with `mmio_map`, and blocks
on `replyWait` for either an interrupt or a client request. `system/drivers/ps2-bus/ps2-bus.zig`
is already that program, minus the file-node client half.
The obstacle was never the file type; it is which hardware a ring-3 driver can reach.
Direct `in`/`out` from user space is still a #GP (no TSS I/O bitmap, IOPL never raised),
but a driver no longer needs it: **`io_read`/`io_write`** grant port access the same way
`mmio_map` grants memory — gated by `device_claim` and the device's discovered `io_port`
resource. So the 16550 UART at `0x3F8` and the PS/2 controller at `0x60`/`0x64` (and thus
`/dev/ttyS0` and a keyboard node) are now writable as ordinary ring-3 drivers; the
low-rate legacy hardware that needs port I/O is fine with a syscall per access. A
memory-mapped device such as the framebuffer, needing no port I/O at all, remains the
easiest first entry.
### Block devices
A block device is addressed in fixed-size blocks and, unlike a character device, the
layer above is free to buffer, reorder, coalesce and retry requests against it. Disks
and other persistent storage are the whole population of this class.
A block driver is now **writable, but not yet memory-safe.** Every storage controller
worth naming is a bus master: it is programmed by handing it the physical address of a
descriptor ring and left to read and write memory on its own. That ring is exactly what
**`dma_alloc`** now provides — physically contiguous, pinned, uncacheable, with its
physical address disclosed — and **`/lib/device/mmio`**'s barriers order the descriptor writes
against the doorbell, and **`msi_bind`** delivers completions. So an AHCI or NVMe driver
can be written today (the M14/M15 work in [driver-model.md](../device-driver-development-guide/driver-model.md); the earlier
"cannot host a block driver at all" is no longer true).
What is *not* yet true is that it is safe. A device programmed with an arbitrary physical
address writes to arbitrary physical memory, and page tables do not sit between a device
and RAM — an IOMMU does. The IOMMU is now *detected* (M16), but no translation domains
are programmed, so granting a DMA-capable device to a driver process is still equivalent
to granting ring 0. Until per-device domains confine a driver's DMA to the buffers it
`dma_alloc`'d, a block driver works but forfeits the isolation that motivates user-space
drivers — enforcement is the next step, and lands with that first driver. A ramdisk over
the initial ramdisk remains the one block-shaped thing that needs no driver process at all.
### Pseudo-devices
A pseudo-device has the interface of a device and no hardware behind it: `/dev/null`
discarding writes and reading as end-of-file, `/dev/zero` reading as an endless run of
zero bytes, `/dev/full` failing writes with `ENOSPC`, `/dev/random` and `/dev/urandom`
yielding unpredictable bytes.
These are the only `/dev` entries danos can implement immediately, and they are the
sensible place to start, because they are exactly the entries that need no driver
process, no `device_claim`, no MMIO grant and no interrupt. A future pseudo-device
service would answer them out of its own address space — `null` and `zero` are a few
lines each in its `read` and `write` handlers — and mount itself at `/dev` the way the
FAT server mounts `/mnt/usb`. The two pieces of structure every later device node
depends on (and that the flat ramfs of the time lacked) exist now: directories, so that
`/dev/null` is a path rather than a name; and a populated `FileStatus.kind`, so that a
caller can tell a character device from a regular file.
`/dev/random` is the one that is not free. It needs an entropy source, and the honest
options on this kernel are `RDRAND`/`RDSEED` where CPUID advertises them, and the HPET
counter's low bits as a poor fallback. Neither is a seeded CSPRNG, and a `/dev/random`
that is merely unpredictable-looking is worse than none — nothing should be keyed from
it until it is a real one.
docs/file-system-development/vfs-protocol.md
+192
View File
@@ -0,0 +1,192 @@
# The VFS wire protocol
> **Status:** built and spoken today between `file_system` (the client) and the
> filesystem BACKENDS (the FAT server). The mount router lives in the
> **kernel** (`system/kernel/vfs.zig`): `fs_resolve` routes a path and either
> serves it directly (the read-only /system initrd mount, via `fs_node`) or
> redirects the caller to the owning backend's endpoint plus the rewritten
> mount-relative path — after which the client speaks THIS protocol to the
> backend, unchanged. The Zig source of truth is `library/protocol/vfs/vfs-protocol.zig`
> (the `vfs-protocol` module), whose unit test pins a sample of the sizes
> and values below. This page is the **language-neutral wire specification**
> of that contract — what a Rust or C client implements ([vdso.md](../os-development-guide/vdso.md)
> explains why the IPC protocols, not the syscall numbers, are danos's
> public ABI).
## Transport
A VFS exchange is one synchronous IPC rendezvous (`ipc_call`,
docs/ipc.md): the client sends one message and blocks; the server replies
with one message. The endpoint comes from the kernel's `fs_resolve` — which
also hands back the path rewritten relative to the mount — not from a
registry lookup. (Service id 1, the old userspace router, is retired.)
- A message is at most **256 bytes** (`message_maximum`).
- A request is a fixed 32-byte **Request** header followed by an inline
payload of at most **224 bytes** (`maximum_payload`) — a path, or write
bytes. There is no multi-message request: paths and single reads/writes
must fit, and larger transfers loop (see *read* / *write*).
- A reply is a fixed 24-byte **Reply** header followed by an inline payload —
read bytes, a `FileStatus`, or a `DirectoryEntry`.
- All integers are **little-endian**; layouts are C layout for x86-64
(`extern struct`), offsets given below so nothing need be inferred.
The kernel resolves NAMES (the mount table) but never parses these
messages — it moves the bytes; file state is entirely the backend's affair.
With clients holding backend node ids directly, a backend records each open
handle's owner and sweeps a dead client's handles via the published process
exit events.
## Request header — 32 bytes
| offset | size | field | meaning |
|-------:|-----:|-------|---------|
| 0 | 4 | `operation` | an **Operation** value (below) |
| 4 | 4 | — | padding |
| 8 | 8 | `node` | the server-side open-node id from a prior `open`; 0 for path-based operations |
| 16 | 8 | `offset` | byte position for read/write; entry index (cursor) for readdir; else 0 |
| 24 | 4 | `len` | payload length for path/write operations; requested byte count for read |
| 28 | 4 | `flags` | open flags (below); else 0 |
## Reply header — 24 bytes
| offset | size | field | meaning |
|-------:|-----:|-------|---------|
| 0 | 4 | `status` | **0 = success**, negative = failure (signed) |
| 4 | 4 | — | padding |
| 8 | 8 | `node` | the new open-node id (for `open`); else 0 |
| 16 | 4 | `len` | reply payload length in bytes |
| 20 | 4 | — | padding |
On failure the backend replies `status = -1`, and that reply reaches the
client directly — there is no party between them on the wire. (Kernel-served
paths produce no wire replies at all: `fs_resolve`/`fs_node` failures are
syscall register statuses.) A richer errno vocabulary is future work —
clients must treat *any* negative status as failure, not match on -1.
## Operations
Values are append-only and never renumbered (the same evolution rule every
danos protocol follows). Send only values from this table: the shipped server
decodes the operation into an exhaustive enum, so an out-of-range value is
not answered with a `status = -1` reply — it trips a safety check in safe
builds and is undefined otherwise. (The `-1` replies cover recognised but
refused operations, such as `mount` sent to a backend.)
| value | operation | request payload | reply |
|------:|-----------|-----------------|-------|
| 0 | `open` | the path (`len` = its length), `flags` as below | `node` = open-node id |
| 1 | `close` | — (`node` set) | status only |
| 2 | `read` | — (`node`, `offset`, `len` = wanted count) | `len` bytes read, payload = the bytes; `len` 0 at end of file |
| 3 | `write` | the bytes (`node`, `offset`, `len` = count) | `len` = bytes accepted (may be short — loop) |
| 4 | `status` | — (`node` set) | payload = **FileStatus** (24 bytes) |
| 5 | `readdir` | — (`node` = a directory, `offset` = cursor) | payload = one **DirectoryEntry** + name; `len` 0 at end |
| 6 | `mount` | the mount-point path; the backend endpoint rides as the call's **capability** | status only |
| 7 | `unmount` | the mount-point path | status only |
| 8 | `mkdir` | the path | status only |
| 9 | `unlink` | the path | status only |
| 10 | `rename` | old path, one `0x00`, new path (`len` = total) | status only |
Notes per operation:
- **open** — the path is the mount-relative path `fs_resolve` handed back
(absolute-shaped: `/notes.txt` under fat's `/mnt/usb` mount). Bare names
(`greeting`) resolve nowhere — the flat ramfs is retired, and `fs_resolve`
refuses non-absolute paths. The returned `node` is the *backend's* own
open-node id: with the router in the kernel there is no forwarding table,
and clients hold backend ids directly (see *Lifetimes and trust*).
- **read / write** — a single exchange moves at most 224 bytes
(`maximum_payload`); the client loops, advancing `offset` by the returned
`len`, until done (read) or the slice is written (write). A `write` reply
shorter than requested is progress, not an error; a `len` of 0 means no
forward progress — stop rather than spin.
- **readdir** — `offset` is a **cursor: the entry index**, not a byte
position. Each call returns exactly one entry; the client increments the
cursor by 1. A reply with `len` 0 is end-of-directory. The directory must
have been opened with the `directory` flag.
- **mount / unmount** — RETIRED from the wire: mounting is the `fs_mount`
syscall now (a filesystem server passes its endpoint handle; possession is
the capability, exactly the trust of the old cap-passing op). The op
numbers stay reserved. Mount-prefix semantics are unchanged: prefixes
match at path boundaries only (`/mnt/usb` never captures `/mnt/usbextra`),
the longest matching prefix wins, and an optional backend-side rewrite
prefix maps a mount into the backend's namespace (fat serves `/mnt/usb`
from its volume root and `/var` from its `/var` subtree).
- **rename** — same-directory rename only: the backend compares the old and
new parent paths and refuses a mismatch. The client (`file_system`) refuses
earlier when the two paths resolve to different backend endpoints, but that
check is coarser than "one mount" — one endpoint can serve several mounts
(fat serves `/mnt/usb` and `/var`), so a cross-mount rename reaches the
backend and fails on its same-directory check.
## Open flags
Bitwise OR in `Request.flags`, meaningful for `open` only:
| bit | name | meaning |
|----:|------|---------|
| 1 | `create` | create the file if it does not exist |
| 2 | `directory` | open a directory node for `readdir` rather than a file |
| 4 | `truncate` | truncate an existing file to zero length on open (replace, don't overwrite in place) |
## FileStatus — 24 bytes (the `status` reply payload)
| offset | size | field | meaning |
|-------:|-----:|-------|---------|
| 0 | 8 | `size` | file size in bytes |
| 8 | 4 | `kind` | a **NodeKind** value |
| 12 | 4 | — | padding |
| 16 | 8 | `mtime` | modification time, Unix epoch seconds UTC; 0 if the backend keeps none |
## DirectoryEntry — 16 bytes + name (the `readdir` reply payload)
| offset | size | field | meaning |
|-------:|-----:|-------|---------|
| 0 | 4 | `kind` | a **NodeKind** value |
| 4 | 4 | `name_len` | length of the name that follows |
| 8 | 8 | `size` | the entry's size in bytes |
| 16 | `name_len` | name | the entry's name, not NUL-terminated |
## NodeKind
Aligned to the FSH file-type table
(docs/danos-file-system-hierarchy-FSH.md):
| value | kind |
|------:|------|
| 0 | regular file |
| 1 | directory |
| 2 | character device |
| 3 | block device |
| 4 | symbolic link |
| 5 | fifo |
| 6 | socket |
Clients should map unknown values to *regular* rather than reject — the
table can grow.
## Lifetimes and trust
Open-node ids live in the backend. A client that dies without closing leaks
nothing permanently: the backend (the FAT server) subscribes to the kernel's
published process-exit events (docs/process-lifecycle.md) and releases a dead
client's handles. The kernel VFS root needs no sweep at all — its node tokens
are permanent for a boot and carry no open state. Ids are plain integers, not
capabilities — a backend trusts its callers with each other's ids today, which
is acceptable while every client is part of the system image and worth
revisiting (per-client id namespaces) before third-party binaries arrive.
## Evolution rules
What a non-Zig implementation may rely on, and what it must not:
- Operation values, flag bits, `NodeKind` values, and struct layouts are
**append-only and frozen once shipped**. The unit test in
`library/protocol/vfs/vfs-protocol.zig` pins a sample of them (the `DirectoryEntry`
size, `NodeKind` 0–1, `Operation` values 0, 4 and 5); this page is the
full record of the frozen values.
- The 256-byte message ceiling is a property of the current IPC transport,
not a promise; clients should read `maximum_payload`-shaped limits from the
reply lengths they actually get (loop-until-done), not hard-code 224.
- Negative statuses beyond -1 will appear (an errno vocabulary); success is
exactly 0.