danos/docs/file-system-development/vfs-protocol.md

10 KiB
Raw Blame History

The VFS wire protocol

Status: built and spoken today between file_system (the client) and the filesystem BACKENDS (the FAT server). The mount router lives in the kernel (system/kernel/vfs.zig): fs_resolve routes a path and either serves it directly (the read-only /system initrd mount, via fs_node) or redirects the caller to the owning backend's endpoint plus the rewritten mount-relative path — after which the client speaks THIS protocol to the backend, unchanged. The Zig source of truth is library/protocol/vfs/vfs-protocol.zig (the vfs-protocol module), whose unit test pins a sample of the sizes and values below. This page is the language-neutral wire specification of that contract — what a Rust or C client implements (vdso.md explains why the IPC protocols, not the syscall numbers, are danos's public ABI).

Transport

A VFS exchange is one synchronous IPC rendezvous (ipc_call, docs/ipc.md): the client sends one message and blocks; the server replies with one message. The endpoint comes from the kernel's fs_resolve — which also hands back the path rewritten relative to the mount — not from a registry lookup. (Service id 1, the old userspace router, is retired.)

  • A message is at most 256 bytes (message_maximum).
  • A request is a fixed 32-byte Request header followed by an inline payload of at most 224 bytes (maximum_payload) — a path, or write bytes. There is no multi-message request: paths and single reads/writes must fit, and larger transfers loop (see read / write).
  • A reply is a fixed 24-byte Reply header followed by an inline payload — read bytes, a FileStatus, or a DirectoryEntry.
  • All integers are little-endian; layouts are C layout for x86-64 (extern struct), offsets given below so nothing need be inferred.

The kernel resolves NAMES (the mount table) but never parses these messages — it moves the bytes; file state is entirely the backend's affair. With clients holding backend node ids directly, a backend records each open handle's owner and sweeps a dead client's handles via the published process exit events.

Request header — 32 bytes

offset size field meaning
0 4 operation an Operation value (below)
4 4 padding
8 8 node the server-side open-node id from a prior open; 0 for path-based operations
16 8 offset byte position for read/write; entry index (cursor) for readdir; else 0
24 4 len payload length for path/write operations; requested byte count for read
28 4 flags open flags (below); else 0

Reply header — 24 bytes

offset size field meaning
0 4 status 0 = success, negative = failure (signed)
4 4 padding
8 8 node the new open-node id (for open); else 0
16 4 len reply payload length in bytes
20 4 padding

On failure the backend replies status = -1, and that reply reaches the client directly — there is no party between them on the wire. (Kernel-served paths produce no wire replies at all: fs_resolve/fs_node failures are syscall register statuses.) A richer errno vocabulary is future work — clients must treat any negative status as failure, not match on -1.

Operations

Values are append-only and never renumbered (the same evolution rule every danos protocol follows). Send only values from this table: the shipped server decodes the operation into an exhaustive enum, so an out-of-range value is not answered with a status = -1 reply — it trips a safety check in safe builds and is undefined otherwise. (The -1 replies cover recognised but refused operations, such as mount sent to a backend.)

value operation request payload reply
0 open the path (len = its length), flags as below node = open-node id
1 close — (node set) status only
2 read — (node, offset, len = wanted count) len bytes read, payload = the bytes; len 0 at end of file
3 write the bytes (node, offset, len = count) len = bytes accepted (may be short — loop)
4 status — (node set) payload = FileStatus (24 bytes)
5 readdir — (node = a directory, offset = cursor) payload = one DirectoryEntry + name; len 0 at end
6 mount the mount-point path; the backend endpoint rides as the call's capability status only
7 unmount the mount-point path status only
8 mkdir the path status only
9 unlink the path status only
10 rename old path, one 0x00, new path (len = total) status only

Notes per operation:

  • open — the path is the mount-relative path fs_resolve handed back (absolute-shaped: /notes.txt under fat's /mnt/usb mount). Bare names (greeting) resolve nowhere — the flat ramfs is retired, and fs_resolve refuses non-absolute paths. The returned node is the backend's own open-node id: with the router in the kernel there is no forwarding table, and clients hold backend ids directly (see Lifetimes and trust).
  • read / write — a single exchange moves at most 224 bytes (maximum_payload); the client loops, advancing offset by the returned len, until done (read) or the slice is written (write). A write reply shorter than requested is progress, not an error; a len of 0 means no forward progress — stop rather than spin.
  • readdiroffset is a cursor: the entry index, not a byte position. Each call returns exactly one entry; the client increments the cursor by 1. A reply with len 0 is end-of-directory. The directory must have been opened with the directory flag.
  • mount / unmount — RETIRED from the wire: mounting is the fs_mount syscall now (a filesystem server passes its endpoint handle; possession is the capability, exactly the trust of the old cap-passing op). The op numbers stay reserved. Mount-prefix semantics are unchanged: prefixes match at path boundaries only (/mnt/usb never captures /mnt/usbextra), the longest matching prefix wins, and an optional backend-side rewrite prefix maps a mount into the backend's namespace (fat serves /mnt/usb from its volume root and /var from its /var subtree).
  • rename — same-directory rename only: the backend compares the old and new parent paths and refuses a mismatch. The client (file_system) refuses earlier when the two paths resolve to different backend endpoints, but that check is coarser than "one mount" — one endpoint can serve several mounts (fat serves /mnt/usb and /var), so a cross-mount rename reaches the backend and fails on its same-directory check.

Open flags

Bitwise OR in Request.flags, meaningful for open only:

bit name meaning
1 create create the file if it does not exist
2 directory open a directory node for readdir rather than a file
4 truncate truncate an existing file to zero length on open (replace, don't overwrite in place)

FileStatus — 24 bytes (the status reply payload)

offset size field meaning
0 8 size file size in bytes
8 4 kind a NodeKind value
12 4 padding
16 8 mtime modification time, Unix epoch seconds UTC; 0 if the backend keeps none

DirectoryEntry — 16 bytes + name (the readdir reply payload)

offset size field meaning
0 4 kind a NodeKind value
4 4 name_len length of the name that follows
8 8 size the entry's size in bytes
16 name_len name the entry's name, not NUL-terminated

NodeKind

Aligned to the node-kind table in the file-system hierarchy (docs/file-system-development/file-system-hierarchy.md):

value kind
0 regular file
1 directory
2 character device
3 block device
4 symbolic link
5 fifo
6 socket
7 protocol

Clients should map unknown values to regular rather than reject — the table can grow. Kind 6 (socket) keeps its wire value but is retired from the design — a named rendezvous point is exactly what a protocol node is, landed as value 7 with the protocol namespace (docs/os-development/protocol-namespace.md). character_device (stream semantics — the tty/console shape) and block_device (raw sector-addressed volumes, reserved) remain part of the design.

An open reply may carry a capability

open rides ipc_call, whose reply direction can hand back an endpoint capability alongside the Reply header. A file backend never uses it — FAT answers with a node id and nothing else — but a synthetic backend does: opening a protocol node returns the provider's endpoint, and possession of that endpoint is the channel. The convention is per-backend, not per-operation, so a client that opens an ordinary file simply receives no capability, exactly as before.

Lifetimes and trust

Open-node ids live in the backend. A client that dies without closing leaks nothing permanently: the backend (the FAT server) subscribes to the kernel's published process-exit events (docs/process-lifecycle.md) and releases a dead client's handles. The kernel VFS root needs no sweep at all — its node tokens are permanent for a boot and carry no open state. Ids are plain integers, not capabilities — a backend trusts its callers with each other's ids today, which is acceptable while every client is part of the system image and worth revisiting (per-client id namespaces) before third-party binaries arrive.

Evolution rules

What a non-Zig implementation may rely on, and what it must not:

  • Operation values, flag bits, NodeKind values, and struct layouts are append-only and frozen once shipped. The unit test in library/protocol/vfs/vfs-protocol.zig pins a sample of them (the DirectoryEntry size, NodeKind 01 and 67, Operation values 0, 4 and 5); this page is the full record of the frozen values.
  • The 256-byte message ceiling is a property of the current IPC transport, not a promise; clients should read maximum_payload-shaped limits from the reply lengths they actually get (loop-until-done), not hard-code 224.
  • Negative statuses beyond -1 will appear (an errno vocabulary); success is exactly 0.