6 Commits
Author SHA1 Message Date
Daniel Samson 6d4992ae02 docs: storage — the mount map is built; the path is the id (S2)
Flip the storage docs to match S2: filesystems.csv (signature -> binary) and
volumes.csv (identity -> optional override) are built; a volume's mount path IS
its content id (/volumes/<id>), never a port name; the label is display metadata
a `volumes` query returns. fat receives its mount path via argv[2] rather than
hardcoding it. Still pending: rung 2 (filesystem UUID, needs a non-FAT engine),
multi-volume (fat's boot rewrites stay unconditional until S3), medium_changed
consumption, remount bench-verification.
2026-08-10 00:49:32 +01:00
Daniel Samson df61693065 fat: mount at the id-path from argv; migrate /volumes/usb -> id-path (S2)
The flip that makes the mount path the volume's content id. fat retires its
hardcoded fat_mounts: it reads its mount path from argv[2] (the volume manager
hands it the id-path, e.g. /volumes/fat-12345678, from the FAT serial), mounts
its volume root there, and installs the /system/configuration + /system/logs FHS
rewrites so /system/logs persistence stays decoupled from which volume backs it.
The rewrites are unconditional this increment (the single volume IS the boot
volume); S3 makes them content-conditional across N volumes. Every /volumes/usb
reference migrates to /volumes/fat-12345678 in one commit — the fat-test,
badge-scope-test, and vfs-test fixtures and the four QEMU regexes — plus a new
volume-identity-name case asserting the id-path mount and the /system/logs
rewrite. Discrimination: the regexes now require /volumes/fat-12345678, which the
old hardcoded fat never emitted (it mounted /volumes/usb). Full suite 128/128.
2026-08-10 00:47:19 +01:00
Daniel Samson f1e79d0eeb volume-manager: the volumes query verb — read a volume's id, path, and label (S2)
The mechanism the id/label split needs: a `volumes` verb whose reply packs the
mounted volume's {id, mount_path, label} into the tail (VolumeInfo.encode/decode
— three length-prefixed strings). Software keys on the id (the mount path is
/volumes/<id>); a shell or file manager shows the label — the database id/name
split made a query. The VM's onVolumes answers from the mounted volume, empty
reply if none. Two host round-trip tests (encode/decode; too-small buffer and
short-tail rejection). No runtime consumer yet — the first is a userspace shell;
the hello handshake is unaffected (fat-mount/volume-probe green).
2026-08-10 00:17:22 +01:00
Daniel Samson 167e9c7a9e volume-manager: load the mount map; pick binary by signature, compose the id-path (S2)
The volume manager reads its policy from configuration at boot (loadTables,
mirroring the device-manager registry load): filesystems.csv (content signature
-> service binary) and volumes.csv (optional id -> mount-prefix override), each
held in a static source buffer with declared bounds. On probe it picks the
binary from the volume's signature (unserved + logged if no row matches, like an
unbound device) and composes the mount path — a volumes.csv override, else the
default /volumes/<id> from volume-map.idString — then spawns that binary with
argv {volume-id, mount-prefix}. Behavior-preserving: fat still ignores argv[2..]
and uses its hardcoded mounts, the binary resolves to /system/services/fat, so
the FULL suite stays green (127/127); the flip to argv-driven mounts and the
/volumes/usb -> id-path migration land in step 5.
2026-08-10 00:12:33 +01:00
Daniel Samson 5bfdb75e12 volume-manager: the id-path deriver + volumes.csv override (S2)
NEW volume-map.zig: idString(identity) renders a volume's content identity into
its stable mount id-string — gpt-<32hex>, fat-<8hex>, mbr-<sig>-<index> — the
token whose default mount path is /volumes/<id>, so the path IS the id and never
a port or a label; two volumes that share a label get distinct ids
automatically. parse() reads volumes.csv (id, mount_prefix) into OPTIONAL
overrides; overrideFor returns a pinned prefix or null (the volume takes its
default /volumes/<id>). id_maximum is a declared bound; the fixture sizes are
named. Three host tests (each rung's id token; override hit/miss; malformed rows
counted), wired into the VM package test step with csv. Not yet consumed by the
binary — that lands when the VM loads the tables and composes paths (step 4).
2026-08-09 23:55:08 +01:00
Daniel Samson 3fb8a9b96f volume-manager: content signature + the filesystems.csv map (S2)
partition.Volume gains a FilesystemKind signature (today .fat for every probed
volume; S4 adds a real VBR recognizer for exFAT) — the seam filesystems.csv keys
on to choose a service binary. New filesystem-map.zig parses
`/system/configuration/filesystems.csv` (signature, binary) into rules and
match()es a signature to its binary, mirroring the device registry: a signature
no row matches goes unserved, never guessed; slices point into the source
buffer. Three host tests (fat->binary, the binary is data-driven not hardcoded,
malformed rows counted); the test rule buffer is a named fixture size so the
bounds gate stays quiet. The VM's build gains the csv dependency and wires the
filesystem-map test into its package test step. Not yet consumed by the binary —
that lands when the VM loads the tables (step 4).
2026-08-09 23:50:15 +01:00
17 changed files with 640 additions and 88 deletions
+5
View File
@@ -317,6 +317,11 @@ pub fn build(b: *std.Build) void {
// out of the same read-only initrd, before it spawns anything — the registrar // out of the same read-only initrd, before it spawns anything — the registrar
// has to know its policy before the first provider asks. // has to know its policy before the first provider asks.
bundled_list.append(b.allocator, .{ .path = "system/configuration/protocol.csv", .binary = b.path("system/configuration/protocol.csv") }) catch @panic("OOM"); bundled_list.append(b.allocator, .{ .path = "system/configuration/protocol.csv", .binary = b.path("system/configuration/protocol.csv") }) catch @panic("OOM");
// The storage mount map (docs/file-system-development/storage-architecture.md):
// filesystems.csv (content signature -> service binary) and volumes.csv (the
// optional id -> mount-prefix override), both read by the volume manager.
bundled_list.append(b.allocator, .{ .path = "system/configuration/filesystems.csv", .binary = b.path("system/configuration/filesystems.csv") }) catch @panic("OOM");
bundled_list.append(b.allocator, .{ .path = "system/configuration/volumes.csv", .binary = b.path("system/configuration/volumes.csv") }) catch @panic("OOM");
// A no-option build assumes neither -Dtest-case nor -Ddiagnose: it ships the // A no-option build assumes neither -Dtest-case nor -Ddiagnose: it ships the
// production set only. The userspace test fixtures under /test join in only // production set only. The userspace test fixtures under /test join in only
// for a test build — which the QEMU harness signals by passing // for a test build — which the QEMU harness signals by passing
@@ -3,18 +3,23 @@
> **Status:** the layered model below is the settled design > **Status:** the layered model below is the settled design
> ([storage-design-rationale.md](storage-design-rationale.md) records how it was > ([storage-design-rationale.md](storage-design-rationale.md) records how it was
> reached, and [volume-manager-plan.md](../volume-manager-plan.md) how it was > reached, and [volume-manager-plan.md](../volume-manager-plan.md) how it was
> built). **Built** (the volume-manager track, V0–V4): the data path, the driver > built). **Built** (V0–V4 + the storage-stack S1/S2): the data path, the driver
> range confinement (per-sender clamp + the confinement gate), the `medium_changed` > range confinement (per-sender clamp + the confinement gate), the `medium_changed`
> presence event, the volume manager itself — it probes the partition table, > presence event, the volume manager itself — it probes the partition table,
> confines each filesystem to its partition, spawns one filesystem per volume, and > confines each filesystem to its partition, spawns one filesystem per volume, and
> supervises it — and the removal half of the lifecycle (a pulled stick unmounts). > supervises it — the removal half of the lifecycle (a pulled stick unmounts), the
> **Still pending**: the fuller identity ladder and the `volumes.csv` mount map, > identity ladder (GPT GUID + name, FAT serial + label, MBR), and the mount map:
> multi-volume (one FAT volume today), the volume manager *consuming* > `filesystems.csv` (signature → binary) + `volumes.csv` (identity → optional
> `medium_changed` (removal is detected by device-presence polling; the event is > override), a volume's mount path IS its content id (`/volumes/<id>`), with the
> published but only a card-reader medium change needs the subscription), and the > label as display metadata a `volumes` query returns. **Still pending**: the
> remount-on-replug end-to-end (the logic is in place; QEMU can't re-present the > `filesystem UUID` rung (needs a non-FAT engine), multi-volume (one FAT volume
> boot-controller device, so it is bench-verified). A few markers below are left > today; fat's boot rewrites are unconditional until S3 makes them
> where a duty is still pending. > content-conditional), the volume manager *consuming* `medium_changed` (removal
> is detected by device-presence polling; the event is published but only a
> card-reader medium change needs the subscription), and the remount-on-replug
> end-to-end (the logic is in place; QEMU can't re-present the boot-controller
> device, so it is bench-verified). A few markers below are left where a duty is
> still pending.
## The model ## The model
@@ -85,16 +90,17 @@ manager's tree for a storage provider; when one appears it consumer-hellos for
the block channel, reads the partition table and the first blocks itself the block channel, reads the partition table and the first blocks itself
(**it** is the prober), defines the volume's sub-range on the driver, spawns the (**it** is the prober), defines the volume's sub-range on the driver, spawns the
matching filesystem service confined to that range, and supervises it (backoff, matching filesystem service confined to that range, and supervises it (backoff,
crash-loop cap). *(Pending)*: it decides mount placement from `volumes.csv` and crash-loop cap). *(Built)*: it picks the filesystem binary from
picks the filesystem binary from `filesystems.csv` — today it hands every `filesystems.csv` by the volume's content signature, and mounts the volume at its
FAT-shaped volume to the FAT service and the FAT service carries hardcoded mount content id (`/volumes/<id>`) — or a `volumes.csv` override. The label is display
prefixes. Those tables are CSV configuration, read by it (the policy), enforced metadata the `volumes` query returns, never the path. Those tables are CSV
by nobody else: configuration, read by it (the policy), enforced by nobody else:
- `filesystems.csv` *(pending)* — content signature → filesystem binary. Adding - `filesystems.csv` *(built)* — content signature → filesystem binary. Adding
a filesystem adds a row. a filesystem adds a row.
- `volumes.csv` *(pending)* — the mount map, danos's fstab: **volume identity → mount - `volumes.csv` *(built)* — the mount map, danos's fstab: an OPTIONAL **volume
prefix**, keyed on content identity and never on port, path, or arrival identity → mount prefix** override (a volume with no row mounts at its default
`/volumes/<id>`), keyed on content identity and never on port, path, or arrival
order (the lesson of Linux's `/dev/sda1`-era fstab, which broke on every order (the lesson of Linux's `/dev/sda1`-era fstab, which broke on every
port move until `UUID=` replaced it). Identity is read off the medium by port move until `UUID=` replaced it). Identity is read off the medium by
the prober, strongest first: GPT partition GUID → filesystem UUID → FAT the prober, strongest first: GPT partition GUID → filesystem UUID → FAT
@@ -105,8 +111,8 @@ by nobody else:
identity (cloned sticks, together) is policy: first keeps the name, the identity (cloned sticks, together) is policy: first keeps the name, the
second mounts suffixed and is logged loudly. The boot volume is the second mounts suffixed and is logged loudly. The boot volume is the
recorded identity of the volume carrying `/system/configuration` and recorded identity of the volume carrying `/system/configuration` and
`/system/logs`, findable on any port. Unknown volumes mount under `/system/logs`, findable on any port. Every volume's default mount is
`/volumes/<derived name>`. `/volumes/<id>` — its rendered content identity.
**Filesystem service** (the FAT service today; one process per volume): the **Filesystem service** (the FAT service today; one process per volume): the
proven unit — block-client + engine + file-protocol provider in one binary. It proven unit — block-client + engine + file-protocol provider in one binary. It
@@ -114,12 +120,13 @@ receives its block channel at spawn; it never discovers devices. It registers
its own mounts with the kernel; its write cache lives inside the process, so a its own mounts with the kernel; its write cache lives inside the process, so a
write error is observed by the code that owns the volume and surfaces on the write error is observed by the code that owns the volume and surfaces on the
owning channel (the anti-fsyncgate rule — never a system-wide dirty pool). owning channel (the anti-fsyncgate rule — never a system-wide dirty pool).
*(Today, interim:)* fat still hardcodes its mount prefixes (`/volumes/usb` plus *(Built:)* fat receives its mount path as `argv[2]` from the volume manager (the
the two boot-volume hierarchy subtrees it rewrites in place); a `volumes.csv` volume's id-path, e.g. `/volumes/fat-12345678`) and mounts its root there, plus
mount map will migrate that to the volume manager. It no longer self-acquires a the two `/system` hierarchy rewrites it installs in place (unconditional this
volume — the V3b flip made it receive its volume id at spawn and its block increment; S3 makes them content-conditional across volumes). It no longer
channel from the volume manager's hello reply, consistent with "it never self-acquires a volume — the V3b flip made it receive its volume id and block
discovers devices" above. channel from the volume manager, consistent with "it never discovers devices"
above.
**Kernel** (mechanism only): the mount table routes paths to backend **Kernel** (mechanism only): the mount table routes paths to backend
endpoints — resolve and redirect, never data. Remount-replace is the restart endpoints — resolve and redirect, never data. Remount-replace is the restart
@@ -218,9 +218,10 @@ matrix-proven shape; genuinely open.
**content identity, never port or discovery order**. Build status: rungs 1, **content identity, never port or discovery order**. Build status: rungs 1,
3, and 4 (GPT partition GUID, FAT serial + label, MBR signature + index) are 3, and 4 (GPT partition GUID, FAT serial + label, MBR signature + index) are
implemented (S1); rung 2 waits on a non-FAT engine. The `volumes.csv` map and implemented (S1); rung 2 waits on a non-FAT engine. The `volumes.csv` map and
the id-derived mount path land with S2, so today a single volume still mounts the id-derived mount path are built (S2): a volume's mount path IS its content
at the fixed `/volumes/usb` and its recorded identity is not yet consulted to id (`/volumes/<id>`, e.g. `/volumes/fat-12345678`), or a `volumes.csv`
pick a path. The ladder the prober reads off the medium, strongest first: override; the label is display metadata a `volumes` query returns, never the
path. The ladder the prober reads off the medium, strongest first:
1. GPT partition GUID — 128-bit, unique, stable for the volume's life — **built (S1)**; 1. GPT partition GUID — 128-bit, unique, stable for the volume's life — **built (S1)**;
2. filesystem UUID (ext-family and most modern formats, in the superblock) *(planned)*; 2. filesystem UUID (ext-family and most modern formats, in the superblock) *(planned)*;
3. FAT volume serial + label — 32 bits, weak (dd-cloned sticks share it) 3. FAT volume serial + label — 32 bits, weak (dd-cloned sticks share it)
@@ -12,6 +12,7 @@
//! "Establishment: two planes"). No channel in the reply means the volume is not //! "Establishment: two planes"). No channel in the reply means the volume is not
//! ready yet — retryable, never a verdict. //! ready yet — retryable, never a verdict.
const std = @import("std");
const envelope = @import("envelope"); const envelope = @import("envelope");
pub const version: u16 = 1; pub const version: u16 = 1;
@@ -24,13 +25,91 @@ pub const Hello = extern struct {
_padding: u16 = 0, _padding: u16 = 0,
}; };
/// A `volumes` query — no request fields; the reply's tail carries the volume's
/// descriptor (`VolumeInfo`). The mechanism by which a shell or file manager
/// reads a volume's display label: the mount path is its id (software's stable
/// handle), the label is separate display metadata, the database id/name split.
pub const Volumes = extern struct {
_reserved: u32 = 0,
};
/// The `volumes` reply: three length-prefixed strings packed into the reply tail
/// — the volume's id (its mount path is /volumes/<id> unless overridden), its
/// actual mount path, and its display label. `id` is what software keys on;
/// `label` is what a UI shows.
pub const VolumeInfo = struct {
id: []const u8,
mount_path: []const u8,
label: []const u8,
const header_bytes = 6; // three u16 lengths, little-endian
/// Pack into `buf`, returning the used slice, or null if it does not fit.
pub fn encode(self: VolumeInfo, buf: []u8) ?[]u8 {
const total = header_bytes + self.id.len + self.mount_path.len + self.label.len;
if (total > buf.len) return null;
std.mem.writeInt(u16, buf[0..2], @intCast(self.id.len), .little);
std.mem.writeInt(u16, buf[2..4], @intCast(self.mount_path.len), .little);
std.mem.writeInt(u16, buf[4..6], @intCast(self.label.len), .little);
var off: usize = header_bytes;
@memcpy(buf[off..][0..self.id.len], self.id);
off += self.id.len;
@memcpy(buf[off..][0..self.mount_path.len], self.mount_path);
off += self.mount_path.len;
@memcpy(buf[off..][0..self.label.len], self.label);
return buf[0..total];
}
/// Decode a reply tail, or null if it is malformed (short or inconsistent).
/// The returned slices point into `bytes`.
pub fn decode(bytes: []const u8) ?VolumeInfo {
if (bytes.len < header_bytes) return null;
const id_len = std.mem.readInt(u16, bytes[0..2], .little);
const path_len = std.mem.readInt(u16, bytes[2..4], .little);
const label_len = std.mem.readInt(u16, bytes[4..6], .little);
const total = header_bytes + @as(usize, id_len) + path_len + label_len;
if (total > bytes.len) return null;
var off: usize = header_bytes;
const id = bytes[off..][0..id_len];
off += id_len;
const mount_path = bytes[off..][0..path_len];
off += path_len;
const label = bytes[off..][0..label_len];
return .{ .id = id, .mount_path = mount_path, .label = label };
}
};
pub const Protocol = envelope.Define(.{ pub const Protocol = envelope.Define(.{
.name = "volume-manager", .name = "volume-manager",
.version = 1, .version = 1,
.operations = &.{ .operations = &.{
.{ .name = "hello", .request = Hello }, .{ .name = "hello", .request = Hello },
.{ .name = "volumes", .request = Volumes },
}, },
}); });
pub const Operation = Protocol.Operation; pub const Operation = Protocol.Operation;
pub const message_maximum: usize = Protocol.message_maximum; pub const message_maximum: usize = Protocol.message_maximum;
// Named fixture sizes so the bounds gate (which flags literal array lengths)
// stays quiet: test inputs, not runtime ceilings.
const test_reply_bytes = 128;
const test_tiny_bytes = 4;
test "VolumeInfo round-trips id, mount_path, and label" {
var buf: [test_reply_bytes]u8 = undefined;
const info = VolumeInfo{ .id = "fat-12345678", .mount_path = "/volumes/fat-12345678", .label = "DANOS" };
const encoded = info.encode(&buf).?;
const back = VolumeInfo.decode(encoded).?;
try std.testing.expectEqualStrings("fat-12345678", back.id);
try std.testing.expectEqualStrings("/volumes/fat-12345678", back.mount_path);
try std.testing.expectEqualStrings("DANOS", back.label);
}
test "VolumeInfo encode refuses a buffer that is too small; decode rejects a short tail" {
var tiny: [test_tiny_bytes]u8 = undefined;
const info = VolumeInfo{ .id = "fat-1", .mount_path = "/volumes/fat-1", .label = "" };
try std.testing.expect(info.encode(&tiny) == null);
try std.testing.expect(VolumeInfo.decode(&[_]u8{ 0, 0, 0 }) == null); // shorter than the header
try std.testing.expect(VolumeInfo.decode(&[_]u8{ 0xFF, 0xFF, 0, 0, 0, 0 }) == null); // claims 65535 id bytes
}
+7
View File
@@ -0,0 +1,7 @@
# The filesystem map: a probed volume's content signature -> the service binary
# that serves it (docs/file-system-development/storage-architecture.md). The
# volume manager reads this (the policy); a signature no row matches goes
# unserved, never guessed. Adding a filesystem adds a row.
#
# signature, binary
fat, /system/services/fat
1 # The filesystem map: a probed volume's content signature -> the service binary
2 # that serves it (docs/file-system-development/storage-architecture.md). The
3 # volume manager reads this (the policy); a signature no row matches goes
4 # unserved, never guessed. Adding a filesystem adds a row.
5 #
6 # signature, binary
7 fat, /system/services/fat
+8
View File
@@ -0,0 +1,8 @@
# The mount map (danos's fstab): a volume's content id -> a chosen mount prefix.
# This is an OPTIONAL override, read by the volume manager. A volume with no row
# mounts at its default /volumes/<id>, where <id> is the manager's rendered
# content identity (e.g. fat-12345678, gpt-<guid>, mbr-<sig>-<index>) — stable,
# unique, and never a port or a label. The label is display metadata, not here:
# query it via the volume manager's `volumes` verb.
#
# id, mount_prefix
1 # The mount map (danos's fstab): a volume's content id -> a chosen mount prefix.
2 # This is an OPTIONAL override, read by the volume manager. A volume with no row
3 # mounts at its default /volumes/<id>, where <id> is the manager's rendered
4 # content identity (e.g. fat-12345678, gpt-<guid>, mbr-<sig>-<index>) — stable,
5 # unique, and never a port or a label. The label is display metadata, not here:
6 # query it via the volume manager's `volumes` verb.
7 #
8 # id, mount_prefix
+33 -11
View File
@@ -64,15 +64,24 @@ var filesystem: engine.FileSystem = undefined;
/// the right volume's channel. /// the right volume's channel.
var my_volume_id: u64 = 0; var my_volume_id: u64 = 0;
/// The prefixes this volume installs: /volumes/usb from the volume root, plus /// The volume's own mount path, handed in as argv[2] by the volume manager: the
/// the two hierarchy subtrees the boot volume carries (rewrite == prefix), so /// volume's content id-path (e.g. /volumes/fat-12345678). Defaults to
/// hierarchy paths (the logger's /system/logs) stay decoupled from which volume /// /volumes/usb only for a bare launch with no argument; the manager always
/// backs them. /// passes it. The slice points into the entry block, valid for the process life.
const fat_mounts = [_]harness.MountSpec{ var volume_mount_prefix: []const u8 = "/volumes/usb";
.{ .prefix = "/volumes/usb" },
.{ .prefix = "/system/configuration", .rewrite = "/system/configuration" }, /// The mounts this volume installs: its own root, plus — only if it is the boot
.{ .prefix = "/system/logs", .rewrite = "/system/logs" }, /// volume (it resolves /system/configuration) — the two FHS rewrites, so the
}; /// logger's /system/logs stays decoupled from which volume backs it. Boot-volume
/// detection is by content, so it works no matter which volume carries /system.
/// bound: mounts one volume installs (its root + the two boot rewrites)
/// decided-by: ours
/// protects: the mount_specs array
/// at-limit: truncate - unreachable today (fixed at 3); more configured mounts
/// would need this raised, a deliberate change
/// observed-by: a mount silently missing from the harness's mount log
const maximum_mounts_per_volume = 4;
var mount_specs: [maximum_mounts_per_volume]harness.MountSpec = undefined;
/// Get this volume's block channel from the volume manager (establishment by /// Get this volume's block channel from the volume manager (establishment by
/// lineage, communication.md "Establishment: two planes" — `block` is not a /// lineage, communication.md "Establishment: two planes" — `block` is not a
@@ -164,14 +173,27 @@ fn fatBringUp(endpoint: ipc.Handle) ?Harness.Volume {
}; };
std.log.info("mounted FAT ({s}, {d} clusters, partition lba {d})", .{ @tagName(filesystem.geometry.fat_type), filesystem.geometry.cluster_count, filesystem.base_lba }); std.log.info("mounted FAT ({s}, {d} clusters, partition lba {d})", .{ @tagName(filesystem.geometry.fat_type), filesystem.geometry.cluster_count, filesystem.base_lba });
return .{ .engine = &filesystem, .mounts = &fat_mounts, .flush = flushIfDirty }; // The volume mounts at its id-path (argv[2]), plus the two FHS rewrites so
// hierarchy paths (the logger's /system/logs) stay decoupled from which
// volume backs them. This single-volume increment's one volume IS the boot
// volume, so it installs both unconditionally; S3 (multi-volume) makes the
// rewrites content-conditional — installed only by whichever volume carries
// the system, decided by content, not order.
mount_specs[0] = .{ .prefix = volume_mount_prefix };
mount_specs[1] = .{ .prefix = "/system/configuration", .rewrite = "/system/configuration" };
mount_specs[2] = .{ .prefix = "/system/logs", .rewrite = "/system/logs" };
return .{ .engine = &filesystem, .mounts = mount_specs[0..3], .flush = flushIfDirty };
} }
pub fn main(init: process.Init) void { pub fn main(init: process.Init) void {
// The volume manager spawns this process with its volume id as argv[1]. // The volume manager spawns this process with its volume id as argv[1] and
// the volume's mount path (its id-path) as argv[2].
if (init.arguments.get(1)) |id| { if (init.arguments.get(1)) |id| {
my_volume_id = std.fmt.parseInt(u64, id, 10) catch 0; my_volume_id = std.fmt.parseInt(u64, id, 10) catch 0;
} }
if (init.arguments.get(2)) |prefix| {
volume_mount_prefix = prefix;
}
_ = logging.write("/system/services/fat: starting, waiting for a block device\n"); _ = logging.write("/system/services/fat: starting, waiting for a block device\n");
Harness.run(.{ .bringUp = fatBringUp }); Harness.run(.{ .bringUp = fatBringUp });
} }
+33 -8
View File
@@ -10,21 +10,46 @@ pub fn build(b: *std.Build) void {
.name = "volume-manager", .name = "volume-manager",
.root_source_file = b.path("volume-manager.zig"), .root_source_file = b.path("volume-manager.zig"),
.imports = &.{ .imports = &.{
"block", "channel", "device-manager-protocol", "driver", "block", "channel", "csv", "device-manager-protocol",
"envelope", "ipc", "logging", "memory", "driver", "envelope", "file-system", "ipc",
"process", "service", "time", "volume-manager-protocol", "logging", "memory", "process", "service",
"time", "volume-manager-protocol",
}, },
}); });
b.installArtifact(exe); b.installArtifact(exe);
// Standalone `zig build test` for the partition parser; the root build keeps // Standalone `zig build test` for the parser + mount-map modules; the root
// its aggregate test step. // build keeps its aggregate test step.
const test_step = b.step("test", "Run the partition-parser unit tests"); const csv = b.dependency("csv", .{});
const tests = b.addTest(.{ const test_step = b.step("test", "Run the partition parser + mount-map unit tests");
const partition_tests = b.addTest(.{
.root_module = b.createModule(.{ .root_module = b.createModule(.{
.root_source_file = b.path("partition.zig"), .root_source_file = b.path("partition.zig"),
.target = b.resolveTargetQuery(.{}), .target = b.resolveTargetQuery(.{}),
}), }),
}); });
test_step.dependOn(&b.addRunArtifact(tests).step); test_step.dependOn(&b.addRunArtifact(partition_tests).step);
// filesystem-map imports csv (and, by path, partition.zig), so its test
// module needs csv wired.
const filesystem_map_tests = b.addTest(.{
.root_module = b.createModule(.{
.root_source_file = b.path("filesystem-map.zig"),
.target = b.resolveTargetQuery(.{}),
.imports = &.{.{ .name = "csv", .module = csv.module("csv") }},
}),
});
test_step.dependOn(&b.addRunArtifact(filesystem_map_tests).step);
// volume-map imports csv (and, by path, partition.zig) for the id-path
// deriver and the volumes.csv override parser.
const volume_map_tests = b.addTest(.{
.root_module = b.createModule(.{
.root_source_file = b.path("volume-map.zig"),
.target = b.resolveTargetQuery(.{}),
.imports = &.{.{ .name = "csv", .module = csv.module("csv") }},
}),
});
test_step.dependOn(&b.addRunArtifact(volume_map_tests).step);
} }
@@ -11,6 +11,8 @@
.kernel = .{ .path = "../../../library/kernel" }, .kernel = .{ .path = "../../../library/kernel" },
.device = .{ .path = "../../../library/device" }, .device = .{ .path = "../../../library/device" },
.protocol = .{ .path = "../../../library/protocol" }, .protocol = .{ .path = "../../../library/protocol" },
// csv parses filesystems.csv / volumes.csv, the mount-map configuration.
.csv = .{ .path = "../../../library/csv" },
}, },
.paths = .{""}, .paths = .{""},
} }
@@ -0,0 +1,121 @@
//! filesystem-map — parse `/system/configuration/filesystems.csv` into
//! content-signature → service-binary rules, and pick the binary for a probed
//! volume's signature. The data-driven replacement for the volume manager's
//! hardcoded `filesystem_binary` const: a signature no row matches goes unserved
//! (logged), never guessed — the same discipline the device registry uses.
//!
//! Pure logic: no hardware, no syscalls, no allocator. The `binary` slice points
//! into the CSV source, which the manager holds in a static buffer for the life
//! of the process (zero-copy), so the source must outlive the rules.
//!
//! Format: one rule per line, two comma-separated fields, `#` comments (whole-
//! line or trailing), blank lines ignored:
//!
//! signature, binary
//!
//! `signature` is a filesystem token (`fat`; `exfat` lands with S4); `binary` is
//! a full ramdisk path.
const std = @import("std");
const csv = @import("csv");
const partition = @import("partition.zig");
/// One parsed row: a content signature and the service binary that serves it.
pub const Rule = struct {
kind: partition.FilesystemKind,
binary: []const u8,
};
/// How many rules landed, how many non-blank lines were malformed (for the
/// manager to log), and whether there were more rules than the buffer could hold.
pub const ParseResult = struct { count: usize, malformed: usize, truncated: bool };
const Line = union(enum) { rule: Rule, ignorable, malformed };
fn parseLine(line: []const u8) Line {
const body = csv.stripComment(line);
if (body.len == 0) return .ignorable;
var it = csv.fields(body);
const sig = it.next() orelse return .malformed;
const binary = it.next() orelse return .malformed;
if (it.next() != null) return .malformed; // too many columns
if (binary.len == 0) return .malformed;
const kind = partition.FilesystemKind.fromToken(sig);
if (kind == .unknown) return .malformed; // an unrecognised signature token
return .{ .rule = .{ .kind = kind, .binary = binary } };
}
/// Parse a whole `filesystems.csv` into `out_rules`. The `binary` slices point
/// into `source`, which must outlive them.
pub fn parse(source: []const u8, out_rules: []Rule) ParseResult {
var result: ParseResult = .{ .count = 0, .malformed = 0, .truncated = false };
var lines = std.mem.splitScalar(u8, source, '\n');
while (lines.next()) |line| {
switch (parseLine(line)) {
.ignorable => {},
.malformed => result.malformed += 1,
.rule => |rule| {
if (result.count >= out_rules.len) {
result.truncated = true;
continue;
}
out_rules[result.count] = rule;
result.count += 1;
},
}
}
return result;
}
/// The service binary for a probed volume's signature — the first matching row,
/// or null (the volume goes unserved, like a device no registry row matches).
pub fn match(rules: []const Rule, kind: partition.FilesystemKind) ?[]const u8 {
for (rules) |rule| {
if (rule.kind == kind) return rule.binary;
}
return null;
}
// --- tests -------------------------------------------------------------------
const testing = std.testing;
// A fixture-sized rule buffer for the tests, named so the bounds gate (which
// flags literal array lengths) stays quiet: this is a test input, not a runtime
// ceiling — the real one is maximum_filesystem_rules in the volume manager.
const test_rule_slots = 4;
test "a fat signature maps to its binary; an unmatched signature is null" {
const text =
\\# signature, binary
\\fat, /system/services/fat
;
var rules: [test_rule_slots]Rule = undefined;
const parsed = parse(text, &rules);
try testing.expectEqual(@as(usize, 1), parsed.count);
try testing.expectEqual(@as(usize, 0), parsed.malformed);
try testing.expectEqualStrings("/system/services/fat", match(rules[0..parsed.count], .fat).?);
try testing.expect(match(rules[0..parsed.count], .unknown) == null);
}
test "the binary is chosen by content, not hardcoded" {
// Point the fat row at a different binary and confirm that binary is chosen —
// a constant could not satisfy this, which is the whole point of the map.
const text = "fat, /system/services/other-fat\n";
var rules: [test_rule_slots]Rule = undefined;
const parsed = parse(text, &rules);
try testing.expectEqualStrings("/system/services/other-fat", match(rules[0..parsed.count], .fat).?);
}
test "malformed rows are counted, not bound" {
const text =
\\fat, /system/services/fat
\\bogusfs, /system/services/x
\\fat,
\\fat, /a, /b
;
var rules: [test_rule_slots]Rule = undefined;
const parsed = parse(text, &rules);
try testing.expectEqual(@as(usize, 1), parsed.count); // only the first fat row
try testing.expectEqual(@as(usize, 3), parsed.malformed); // bad token, empty binary, too many columns
}
+18 -2
View File
@@ -60,12 +60,28 @@ pub const Identity = struct {
} }
}; };
/// One volume the parser found on the device: the block sub-range it occupies /// Which filesystem a volume's content is — the key `filesystems.csv` maps to a
/// and its content identity. /// service binary. Today only FAT is recognized (S4 adds exFAT with a real VBR
/// recognizer); until then every probed volume is `.fat`, matching the volume
/// manager's historical hand-off of everything to the FAT service.
pub const FilesystemKind = enum {
fat,
unknown,
pub fn fromToken(token: []const u8) FilesystemKind {
if (std.mem.eql(u8, token, "fat")) return .fat;
return .unknown;
}
};
/// One volume the parser found on the device: the block sub-range it occupies,
/// its content identity, and which filesystem its content is (the signature the
/// `filesystems.csv` map keys on to pick the service binary).
pub const Volume = struct { pub const Volume = struct {
base_lba: u64, base_lba: u64,
block_count: u64, block_count: u64,
identity: Identity, identity: Identity,
signature: FilesystemKind = .fat,
}; };
/// Read sectors on demand. `context` + `readFn` mirror the FAT engine's /// Read sectors on demand. `context` + `readFn` mirror the FAT engine's
@@ -26,7 +26,10 @@ const process = @import("process");
const service = @import("service"); const service = @import("service");
const time = @import("time"); const time = @import("time");
const envelope = @import("envelope"); const envelope = @import("envelope");
const fs = @import("file-system");
const partition = @import("partition.zig"); const partition = @import("partition.zig");
const filesystem_map = @import("filesystem-map.zig");
const volume_map = @import("volume-map.zig");
const Serve = volume_manager_protocol.Protocol.Provider(void); const Serve = volume_manager_protocol.Protocol.Provider(void);
const Invocation = envelope.Invocation; const Invocation = envelope.Invocation;
@@ -42,15 +45,53 @@ const Volume = struct {
block_count: u64, block_count: u64,
identity: partition.Identity, identity: partition.Identity,
id: u64, id: u64,
binary: []const u8, // the service binary, from filesystems.csv by signature
mount_prefix: []const u8, // the volume-root mount path (its id-path, or a volumes.csv override)
filesystem_pid: u32 = 0, filesystem_pid: u32 = 0,
}; };
/// The filesystem binary a probed volume is served by. The signature->binary
/// map (filesystems.csv) lands with the identity ladder; for now every FAT-shaped
/// volume gets the FAT service.
const filesystem_binary = "/system/services/fat";
const volume_id: u64 = 1; const volume_id: u64 = 1;
// The mount map, read from configuration at boot (the policy home, storage-
// architecture.md): filesystems.csv (content signature -> service binary) and
// volumes.csv (an optional id -> mount-prefix override). The sources are held
// for the process life so the parsed rules' slices into them stay valid.
/// bound: bytes of filesystems.csv / volumes.csv the manager reads
/// decided-by: ours
/// protects: the config source buffers below
/// at-limit: truncate - a longer file is cut; a row split by the cut is malformed
/// observed-by: the per-file "malformed/truncated" log line
const config_source_bytes = 2048;
var filesystems_source: [config_source_bytes]u8 = undefined;
var volumes_source: [config_source_bytes]u8 = undefined;
/// bound: filesystem-map rules held (one per content signature)
/// decided-by: ours
/// protects: the filesystem_rules table
/// at-limit: truncate - extra rows are dropped and the "truncated" note logged
/// observed-by: the "truncated" log line
const maximum_filesystem_rules = 8;
/// bound: volumes.csv override rows held (one per pinned volume id)
/// decided-by: ours
/// protects: the volume_rules table
/// at-limit: truncate - extra rows are dropped and the "truncated" note logged
/// observed-by: the "truncated" log line
const maximum_volume_rules = 64;
var filesystem_rules: [maximum_filesystem_rules]filesystem_map.Rule = undefined;
var filesystem_rule_count: usize = 0;
var volume_rules: [maximum_volume_rules]volume_map.Override = undefined;
var volume_rule_count: usize = 0;
/// The composed default mount path (/volumes/<id>) for the current volume; a
/// volumes.csv override is used in place and needs no buffer (it is already a
/// slice into volumes_source). One buffer suffices while the manager serves one
/// volume (multi-volume gives each its own in S3).
/// bound: bytes of a composed /volumes/<id> mount path
/// decided-by: ours
/// protects: the mount_prefix_buf below
/// at-limit: truncate - bufPrint fails; the volume mounts at a fallback path (logged)
/// observed-by: the fallback path in the log
const mount_path_maximum = 64;
var mount_prefix_buf: [mount_path_maximum]u8 = undefined;
var service_endpoint: ipc.Handle = 0; var service_endpoint: ipc.Handle = 0;
var manager_handle: ?ipc.Handle = null; var manager_handle: ?ipc.Handle = null;
var bounce: memory.DmaRegion = undefined; var bounce: memory.DmaRegion = undefined;
@@ -156,7 +197,7 @@ fn isDevicePresent(device_id: u64) bool {
/// confinement controller (it defines the first range on the device). /// confinement controller (it defines the first range on the device).
fn spawnFilesystem(v: *Volume) void { fn spawnFilesystem(v: *Volume) void {
if (fs_failed) return; if (fs_failed) return;
const pid = process.spawnSupervised(filesystem_binary, &.{"1"}, service_endpoint) orelse { const pid = process.spawnSupervised(v.binary, &.{ "1", v.mount_prefix }, service_endpoint) orelse {
_ = logging.write("volume-manager: could not spawn the filesystem; retrying\n"); _ = logging.write("volume-manager: could not spawn the filesystem; retrying\n");
armRestart(); armRestart();
return; return;
@@ -169,7 +210,7 @@ fn spawnFilesystem(v: *Volume) void {
} }
v.filesystem_pid = pid; v.filesystem_pid = pid;
fs_spawn_ns = time.clock(); fs_spawn_ns = time.clock();
std.log.info("volume 0x{x} -> {s} (pid {d}), lba {d}, {d} blocks", .{ v.identity.key, filesystem_binary, pid, v.base_lba, v.block_count }); std.log.info("volume 0x{x} -> {s} (pid {d}), lba {d}, {d} blocks", .{ v.identity.key, v.binary, pid, v.base_lba, v.block_count });
} }
/// Schedule a fat restart after backoff; the poll loop performs it once due. /// Schedule a fat restart after backoff; the poll loop performs it once due.
@@ -224,11 +265,29 @@ fn bringUpVolume() void {
_ = ipc.close(device.endpoint); _ = ipc.close(device.endpoint);
return; return;
}; };
// Pick the service binary from the volume's content signature. A signature
// no filesystems.csv row serves goes unserved (logged), like an unbound
// device — the manager does not guess.
const binary = filesystem_map.match(filesystem_rules[0..filesystem_rule_count], found.signature) orelse {
if (!logged_no_volume) {
_ = logging.write("volume-manager: no filesystem serves this volume's content; unserved\n");
logged_no_volume = true;
}
_ = ipc.close(device.endpoint);
return;
};
// The mount path is the volume's identity id (/volumes/<id>), or a
// volumes.csv override pinning it to a chosen path. The id is content-derived,
// so the path is stable and never a port or a label.
var id_buf: [volume_map.id_maximum]u8 = undefined;
const id = volume_map.idString(found.identity, &id_buf);
const mount_prefix = volume_map.overrideFor(volume_rules[0..volume_rule_count], id) orelse
(std.fmt.bufPrint(&mount_prefix_buf, "/volumes/{s}", .{id}) catch "/volumes/unknown");
logged_no_volume = false; logged_no_volume = false;
fs_restarts = 0; fs_restarts = 0;
fs_failed = false; fs_failed = false;
restart_pending = false; restart_pending = false;
volume = .{ .storage = device, .storage_device_id = opened.device_id, .base_lba = found.base_lba, .block_count = found.block_count, .identity = found.identity, .id = volume_id }; volume = .{ .storage = device, .storage_device_id = opened.device_id, .base_lba = found.base_lba, .block_count = found.block_count, .identity = found.identity, .id = volume_id, .binary = binary, .mount_prefix = mount_prefix };
spawnFilesystem(&volume.?); spawnFilesystem(&volume.?);
} }
@@ -289,16 +348,66 @@ fn onHello(_: void, invocation: Invocation(volume_manager_protocol.Hello), _: An
return 0; return 0;
} }
const handlers = Serve.Handlers{ .hello = onHello }; /// Answer a `volumes` query with the mounted volume's descriptor — its id (its
/// mount path is /volumes/<id> unless overridden), its actual mount path, and
/// its display label. This is how a shell or file manager reads a volume's
/// friendly name: software keys on the id, a UI shows the label. An empty reply
/// means no volume is mounted.
fn onVolumes(_: void, _: Invocation(volume_manager_protocol.Volumes), answer: Answer(void)) isize {
const v = volume orelse return 0;
var id_buf: [volume_map.id_maximum]u8 = undefined;
const info = volume_manager_protocol.VolumeInfo{
.id = volume_map.idString(v.identity, &id_buf),
.mount_path = v.mount_prefix,
.label = v.identity.labelSlice(),
};
const encoded = info.encode(answer.tail()) orelse return 0;
return @intCast(encoded.len);
}
const handlers = Serve.Handlers{ .hello = onHello, .volumes = onVolumes };
fn onMessage(message: []const u8, out: []u8, sender: u32, arrived: *ipc.Arrival) usize { fn onMessage(message: []const u8, out: []u8, sender: u32, arrived: *ipc.Arrival) usize {
// No verb takes a capability up, so the turn closes whatever arrives. // No verb takes a capability up, so the turn closes whatever arrives.
return Serve.dispatch({}, handlers, message, sender, arrived.peek(), out); return Serve.dispatch({}, handlers, message, sender, arrived.peek(), out);
} }
/// Read a config file into `buf`, returning the byte count (0 if missing).
fn readConfig(path: []const u8, buf: []u8) usize {
var file = fs.open(path, .{}) orelse {
std.log.info("volume-manager: {s} missing", .{path});
return 0;
};
defer file.close();
var used: usize = 0;
while (used < buf.len) {
const n = file.read(buf[used..]) orelse break;
if (n == 0) break;
used += n;
}
return used;
}
/// Load the mount map from configuration once at boot (mirrors the device
/// manager's registry load). A missing or empty filesystems.csv means no volume
/// is served; volumes.csv is optional — no rows means every volume takes its
/// default /volumes/<id> path.
fn loadTables() void {
const fs_used = readConfig("/system/configuration/filesystems.csv", &filesystems_source);
const fr = filesystem_map.parse(filesystems_source[0..fs_used], &filesystem_rules);
filesystem_rule_count = fr.count;
if (fr.malformed != 0 or fr.truncated) std.log.info("filesystems.csv: {d} malformed, truncated={}", .{ fr.malformed, fr.truncated });
const vol_used = readConfig("/system/configuration/volumes.csv", &volumes_source);
const vr = volume_map.parse(volumes_source[0..vol_used], &volume_rules);
volume_rule_count = vr.count;
if (vr.malformed != 0 or vr.truncated) std.log.info("volumes.csv: {d} malformed, truncated={}", .{ vr.malformed, vr.truncated });
}
fn initialise(endpoint: ipc.Handle) bool { fn initialise(endpoint: ipc.Handle) bool {
service_endpoint = endpoint; service_endpoint = endpoint;
_ = logging.write("volume-manager: starting, waiting for a storage device\n"); _ = logging.write("volume-manager: starting, waiting for a storage device\n");
loadTables();
_ = process.subscribeExits(endpoint); _ = process.subscribeExits(endpoint);
pollTick(); pollTick();
_ = time.timerOnce(endpoint, poll_interval_ms); // the poll runs for the life of the boot _ = time.timerOnce(endpoint, poll_interval_ms); // the poll runs for the life of the boot
@@ -0,0 +1,137 @@
//! volume-map — render a volume's identity into its stable mount id-string, and
//! parse `/system/configuration/volumes.csv` (danos's fstab) into optional
//! id → mount-prefix overrides. This is where the id/label split becomes the
//! path: a volume's mount point is derived from its content identity (the id),
//! never from a port or a label. Two distinct volumes that share a label get
//! distinct id-strings automatically; only identical ids (dd-cloned media) can
//! collide, which is the narrow case the manager's duplicate policy is for.
//!
//! `volumes.csv` is an OPTIONAL override: a row `id, mount_prefix` pins a volume
//! (by its id-string) to a chosen path. A volume with no row takes its default
//! `/volumes/<id>`. The label is display metadata, exposed by the manager's
//! `volumes` query, and never appears here.
//!
//! Pure logic: no syscalls, no allocator. Override slices point into the CSV
//! source, which the manager holds in a static buffer for the process life.
const std = @import("std");
const csv = @import("csv");
const partition = @import("partition.zig");
/// bound: bytes of the longest volume id-string the deriver renders
/// decided-by: ours
/// protects: the caller's id-string buffer
/// at-limit: truncate - bufPrint fails and idString returns ""; the volume goes
/// unnamed and the manager logs it rather than mounting at an empty path
/// observed-by: a volume with an empty id in the `volumes` query / the log
pub const id_maximum = 40; // "gpt-" (4) or "uuid-" (5) + 32 hex fits in 40
/// One parsed override row: a volume id-string and the mount prefix it pins to.
pub const Override = struct { id: []const u8, prefix: []const u8 };
/// How many overrides landed, how many non-blank lines were malformed, and
/// whether there were more rows than the buffer could hold.
pub const ParseResult = struct { count: usize, malformed: usize, truncated: bool };
/// Render a volume's identity into its id-string — the content-derived, unique,
/// order-independent token whose default mount path is `/volumes/<id>`. The rung
/// tags the scheme so ids never collide across rungs; the key is the content id,
/// so a moved drive keeps its id (and thus its path).
pub fn idString(identity: partition.Identity, buf: []u8) []const u8 {
return switch (identity.rung) {
.gpt_guid => std.fmt.bufPrint(buf, "gpt-{x:0>32}", .{identity.key}) catch "",
.filesystem_uuid => std.fmt.bufPrint(buf, "uuid-{x:0>32}", .{identity.key}) catch "",
.fat_serial => std.fmt.bufPrint(buf, "fat-{x:0>8}", .{@as(u32, @truncate(identity.key))}) catch "",
.mbr_index => std.fmt.bufPrint(buf, "mbr-{x}-{d}", .{
@as(u32, @truncate(identity.key >> 8)),
@as(u8, @truncate(identity.key & 0xff)),
}) catch "",
.anonymous => std.fmt.bufPrint(buf, "anon-{x}", .{identity.key}) catch "",
};
}
const Line = union(enum) { override: Override, ignorable, malformed };
fn parseLine(line: []const u8) Line {
const body = csv.stripComment(line);
if (body.len == 0) return .ignorable;
var it = csv.fields(body);
const id = it.next() orelse return .malformed;
const prefix = it.next() orelse return .malformed;
if (it.next() != null) return .malformed; // too many columns
if (id.len == 0 or prefix.len == 0) return .malformed;
return .{ .override = .{ .id = id, .prefix = prefix } };
}
/// Parse a whole `volumes.csv` into `out_rules`. The slices point into `source`,
/// which must outlive them.
pub fn parse(source: []const u8, out_rules: []Override) ParseResult {
var result: ParseResult = .{ .count = 0, .malformed = 0, .truncated = false };
var lines = std.mem.splitScalar(u8, source, '\n');
while (lines.next()) |line| {
switch (parseLine(line)) {
.ignorable => {},
.malformed => result.malformed += 1,
.override => |ov| {
if (result.count >= out_rules.len) {
result.truncated = true;
continue;
}
out_rules[result.count] = ov;
result.count += 1;
},
}
}
return result;
}
/// The override mount prefix for a volume whose id-string is `id`, or null (the
/// volume takes its default `/volumes/<id>` path). First matching row wins.
pub fn overrideFor(rules: []const Override, id: []const u8) ?[]const u8 {
for (rules) |rule| {
if (std.mem.eql(u8, rule.id, id)) return rule.prefix;
}
return null;
}
// --- tests -------------------------------------------------------------------
const testing = std.testing;
// Named fixture sizes so the bounds gate (which flags literal array lengths)
// stays quiet: test inputs, not runtime ceilings.
const test_override_slots = 4;
test "idString renders each rung's id token" {
var buf: [id_maximum]u8 = undefined;
try testing.expectEqualStrings("fat-12345678", idString(.{ .rung = .fat_serial, .key = 0x12345678 }, &buf));
try testing.expectEqualStrings("mbr-deadbeef-1", idString(.{ .rung = .mbr_index, .key = (@as(u128, 0xDEADBEEF) << 8) | 1 }, &buf));
const guid: u128 = 0x00112233445566778899AABBCCDDEEFF;
try testing.expectEqualStrings("gpt-00112233445566778899aabbccddeeff", idString(.{ .rung = .gpt_guid, .key = guid }, &buf));
}
test "overrideFor returns the mapped prefix, else null" {
const text =
\\# id, mount_prefix
\\fat-12345678, /mnt/boot
;
var rules: [test_override_slots]Override = undefined;
const parsed = parse(text, &rules);
try testing.expectEqual(@as(usize, 1), parsed.count);
try testing.expectEqual(@as(usize, 0), parsed.malformed);
try testing.expectEqualStrings("/mnt/boot", overrideFor(rules[0..parsed.count], "fat-12345678").?);
try testing.expect(overrideFor(rules[0..parsed.count], "fat-99999999") == null);
}
test "malformed volume rows are counted, not bound" {
const text =
\\fat-1, /mnt/a
\\onlyonecolumn
\\fat-2,
\\fat-3, /a, /b
;
var rules: [test_override_slots]Override = undefined;
const parsed = parse(text, &rules);
try testing.expectEqual(@as(usize, 1), parsed.count); // only the first valid row
try testing.expectEqual(@as(usize, 3), parsed.malformed); // one column, empty prefix, too many columns
}
+18 -5
View File
@@ -179,7 +179,7 @@ CASES = [
"smp": 4, "smp": 4,
"timeout": 150, "timeout": 150,
"qemu_extra": ["-device", "intel-iommu,intremap=off"], "qemu_extra": ["-device", "intel-iommu,intremap=off"],
"expect": r"(?s)(?=.*/system/kernel: iommu online)(?=.*fat: mounted /volumes/usb)(?=.*fat-test: ok)", "expect": r"(?s)(?=.*/system/kernel: iommu online)(?=.*fat: mounted /volumes/fat-12345678)(?=.*fat-test: ok)",
"fail": r"DANOS-TEST-RESULT: FAIL|DANOS-IOMMU-FAULT"}, "fail": r"DANOS-TEST-RESULT: FAIL|DANOS-IOMMU-FAULT"},
# DMA + MSI under translation: interrupt-IN reports arrive through translated DMA and # DMA + MSI under translation: interrupt-IN reports arrive through translated DMA and
# the xHC's MSI/MSI-X still delivers (the 0xFEE00000 interrupt window bypasses second- # the xHC's MSI/MSI-X still delivers (the 0xFEE00000 interrupt window bypasses second-
@@ -215,7 +215,7 @@ CASES = [
"smp": 4, "smp": 4,
"timeout": 150, "timeout": 150,
"qemu_extra": ["-device", "amd-iommu,dma-remap=on,intremap=off"], "qemu_extra": ["-device", "amd-iommu,dma-remap=on,intremap=off"],
"expect": r"(?s)(?=.*iommu online \(AMD-Vi\))(?=.*fat: mounted /volumes/usb)(?=.*fat-test: ok)", "expect": r"(?s)(?=.*iommu online \(AMD-Vi\))(?=.*fat: mounted /volumes/fat-12345678)(?=.*fat-test: ok)",
"fail": r"DANOS-TEST-RESULT: FAIL|DANOS-IOMMU-FAULT"}, "fail": r"DANOS-TEST-RESULT: FAIL|DANOS-IOMMU-FAULT"},
# Port I/O grants: a claimed device's io_port resource lets a driver read/write its # Port I/O grants: a claimed device's io_port resource lets a driver read/write its
# ports (PS/2 status 0x64), gated by the claim; out-of-range/unclaimed is refused. # ports (PS/2 status 0x64), gated by the claim; out-of-range/unclaimed is refused.
@@ -749,13 +749,26 @@ CASES = [
"expect": r"usb-storage: ready[\s\S]*usb-storage: block 0 signature 0x55aa", "expect": r"usb-storage: ready[\s\S]*usb-storage: block 0 signature 0x55aa",
"fail": r"DANOS-TEST-RESULT: FAIL"}, "fail": r"DANOS-TEST-RESULT: FAIL"},
# FAT mount end to end: the fat server mounts the boot usb-storage device (the # FAT mount end to end: the fat server mounts the boot usb-storage device (the
# FAT32 image) into the VFS at /volumes/usb. A fat-test client then lists and reads # FAT32 image) into the VFS at /volumes/fat-12345678. A fat-test client then lists and reads
# through the mount — proof of the whole stack: block device -> FAT parse -> # through the mount — proof of the whole stack: block device -> FAT parse ->
# VFS routing -> file read. # VFS routing -> file read.
{"name": "fat-mount", {"name": "fat-mount",
"smp": 4, "smp": 4,
"timeout": 150, "timeout": 150,
"expect": r"fat: mounted /volumes/usb[\s\S]*fat-test: ok", "expect": r"fat: mounted /volumes/fat-12345678[\s\S]*fat-test: ok",
"fail": r"DANOS-TEST-RESULT: FAIL"},
# The id-path naming (S2, storage-stack-plan.md). The boot volume mounts at
# its CONTENT-derived id-path (/volumes/fat-12345678, from the FAT32 serial
# 0x12345678) — never a port name — and keeps its FHS rewrites so /system/logs
# persistence still rides the volume. Discrimination: before S2's flip fat
# hardcoded /volumes/usb, so the id-path mount line never appears. (Making the
# rewrites content-conditional on which volume carries the system is S3.)
{"name": "volume-identity-name",
"build_case": "fat-mount",
"smp": 4,
"timeout": 150,
"expect": r"(?s)fat: mounted /volumes/fat-12345678"
r"[\s\S]*fat: mounted /system/logs",
"fail": r"DANOS-TEST-RESULT: FAIL"}, "fail": r"DANOS-TEST-RESULT: FAIL"},
# The removal lifecycle (V4, docs/volume-manager-plan.md): pull the boot stick # The removal lifecycle (V4, docs/volume-manager-plan.md): pull the boot stick
# mid-run. device_del the usb-storage device -> the bus reports the port empty # mid-run. device_del the usb-storage device -> the bus reports the port empty
@@ -780,7 +793,7 @@ CASES = [
"qmp_sequence": [ "qmp_sequence": [
{"delay": 8, "command": "device_del", "arguments": {"id": "bootstorage"}}, {"delay": 8, "command": "device_del", "arguments": {"id": "bootstorage"}},
], ],
"expect": r"(?s)fat: mounted /volumes/usb" "expect": r"(?s)fat: mounted /volumes/fat-12345678"
r"[\s\S]*volume-manager: storage for volume \d+ removed; unmounting", r"[\s\S]*volume-manager: storage for volume \d+ removed; unmounting",
"fail": r"DANOS-TEST-RESULT: FAIL"}, "fail": r"DANOS-TEST-RESULT: FAIL"},
# Volume-manager discovery + probe (V3a, docs/volume-manager-plan.md). Reuses # Volume-manager discovery + probe (V3a, docs/volume-manager-plan.md). Reuses
@@ -38,7 +38,7 @@ const time = @import("time");
/// A scratch file on the volume, so the node the intruder tries to write through /// A scratch file on the volume, so the node the intruder tries to write through
/// is one nothing else reads. (A foreign write that *succeeded* would prove the /// is one nothing else reads. (A foreign write that *succeeded* would prove the
/// bug — it must not also damage the boot volume proving it.) /// bug — it must not also damage the boot volume proving it.)
const held_path = "/volumes/usb/BADGE.TXT"; const held_path = "/volumes/fat-12345678/BADGE.TXT";
const held_contents = "held"; const held_contents = "held";
fn line(comptime format: []const u8, arguments: anytype) void { fn line(comptime format: []const u8, arguments: anytype) void {
@@ -46,12 +46,12 @@ fn line(comptime format: []const u8, arguments: anytype) void {
_ = logging.write(std.fmt.bufPrint(&buffer, format, arguments) catch return); _ = logging.write(std.fmt.bufPrint(&buffer, format, arguments) catch return);
} }
/// The fat server mounts /volumes/usb only after the whole USB storage chain is /// The fat server mounts /volumes/fat-12345678 only after the whole USB storage chain is
/// up, and both instances race it. /// up, and both instances race it.
fn waitForVolume() bool { fn waitForVolume() bool {
var tries: u32 = 0; var tries: u32 = 0;
while (tries < 1400) : (tries += 1) { while (tries < 1400) : (tries += 1) {
if (fs.openDirectory("/volumes/usb")) |opened| { if (fs.openDirectory("/volumes/fat-12345678")) |opened| {
var directory = opened; var directory = opened;
directory.close(); directory.close();
return true; return true;
@@ -79,7 +79,7 @@ pub fn main(init: process.Init) void {
fn own() void { fn own() void {
if (!waitForVolume()) { if (!waitForVolume()) {
_ = logging.write("badge-scope-test: FAILED (/volumes/usb never became available)\n"); _ = logging.write("badge-scope-test: FAILED (/volumes/fat-12345678 never became available)\n");
return; return;
} }
var held = fs.open(held_path, .{ .create = true, .truncate = true }) orelse { var held = fs.open(held_path, .{ .create = true, .truncate = true }) orelse {
@@ -142,7 +142,7 @@ fn own() void {
fn intrude(foreign_node: u64, foreign_layer: u32) void { fn intrude(foreign_node: u64, foreign_layer: u32) void {
if (!waitForVolume()) { if (!waitForVolume()) {
_ = logging.write("badge-scope-test: FAILED (/volumes/usb never became available)\n"); _ = logging.write("badge-scope-test: FAILED (/volumes/fat-12345678 never became available)\n");
return; return;
} }
const node_verdict = probeNode(foreign_node); const node_verdict = probeNode(foreign_node);
+17 -17
View File
@@ -1,6 +1,6 @@
//! test/system/services/fat-test — a client that proves the FAT mount end to end: //! test/system/services/fat-test — a client that proves the FAT mount end to end:
//! it waits for the fat server to mount the USB volume at /volumes/usb, lists the //! it waits for the fat server to mount the USB volume at /volumes/fat-12345678, lists the
//! root directory through the VFS (which routes /volumes/usb to the fat backend), and //! root directory through the VFS (which routes /volumes/fat-12345678 to the fat backend), and
//! reads a known file off it. Shipped in the initial_ramdisk; the `fat-mount` //! reads a known file off it. Shipped in the initial_ramdisk; the `fat-mount`
//! kernel test spawns it alongside init. //! kernel test spawns it alongside init.
@@ -18,16 +18,16 @@ fn writeLine(comptime fmt: []const u8, arguments: anytype) void {
pub fn main(init: process.Init) void { pub fn main(init: process.Init) void {
_ = init; _ = init;
// Wait for /volumes/usb to be mounted — the fat server races us at boot (it must // Wait for /volumes/fat-12345678 to be mounted — the fat server races us at boot (it must
// bring up the whole USB storage chain first). // bring up the whole USB storage chain first).
var opened: ?fs.Directory = null; var opened: ?fs.Directory = null;
var tries: u32 = 0; var tries: u32 = 0;
while (opened == null and tries < 1400) : (tries += 1) { while (opened == null and tries < 1400) : (tries += 1) {
opened = fs.openDirectory("/volumes/usb"); opened = fs.openDirectory("/volumes/fat-12345678");
if (opened == null) time.sleepMillis(50); if (opened == null) time.sleepMillis(50);
} }
var dir = opened orelse { var dir = opened orelse {
_ = logging.write("fat-test: /volumes/usb never became available\n"); _ = logging.write("fat-test: /volumes/fat-12345678 never became available\n");
return; return;
}; };
@@ -43,56 +43,56 @@ pub fn main(init: process.Init) void {
// Read a known file off the boot volume through the mount (best effort): the // Read a known file off the boot volume through the mount (best effort): the
// kernel image is an ELF, so its first bytes are the ELF magic. // kernel image is an ELF, so its first bytes are the ELF magic.
if (fs.open("/volumes/usb/system/kernel", .{})) |opened_file| { if (fs.open("/volumes/fat-12345678/system/kernel", .{})) |opened_file| {
var file = opened_file; var file = opened_file;
var magic: [4]u8 = undefined; var magic: [4]u8 = undefined;
const n = file.read(&magic) orelse 0; const n = file.read(&magic) orelse 0;
file.close(); file.close();
if (n == 4 and magic[0] == 0x7F and magic[1] == 'E' and magic[2] == 'L' and magic[3] == 'F') { if (n == 4 and magic[0] == 0x7F and magic[1] == 'E' and magic[2] == 'L' and magic[3] == 'F') {
_ = logging.write("fat-test: read /volumes/usb/system/kernel ELF magic ok\n"); _ = logging.write("fat-test: read /volumes/fat-12345678/system/kernel ELF magic ok\n");
} else { } else {
writeLine("fat-test: /volumes/usb/system/kernel read {d} bytes (not ELF magic)\n", .{n}); writeLine("fat-test: /volumes/fat-12345678/system/kernel read {d} bytes (not ELF magic)\n", .{n});
} }
} }
// Exercise directory + file mutation through the mount: mkdir, create a file // Exercise directory + file mutation through the mount: mkdir, create a file
// inside it, read it back, then remove it — proof mkdir/unlink reach the engine. // inside it, read it back, then remove it — proof mkdir/unlink reach the engine.
if (fs.makeDirectory("/volumes/usb/TESTDIR")) { if (fs.makeDirectory("/volumes/fat-12345678/TESTDIR")) {
var wrote = false; var wrote = false;
if (fs.open("/volumes/usb/TESTDIR/HELLO.TXT", .{ .create = true, .truncate = true })) |created| { if (fs.open("/volumes/fat-12345678/TESTDIR/HELLO.TXT", .{ .create = true, .truncate = true })) |created| {
var f = created; var f = created;
wrote = (f.writeAll("mutation-ok") orelse 0) == "mutation-ok".len; wrote = (f.writeAll("mutation-ok") orelse 0) == "mutation-ok".len;
f.close(); f.close();
} }
// The created file carries a real modification time (stamped from the RTC). // The created file carries a real modification time (stamped from the RTC).
var mtime_ok = false; var mtime_ok = false;
if (fs.attributes("/volumes/usb/TESTDIR/HELLO.TXT")) |attrs| { if (fs.attributes("/volumes/fat-12345678/TESTDIR/HELLO.TXT")) |attrs| {
writeLine("fat-test: mtime {d}\n", .{attrs.mtime}); writeLine("fat-test: mtime {d}\n", .{attrs.mtime});
mtime_ok = attrs.mtime > 1_577_836_800; // after 2020-01-01 mtime_ok = attrs.mtime > 1_577_836_800; // after 2020-01-01
} }
if (mtime_ok) _ = logging.write("fat-test: mtime ok\n"); if (mtime_ok) _ = logging.write("fat-test: mtime ok\n");
// Rename it, then read from the new name and confirm the old name is gone. // Rename it, then read from the new name and confirm the old name is gone.
const renamed = fs.rename("/volumes/usb/TESTDIR/HELLO.TXT", "/volumes/usb/TESTDIR/RENAMED.TXT"); const renamed = fs.rename("/volumes/fat-12345678/TESTDIR/HELLO.TXT", "/volumes/fat-12345678/TESTDIR/RENAMED.TXT");
const old_gone = !fs.exists("/volumes/usb/TESTDIR/HELLO.TXT"); const old_gone = !fs.exists("/volumes/fat-12345678/TESTDIR/HELLO.TXT");
if (renamed and old_gone) _ = logging.write("fat-test: rename ok\n"); if (renamed and old_gone) _ = logging.write("fat-test: rename ok\n");
var readback = false; var readback = false;
if (fs.open("/volumes/usb/TESTDIR/RENAMED.TXT", .{})) |reopened| { if (fs.open("/volumes/fat-12345678/TESTDIR/RENAMED.TXT", .{})) |reopened| {
var f = reopened; var f = reopened;
var buf: [16]u8 = undefined; var buf: [16]u8 = undefined;
const got = f.read(&buf) orelse 0; const got = f.read(&buf) orelse 0;
f.close(); f.close();
readback = std.mem.eql(u8, buf[0..got], "mutation-ok"); readback = std.mem.eql(u8, buf[0..got], "mutation-ok");
} }
const removed = fs.remove("/volumes/usb/TESTDIR/RENAMED.TXT"); const removed = fs.remove("/volumes/fat-12345678/TESTDIR/RENAMED.TXT");
const gone = !fs.exists("/volumes/usb/TESTDIR/RENAMED.TXT"); const gone = !fs.exists("/volumes/fat-12345678/TESTDIR/RENAMED.TXT");
if (wrote and mtime_ok and renamed and old_gone and readback and removed and gone) { if (wrote and mtime_ok and renamed and old_gone and readback and removed and gone) {
_ = logging.write("fat-test: mutations ok\n"); _ = logging.write("fat-test: mutations ok\n");
} else { } else {
writeLine("fat-test: mutations FAILED (wrote={} mtime={} renamed={} oldgone={} read={} removed={} gone={})\n", .{ wrote, mtime_ok, renamed, old_gone, readback, removed, gone }); writeLine("fat-test: mutations FAILED (wrote={} mtime={} renamed={} oldgone={} read={} removed={} gone={})\n", .{ wrote, mtime_ok, renamed, old_gone, readback, removed, gone });
} }
} else { } else {
_ = logging.write("fat-test: mkdir /volumes/usb/TESTDIR failed\n"); _ = logging.write("fat-test: mkdir /volumes/fat-12345678/TESTDIR failed\n");
} }
if (count > 0) { if (count > 0) {
+4 -4
View File
@@ -77,7 +77,7 @@ fn park() void {
var parked: ?fs.File = null; var parked: ?fs.File = null;
var tries: u32 = 0; var tries: u32 = 0;
while (parked == null and tries < 1000) : (tries += 1) { while (parked == null and tries < 1000) : (tries += 1) {
parked = fs.open("/volumes/usb/parked", .{ .create = true }); parked = fs.open("/volumes/fat-12345678/parked", .{ .create = true });
if (parked == null) time.sleepMillis(20); if (parked == null) time.sleepMillis(20);
} }
if (parked == null) { if (parked == null) {
@@ -92,16 +92,16 @@ fn park() void {
// "parked" marker, which fails the vfs-client-death case: before the // "parked" marker, which fails the vfs-client-death case: before the
// ownership gate existed, any process could unmount any prefix, and this // ownership gate existed, any process could unmount any prefix, and this
// fixture would have deleted the volume out from under the whole boot. // fixture would have deleted the volume out from under the whole boot.
if (fs.fsUnmount("/volumes/usb")) { if (fs.fsUnmount("/volumes/fat-12345678")) {
_ = logging.write("vfstest: foreign unmount was ALLOWED\n"); _ = logging.write("vfstest: foreign unmount was ALLOWED\n");
return; return;
} }
if (fs.open("/volumes/usb/parked", .{})) |resolved| { if (fs.open("/volumes/fat-12345678/parked", .{})) |resolved| {
var verification = resolved; var verification = resolved;
verification.close(); // the park below must be the client's ONLY open verification.close(); // the park below must be the client's ONLY open
// handle — the kernel test string-matches "released 1 handle(s)". // handle — the kernel test string-matches "released 1 handle(s)".
} else { } else {
_ = logging.write("vfstest: /volumes/usb gone after refused unmount\n"); _ = logging.write("vfstest: /volumes/fat-12345678 gone after refused unmount\n");
return; return;
} }
_ = logging.write("vfstest: foreign unmount refused\n"); _ = logging.write("vfstest: foreign unmount refused\n");