daniel/danos
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

Phase 2a: FAT engine mutations + O_TRUNC (fix the overwrite corruption)

Browse Source 
The FAT engine could create, read, write, and grow files, but never free clusters
or make directories — so overwriting a shorter file left a stale tail (a real bug:
corrupt boot-log re-flushes, and later corrupt compiler cache/.o files). This adds
the mutation half of the engine, with the corruption fix wired all the way through.

Engine (system/services/fat/engine.zig), all host-tested:
- freeChain: return a cluster chain to the pool (bounded against a corrupt cycle) —
  the shared primitive under truncate and remove.
- truncate: free the chain and zero the entry's size/first-cluster (O_TRUNC).
- createDirectory (mkdir): allocate + initialise a cluster with "." and ".." and add
  the directory entry to the parent.
- removeFile (unlink): free the chain and mark the 8.3 entry plus any preceding
  long-name entries deleted, so a reused slot can't inherit an orphaned long name.
  Refuses directories.
- createFile now shares a common addEntry helper with createDirectory.

O_TRUNC wired end to end: a truncate open-flag (vfs protocol) that the router already
forwards; runtime.fs.OpenOptions.truncate; and fat's handleOpen calls engine.truncate
on an existing file. The boot-log flush (log-flush + init) now opens with truncate, so
a shorter log on a later boot of the same stick leaves no stale tail — closing the
caveat from the boot-log work.

mkdir/unlink are engine-complete and host-tested but not yet exposed as VFS
operations / runtime.fs methods (they are new path-based ops needing router cases);
rename and the richer stat (mtime/mode, blocked on wall-clock) remain. See
docs/zig-self-hosting.md (Phase 2).

Verified: zig build, zig build test (8 engine host tests, incl. truncate, the
overwrite-no-stale-tail regression, remove, and mkdir), zig build check-fat-image, and
a sequential QEMU sweep — fat-mount, log-flush (DANOS.LOG read back at 11804 bytes,
clean, with the truncate-based final flush), vfs, vfs-client-death, usb-storage,
orderly-shutdown, initial-ramdisk, smoke — all green.
This commit is contained in:
Daniel Samson
2026-07-13 20:02:01 +01:00
parent 347a041d85
commit a32eed877d
7 changed files with 301 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
system/services/fat/engine.zig
+269 -11
View File
@@ -1,6 +1,8 @@
//! The FAT filesystem engine: mount a block device, walk the FAT and directory
//! structures, and read / write / create files. FAT12/16/32 (the type is
//! detected from the cluster count). Pure logic over a `BlockDevice` interface —
//! structures, and read / write / create / truncate files, remove files, and make
//! directories. FAT12/16/32 (the type is detected from the cluster count). The
//! crash-safe write order is data -> FAT -> directory; truncate and remove free the
//! cluster chain, then update the directory. Pure logic over a `BlockDevice` interface —
//! no IPC — so it is host-testable against a RAM-backed image (see the tests at
//! the bottom). The fat.zig server wraps a real `.block` device in a BlockDevice
//! and serves this over the VFS protocol.
@@ -238,6 +240,21 @@ pub const FileSystem = struct {
return null;
}
// Free every cluster of the chain starting at `first`, returning them to the
// pool. A first < 2 (an empty file) frees nothing. Bounded against a corrupt
// cyclic chain by the cluster count so it can never loop forever.
fn freeChain(self: *FileSystem, first: u32) void {
var cluster = first;
var guard: u32 = 0;
const limit = self.geometry.cluster_count + 2;
while (cluster >= 2 and cluster < limit and guard < limit) : (guard += 1) {
const next = self.readFatEntry(cluster);
_ = self.writeFatEntry(cluster, on_disk.free_cluster);
if (self.isEndOfChain(next) or next < 2) break;
cluster = next;
}
}
// --- directory iteration ------------------------------------------------
// The absolute LBA of the `sector_index`th sector of directory `dir`, or null
@@ -546,6 +563,18 @@ pub const FileSystem = struct {
return consumed;
}
/// Truncate a file node to zero length: free its cluster chain and clear its
/// size and first cluster in the directory entry. This is O_TRUNC — the fix for
/// re-opening and overwriting an existing file, whose old (longer) contents
/// would otherwise linger past the new end (a silent-corruption bug for anything
/// that rewrites a file in place, like the boot-log flush).
pub fn truncate(self: *FileSystem, node: *Node) void {
self.freeChain(node.first_cluster);
node.first_cluster = 0;
node.size = 0;
self.updateEntry(node.*);
}
// Write a node's size and first cluster back into its 8.3 directory entry.
fn updateEntry(self: *FileSystem, node: Node) void {
if (!node.has_entry) return;
@@ -557,11 +586,10 @@ pub const FileSystem = struct {
_ = self.blockWrite(node.entry_sector, &self.dir_sector);
}
/// Create an 8.3-named file in directory `dir`. Returns the new (empty) node,
/// or null if the name is not 8.3-representable or no directory slot is free.
pub fn createFile(self: *FileSystem, dir: Node, name: []const u8) ?Node {
const raw = to83(name) orelse return null;
// Find a free directory slot (a 0x00 or 0xE5 entry), growing the directory.
// Add an 8.3 directory entry to `dir` with the given attributes, first cluster,
// and size, reusing a free (0x00 or 0xE5) slot and growing the directory chain
// if needed. Returns the new node (with its entry location) or null if full.
fn addEntry(self: *FileSystem, dir: Node, raw: [11]u8, attributes: u8, first_cluster: u32, size: u32) ?Node {
var sector_index: u32 = 0;
while (self.dirSectorLba(dir, sector_index, true)) |lba| : (sector_index += 1) {
if (!self.blockRead(lba, &self.dir_sector)) return null;
@@ -572,13 +600,15 @@ pub const FileSystem = struct {
if (existing.isFree()) {
var entry = std.mem.zeroes(on_disk.DirectoryEntry);
entry.name = raw;
entry.attributes = on_disk.attribute_archive;
entry.attributes = attributes;
entry.file_size = size;
entry.setFirstCluster(first_cluster);
@memcpy(self.dir_sector[offset .. offset + @sizeOf(on_disk.DirectoryEntry)], std.mem.asBytes(&entry));
if (!self.blockWrite(lba, &self.dir_sector)) return null;
return .{
.first_cluster = 0,
.size = 0,
.is_directory = false,
.first_cluster = first_cluster,
.size = size,
.is_directory = attributes & on_disk.attribute_directory != 0,
.entry_sector = lba,
.entry_offset = offset,
.has_entry = true,
@@ -590,6 +620,125 @@ pub const FileSystem = struct {
}
return null;
}
/// Create an 8.3-named file in directory `dir`. Returns the new (empty) node,
/// or null if the name is not 8.3-representable or no directory slot is free.
pub fn createFile(self: *FileSystem, dir: Node, name: []const u8) ?Node {
const raw = to83(name) orelse return null;
return self.addEntry(dir, raw, on_disk.attribute_archive, 0, 0);
}
/// Create an 8.3-named subdirectory in `dir`: allocate and initialise its first
/// cluster with "." (itself) and ".." (the parent) entries, then add its
/// directory entry to `dir`. Returns the new directory node, or null (bad name,
/// no free cluster, or the directory is full). Long names are not created.
pub fn createDirectory(self: *FileSystem, dir: Node, name: []const u8) ?Node {
const raw = to83(name) orelse return null;
const cluster = self.allocateCluster() orelse return null;
self.zeroCluster(cluster);
// ".." points at the parent: 0 for the fixed root on FAT12/16, the root
// cluster on FAT32, else the parent's own first cluster.
const parent_cluster: u32 = if (dir.first_cluster != 0)
dir.first_cluster
else if (self.geometry.fat_type == .fat32)
self.geometry.root_cluster
else
0;
var dot = std.mem.zeroes(on_disk.DirectoryEntry);
dot.name = [_]u8{'.'} ++ ([_]u8{' '} ** 10);
dot.attributes = on_disk.attribute_directory;
dot.setFirstCluster(cluster);
var dotdot = std.mem.zeroes(on_disk.DirectoryEntry);
dotdot.name = [_]u8{ '.', '.' } ++ ([_]u8{' '} ** 9);
dotdot.attributes = on_disk.attribute_directory;
dotdot.setFirstCluster(parent_cluster);
var first_sector = [_]u8{0} ** sector_size;
const entry_size = @sizeOf(on_disk.DirectoryEntry);
@memcpy(first_sector[0..entry_size], std.mem.asBytes(&dot));
@memcpy(first_sector[entry_size .. 2 * entry_size], std.mem.asBytes(&dotdot));
if (!self.blockWrite(self.clusterSector(cluster, 0), &first_sector)) {
self.freeChain(cluster);
return null;
}
return self.addEntry(dir, raw, on_disk.attribute_directory, cluster, 0) orelse {
self.freeChain(cluster);
return null;
};
}
const EntryLoc = struct { sector: u64, offset: u32 };
// Mark a directory entry deleted in place (its name[0] set to 0xE5).
fn markDeleted(self: *FileSystem, sector: u64, offset: u32) void {
if (!self.blockRead(sector, &self.dir_sector)) return;
self.dir_sector[offset] = 0xE5;
_ = self.blockWrite(sector, &self.dir_sector);
}
/// Remove a file named `name` from directory `dir`: free its cluster chain and
/// mark its 8.3 entry — and any long-name entries immediately preceding it —
/// deleted, so the slots (and the long name) are reusable without a later entry
/// that reuses them inheriting the orphaned long name. Refuses a directory (a
/// separate rmdir would have to check emptiness). Returns true if removed.
pub fn removeFile(self: *FileSystem, dir: Node, name: []const u8) bool {
var run: [21]EntryLoc = undefined; // the long-name entries before the 8.3 one
var run_len: usize = 0;
var long_name: [260]u8 = undefined;
var long_len: usize = 0;
var sector_index: u32 = 0;
while (self.dirSectorLba(dir, sector_index, false)) |lba| : (sector_index += 1) {
if (!self.blockRead(lba, &self.dir_sector)) return false;
var i: u32 = 0;
while (i < entries_per_sector) : (i += 1) {
const offset = i * @sizeOf(on_disk.DirectoryEntry);
const entry = std.mem.bytesToValue(on_disk.DirectoryEntry, self.dir_sector[offset .. offset + @sizeOf(on_disk.DirectoryEntry)]);
if (entry.isEnd()) return false;
if (entry.name[0] == 0xE5) {
run_len = 0;
long_len = 0;
continue;
}
if (entry.isLongName()) {
if (run_len < run.len) {
run[run_len] = .{ .sector = lba, .offset = offset };
run_len += 1;
}
const lfn = std.mem.bytesToValue(on_disk.LongNameEntry, self.dir_sector[offset .. offset + @sizeOf(on_disk.LongNameEntry)]);
const order = lfn.order & 0x1F;
if (order >= 1 and order <= 20) {
var chunk: [13]u8 = undefined;
const n = longNameChars(lfn, &chunk);
const start = (order - 1) * 13;
if (start + n <= long_name.len) {
@memcpy(long_name[start .. start + n], chunk[0..n]);
if (lfn.order & 0x40 != 0) long_len = start + n;
}
}
continue;
}
if (entry.isVolumeLabel()) {
run_len = 0;
long_len = 0;
continue;
}
// A real 8.3 entry.
var short: [12]u8 = undefined;
const display = if (long_len > 0) long_name[0..long_len] else format83(entry.name, &short);
if (nameMatches(display, name)) {
if (entry.isDirectory()) return false; // not for directories
self.freeChain(entry.firstCluster());
self.markDeleted(lba, offset);
var r: usize = 0;
while (r < run_len) : (r += 1) self.markDeleted(run[r].sector, run[r].offset);
return true;
}
run_len = 0;
long_len = 0;
}
}
return false;
}
};
// --- tests: a RAM-backed FAT16 image ----------------------------------------
@@ -705,3 +854,112 @@ test "create, write, read back a file through the engine" {
try std.testing.expectEqualStrings("HELLO.TXT", listing.name_buffer[0..listing.name_len]);
try std.testing.expect(fs.listEntry(fs.rootNode(), 1) == null);
}
test "truncate frees the chain and zeroes the file" {
const allocator = std.testing.allocator;
const bytes = try allocator.alloc(u8, 5000 * sector_size);
defer allocator.free(bytes);
formatFat16(bytes);
var disk = RamDisk{ .bytes = bytes };
var fs = FileSystem.mount(disk.device()).?;
var node = fs.createFile(fs.rootNode(), "BIG.BIN").?;
var payload: [2000]u8 = undefined;
for (&payload, 0..) |*b, i| b.* = @truncate(i);
_ = fs.writeFile(&node, 0, &payload);
const cluster = node.first_cluster;
try std.testing.expect(cluster >= 2);
fs.truncate(&node);
try std.testing.expectEqual(@as(u32, 0), node.size);
try std.testing.expectEqual(@as(u32, 0), node.first_cluster);
// The old first cluster is free again.
try std.testing.expectEqual(on_disk.free_cluster, fs.readFatEntry(cluster));
// Re-resolve: the persisted entry is empty, and reads produce nothing.
const resolved = fs.resolve("/BIG.BIN").?;
try std.testing.expectEqual(@as(u32, 0), resolved.size);
var buf: [16]u8 = undefined;
try std.testing.expectEqual(@as(usize, 0), fs.readFile(resolved, 0, &buf));
}
test "overwrite after truncate leaves no stale tail (the O_TRUNC corruption fix)" {
const allocator = std.testing.allocator;
const bytes = try allocator.alloc(u8, 5000 * sector_size);
defer allocator.free(bytes);
formatFat16(bytes);
var disk = RamDisk{ .bytes = bytes };
var fs = FileSystem.mount(disk.device()).?;
// Write a long file, then truncate-and-rewrite a short one — the O_TRUNC flow.
var node = fs.createFile(fs.rootNode(), "LOG.TXT").?;
_ = fs.writeFile(&node, 0, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"); // 32 bytes
fs.truncate(&node);
_ = fs.writeFile(&node, 0, "bb");
// Size is the short length — no lingering old bytes past the new end.
const resolved = fs.resolve("/LOG.TXT").?;
try std.testing.expectEqual(@as(u32, 2), resolved.size);
var buf: [8]u8 = undefined;
const n = fs.readFile(resolved, 0, &buf);
try std.testing.expectEqualStrings("bb", buf[0..n]);
}
test "remove a file frees its slot and its cluster chain" {
const allocator = std.testing.allocator;
const bytes = try allocator.alloc(u8, 5000 * sector_size);
defer allocator.free(bytes);
formatFat16(bytes);
var disk = RamDisk{ .bytes = bytes };
var fs = FileSystem.mount(disk.device()).?;
var node = fs.createFile(fs.rootNode(), "GONE.TXT").?;
var payload: [1000]u8 = undefined;
for (&payload, 0..) |*b, i| b.* = @truncate(i);
_ = fs.writeFile(&node, 0, &payload);
const cluster = fs.resolve("/GONE.TXT").?.first_cluster;
try std.testing.expect(cluster >= 2);
try std.testing.expect(fs.removeFile(fs.rootNode(), "GONE.TXT"));
// Gone from the directory, its cluster free, root empty again.
try std.testing.expect(fs.resolve("/GONE.TXT") == null);
try std.testing.expectEqual(on_disk.free_cluster, fs.readFatEntry(cluster));
try std.testing.expect(fs.listEntry(fs.rootNode(), 0) == null);
// Removing a missing file reports false.
try std.testing.expect(!fs.removeFile(fs.rootNode(), "GONE.TXT"));
// A directory is refused (it is not a file).
_ = fs.createDirectory(fs.rootNode(), "ADIR").?;
try std.testing.expect(!fs.removeFile(fs.rootNode(), "ADIR"));
}
test "create a subdirectory with . and .. and a file inside" {
const allocator = std.testing.allocator;
const bytes = try allocator.alloc(u8, 5000 * sector_size);
defer allocator.free(bytes);
formatFat16(bytes);
var disk = RamDisk{ .bytes = bytes };
var fs = FileSystem.mount(disk.device()).?;
const made = fs.createDirectory(fs.rootNode(), "SUB").?;
try std.testing.expect(made.is_directory);
try std.testing.expect(made.first_cluster >= 2);
// It resolves as a directory, with "." and ".." as its first two entries.
const dir = fs.resolve("/SUB").?;
try std.testing.expect(dir.is_directory);
const dot = fs.listEntry(dir, 0).?;
try std.testing.expectEqualStrings(".", dot.name_buffer[0..dot.name_len]);
const dotdot = fs.listEntry(dir, 1).?;
try std.testing.expectEqualStrings("..", dotdot.name_buffer[0..dotdot.name_len]);
// A file created inside is reachable by its full path.
var child = fs.createFile(dir, "INNER.TXT").?;
_ = fs.writeFile(&child, 0, "hi");
const inner = fs.resolve("/SUB/INNER.TXT").?;
try std.testing.expectEqual(@as(u32, 2), inner.size);
// The root lists SUB as a directory.
const listing = fs.listEntry(fs.rootNode(), 0).?;
try std.testing.expectEqualStrings("SUB", listing.name_buffer[0..listing.name_len]);
try std.testing.expect(listing.is_directory);
}