103 lines
5.3 KiB
Zig
103 lines
5.3 KiB
Zig
//! The big kernel lock (BKL) — the coarse mutual exclusion that lets more than one
|
|
//! CPU run kernel code safely.
|
|
//!
|
|
//! Until SMP, the kernel's mutual exclusion *was* the interrupt flag: a critical
|
|
//! section did `cli`, and since only one core existed, nothing else could touch
|
|
//! kernel state (the discipline in docs/scheduling.md). That invariant dies the
|
|
//! instant a second core runs kernel code — `cli` on one core does nothing to
|
|
//! another. So the kernel's shared state (the scheduler queues, IPC channels) is
|
|
//! guarded by a spinlock, and the lock is **always held with local interrupts
|
|
//! disabled**, so a core's own timer interrupt can't re-enter the kernel and
|
|
//! deadlock against the lock it already holds.
|
|
//!
|
|
//! This is deliberately *one coarse lock*, not many fine ones: it's philosophically
|
|
//! aligned with a tiny kernel and it keeps the single-core correctness model
|
|
//! (docs/scheduling.md) largely intact — one lock around kernel entry instead of
|
|
//! rethinking every critical section. It's the first-design choice seL4 makes and
|
|
//! docs/smp.md endorses; per-core run queues + fine-grained locking come later, if
|
|
//! contention ever bites. Because the kernel does little, the lock is held briefly.
|
|
//!
|
|
//! **The hand-off rule.** The lock is held *across* a context switch and released
|
|
//! by whichever task resumes, not by the one that switched away. A task that blocks
|
|
//! or yields calls `enter`, mutates the queues, `schedule()`s — switching to another
|
|
//! task *with the lock still held* — and only calls `leave` once it is eventually
|
|
//! resumed and its critical section runs to the end. So every call into `schedule()`
|
|
//! (and thus `switch_context`) happens with the lock held, and every task resumes
|
|
//! from a switch holding it. A freshly-spawned task has no `enter`/`leave` frame to
|
|
//! resume into, so `task_trampoline` releases the lock explicitly on its behalf via
|
|
//! `releaseForFreshTask` before running the task body.
|
|
|
|
const std = @import("std");
|
|
const architecture = @import("architecture");
|
|
|
|
/// 0 = free, 1 = held. A single global lock for the whole kernel.
|
|
var held = std.atomic.Value(u32).init(0);
|
|
|
|
/// The per-CPU base pointer (`architecture.cpuLocal()`) of the core currently holding
|
|
/// the lock, or 0 when free. Metadata only — `held` is what enforces exclusion — read
|
|
/// solely by `releaseIfHeldHere` on the fatal-fault path. `cpuLocal()` is a unique,
|
|
/// architecture-level token per core (0 before this core's GS base is published, which
|
|
/// is fine: that window is single-core early boot, where no other core can deadlock).
|
|
var owner = std.atomic.Value(usize).init(0);
|
|
|
|
/// Enter the kernel: disable interrupts on this core, then spin until we own the
|
|
/// lock. Returns the caller's prior interrupt flags for `leave` to restore.
|
|
/// Interrupts stay off for the whole critical section so this core's timer tick
|
|
/// can't try to re-acquire the lock we're holding.
|
|
pub fn enter() u64 {
|
|
const flags = architecture.saveInterrupts();
|
|
acquire();
|
|
return flags;
|
|
}
|
|
|
|
/// Release the lock and restore the interrupt flags `enter` returned (re-enabling
|
|
/// interrupts only if they were on beforehand). The normal exit for a critical
|
|
/// section reached from task context (`yield`, `sleep`, `wait`, `wake`, IPC).
|
|
pub fn leave(flags: u64) void {
|
|
release();
|
|
architecture.restoreInterrupts(flags);
|
|
}
|
|
|
|
/// Release the lock but leave interrupts as they are. The exit for a critical
|
|
/// section running inside an interrupt handler (the timer `tick`): the handler's
|
|
/// `iretq` is what restores the interrupted context's flags, so restoring them
|
|
/// here too would open a nested-interrupt window before the return. Release only.
|
|
pub fn leaveIsr() void {
|
|
release();
|
|
}
|
|
|
|
/// Release the lock on behalf of a freshly-spawned task. Such a task is switched to
|
|
/// (with the lock held) but has no `enter`/`leave` frame of its own to release
|
|
/// through — `task_trampoline` calls this before running the task body. Interrupts
|
|
/// are enabled separately by the trampoline. Exported for the assembly trampoline.
|
|
export fn releaseForFreshTask() callconv(.c) void {
|
|
release();
|
|
}
|
|
|
|
/// Release the big kernel lock **only if this core is the one holding it** — a no-op
|
|
/// otherwise. For the fatal-fault path (a kernel-mode fault, or a nested fault inside the
|
|
/// recovery teardown, both of which run under the lock): a core that dies holding the BKL
|
|
/// must free it, or every other core spins forever in `acquire` and the whole machine
|
|
/// deadlocks instead of just that core stopping. It must NOT free a lock another core
|
|
/// owns, hence the owner check. Caveat: if we held it mid-mutation the shared state may be
|
|
/// inconsistent — but letting the other cores (and the supervisor) run on possibly-degraded
|
|
/// state is strictly more recoverable than a guaranteed total hang.
|
|
pub fn releaseIfHeldHere() void {
|
|
const me = architecture.cpuLocal();
|
|
if (me != 0 and owner.load(.monotonic) == me) release();
|
|
}
|
|
|
|
fn acquire() void {
|
|
// Test-and-test-and-set: try once, then spin read-only until the lock looks
|
|
// free before retrying the (bus-locked) swap — cheaper on the coherency fabric.
|
|
while (held.swap(1, .acquire) != 0) {
|
|
while (held.load(.monotonic) != 0) architecture.cpuRelax();
|
|
}
|
|
owner.store(architecture.cpuLocal(), .monotonic);
|
|
}
|
|
|
|
fn release() void {
|
|
owner.store(0, .monotonic);
|
|
held.store(0, .release);
|
|
}
|