//! The big kernel lock (BKL) — the coarse mutual exclusion that lets more than one //! CPU run kernel code safely. //! //! Until SMP, the kernel's mutual exclusion *was* the interrupt flag: a critical //! section did `cli`, and since only one core existed, nothing else could touch //! kernel state (the discipline in docs/scheduling.md). That invariant dies the //! instant a second core runs kernel code — `cli` on one core does nothing to //! another. So the kernel's shared state (the scheduler queues, IPC channels) is //! guarded by a spinlock, and the lock is **always held with local interrupts //! disabled**, so a core's own timer interrupt can't re-enter the kernel and //! deadlock against the lock it already holds. //! //! This is deliberately *one coarse lock*, not many fine ones: it's philosophically //! aligned with a tiny kernel and it keeps the single-core correctness model //! (docs/scheduling.md) largely intact — one lock around kernel entry instead of //! rethinking every critical section. It's the first-design choice seL4 makes and //! docs/smp.md endorses; per-core run queues + fine-grained locking come later, if //! contention ever bites. Because the kernel does little, the lock is held briefly. //! //! **The hand-off rule.** The lock is held *across* a context switch and released //! by whichever task resumes, not by the one that switched away. A task that blocks //! or yields calls `enter`, mutates the queues, `schedule()`s — switching to another //! task *with the lock still held* — and only calls `leave` once it is eventually //! resumed and its critical section runs to the end. So every call into `schedule()` //! (and thus `switch_context`) happens with the lock held, and every task resumes //! from a switch holding it. A freshly-spawned task has no `enter`/`leave` frame to //! resume into, so `task_trampoline` releases the lock explicitly on its behalf via //! `releaseForFreshTask` before running the task body. const std = @import("std"); const architecture = @import("architecture"); /// 0 = free, 1 = held. A single global lock for the whole kernel. var held = std.atomic.Value(u32).init(0); /// The per-CPU base pointer (`architecture.cpuLocal()`) of the core currently holding /// the lock, or 0 when free. Metadata only — `held` is what enforces exclusion — read /// solely by `releaseIfHeldHere` on the fatal-fault path. `cpuLocal()` is a unique, /// architecture-level token per core (0 before this core's GS base is published, which /// is fine: that window is single-core early boot, where no other core can deadlock). var owner = std.atomic.Value(usize).init(0); /// Enter the kernel: disable interrupts on this core, then spin until we own the /// lock. Returns the caller's prior interrupt flags for `leave` to restore. /// Interrupts stay off for the whole critical section so this core's timer tick /// can't try to re-acquire the lock we're holding. pub fn enter() u64 { const flags = architecture.saveInterrupts(); acquire(); return flags; } /// Release the lock and restore the interrupt flags `enter` returned (re-enabling /// interrupts only if they were on beforehand). The normal exit for a critical /// section reached from task context (`yield`, `sleep`, `wait`, `wake`, IPC). pub fn leave(flags: u64) void { release(); architecture.restoreInterrupts(flags); } /// Release the lock but leave interrupts as they are. The exit for a critical /// section running inside an interrupt handler (the timer `tick`): the handler's /// `iretq` is what restores the interrupted context's flags, so restoring them /// here too would open a nested-interrupt window before the return. Release only. pub fn leaveIsr() void { release(); } /// Release the lock on behalf of a freshly-spawned task. Such a task is switched to /// (with the lock held) but has no `enter`/`leave` frame of its own to release /// through — `task_trampoline` calls this before running the task body. Interrupts /// are enabled separately by the trampoline. Exported for the assembly trampoline. export fn releaseForFreshTask() callconv(.c) void { release(); } /// Release the big kernel lock **only if this core is the one holding it** — a no-op /// otherwise. For the fatal-fault path (a kernel-mode fault, or a nested fault inside the /// recovery teardown, both of which run under the lock): a core that dies holding the BKL /// must free it, or every other core spins forever in `acquire` and the whole machine /// deadlocks instead of just that core stopping. It must NOT free a lock another core /// owns, hence the owner check. Caveat: if we held it mid-mutation the shared state may be /// inconsistent — but letting the other cores (and the supervisor) run on possibly-degraded /// state is strictly more recoverable than a guaranteed total hang. pub fn releaseIfHeldHere() void { const me = architecture.cpuLocal(); if (me != 0 and owner.load(.monotonic) == me) release(); } fn acquire() void { // Test-and-test-and-set: try once, then spin read-only until the lock looks // free before retrying the (bus-locked) swap — cheaper on the coherency fabric. while (held.swap(1, .acquire) != 0) { while (held.load(.monotonic) != 0) architecture.cpuRelax(); } owner.store(architecture.cpuLocal(), .monotonic); } fn release() void { owner.store(0, .monotonic); held.store(0, .release); }