danos/docs/device-driver-development/ipc.md

# IPC: the kernel-ipc transport

Inter-process communication is the **backbone of a microkernel**. Once drivers and
services run isolated in their own address spaces ([vision](../vision.md)), they can't
just call each other — a request becomes bytes on a wire. In a microkernel, whatever
was a function call across a monolithic kernel is IPC, so it's a first-class
concern, not an afterthought.

This document describes **one transport** — the bottom layer (L0) of the
communication stack defined in
[communication.md](../os-development/communication.md), which owns the model
and the vocabulary (*protocol*, *channel*, *packet*, *signal*, *endpoint*).
kernel-ipc is the **first** transport, not the only possible one: in
buffer-plus-doorbell terms it is a kernel-owned mailbox with the scheduler as
the doorbell. Its distinguishing properties, which the layers above may rely
on where they say so:

- **Rendezvous.** A call is a synchronous meeting, copied sender-page to
  receiver-page — natural backpressure, no queue to size.
- **Capability carriage.** The *only* transport that can move a handle
  between processes. Channels are therefore always established over
  kernel-ipc, and it remains every channel's control path even when bulk
  data is negotiated onto a fatter transport (a shared-memory ring).
- **Verified source.** Every delivery carries the kernel-stamped badge — the
  identity the channel layer attaches to received packets.
- **Bounded packets.** 256 bytes call/reply, 64 pushed — the floor every
  protocol may assume on any transport.

Three properties keep the networking analogy honest — kernel-ipc is
networking-*shaped*, not TCP:

- **Channels over it are RPC-shaped, not streams.** Packets, call/reply,
  datagram pushes — closer to UDP plus RPC than to a byte stream. Ordering
  exists per exchange (a reply answers its call), not across a channel.
- **Possession is the connection.** There is no handshake state in the
  kernel: holding the capability *is* having the channel. A provider's one
  endpoint terminates every client's channel at once, demultiplexed by badge
  — like every client sharing the server's listening socket, with
  per-connection state living in the provider, keyed by badge. A *private*
  channel (a dedicated endpoint pair) is built when wanted: that is exactly
  what `subscribe` does.
- **Packets never fragment.** If it doesn't fit in a packet, it isn't a
  packet: bulk data lives in shared memory and a packet (or signal) is the
  doorbell. The display path already works this way.

The rest of this document is the implementation, bottom-up: the kernel-thread
queue the blocking discipline was worked out on, then endpoints — this
transport's termination points.

## The kernel-thread queue

The first form is a **bounded blocking queue** (`system/kernel/ipc.zig`): a
fixed-size ring buffer of messages with a producer/consumer rendezvous, built
on the scheduler's [wait queues](../os-development/scheduling.md). (Its type
is still named `Channel(T, capacity)` — it predates the vocabulary above, and
is a *queue between kernel threads in one address space*, not a channel in
the model's sense; a rename can ride a later flag-day.)

`Channel(T, capacity)` is generic over the message type and buffer size. It holds a
ring buffer, a count, and two wait queues:

- **`send(msg)`** — if the queue is full, block on the *not-full* queue; otherwise
  write the message, bump the count, and wake a waiting receiver.
- **`receive()`** — if the queue is empty, block on the *not-empty* queue; otherwise
  take a message, drop the count, and wake a waiting sender.

Neither side busy-waits: a full queue parks the sender, an empty one parks the
receiver, and each operation wakes the other side when it makes progress possible.

Two details make it correct:

- **Recheck in a loop.** A woken task re-tests the condition (`while (full) wait`)
  rather than assuming the slot is still available — another waiter may have taken
  it first. This is the standard guard against spurious or racing wakeups.
- **One critical section.** `send`/`receive` run under the [big kernel
  lock](../os-development/smp.md) (`sync.enter` / `sync.leave`), which disables interrupts on this
  core *and* takes the kernel's one spinlock — since SMP, the interrupt flag alone
  is not atomicity, because `cli` on one core does nothing to another. So checking
  the condition and committing the block/enqueue happen atomically both with respect
  to the timer preempting mid-operation and to the other side running on another
  CPU. `waitLocked` / `wakeLocked` are the variants that assume the caller already
  holds that critical section.

### Verifying it

The `ipc` test (see [testing.md](../testing.md)) runs a producer and a consumer passing
**100 messages through a 4-slot queue**. The small buffer means the queue goes
full and empty over and over, so both the blocking-send and blocking-receive paths are
exercised heavily. The messages arrive intact and in order (their sum is the
expected `5050`), and neither task busy-waits — they block and wake each other.

## Endpoints: the termination points

A queue connects two kernel threads sharing one address space. Real providers are
*processes*, so a packet has to cross an address-space boundary. That's
`system/kernel/ipc-synchronous.zig`, and its shape is L4's: a synchronous **rendezvous** at an
`Endpoint`, with the packet copied directly from the sender's pages to the receiver's
(`copyAcross` walks both sets of page tables through the physmap — no CR3 switch, no
bounce buffer).

Two syscalls carry the request/reply exchange:

- **`ipc_call(h, msg, reply)`** — copy the request packet to the provider, block
  until the reply packet comes back.
- **`ipc_reply_wait(h, reply, recv)`** — reply to the client you're still holding (if
  any), then block for the next request. One syscall, because a provider's steady state
  is *always* "finish the last one, wait for the next".

An endpoint is reached by **handle** — a small integer index into the process's handle
table (`Task.handles`), exactly like a file descriptor, and just as unforgeable.
The provider never learns the client's identity beyond the **badge** delivered
alongside each packet: the caller's task id, stamped by the kernel —
unforgeable source addressing, a property a network's source field lacks.

The bootstrap problem — how a channel is first established — is the subject of
[protocol-namespace.md](../os-development/protocol-namespace.md): a protocol is
resolved by name and the channel arrives as a capability. (The mechanism it
replaced — `ipc_register`/`ipc_lookup` under compile-time `ServiceId` integers —
is gone: both syscalls and the enum were deleted when the registry landed, and
their syscall numbers are left vacant.)

### Interrupts are signals

`notifyFromIsr` posts an *asynchronous* signal to an endpoint — no payload, no
reply owed — and wakes whoever is blocked in `reply_wait`. Its badge has the top bit
set (`notify_badge_bit`), which is how a driver's single event loop distinguishes "a
client wants something" from "the hardware wants something". Signals sit in a
small coalescing ring on the endpoint, so an interrupt taken while the driver was busy
elsewhere is not lost — coalesced, never dropped, which is exactly a signal's
contract (the *count* may collapse; the *fact* may not).

This is what makes a user-space driver possible at all, and it's the subject of
[drivers.md](drivers.md).

## What's next (partly done since)

- **Priority inheritance** through IPC — still open: a high-priority client
  blocked on a low-priority provider suffers unbounded priority inversion.
- **Handle transfer.** *Landed as cap-passing (M13)*: `ipc_call` and
  `ipc_reply_wait` carry an optional capability alongside the bytes (`send_cap`),
  copying an endpoint or shared-memory handle into the peer's table — the
  mechanism by which channels are established and private channels built. First
  user: [input](input.md) subscribers register by handing over their own
  endpoint, and class drivers get a private channel to one device.
- **Asynchronous / buffered send** for the cases where a rendezvous is the wrong
  shape (logging, event fan-out). *Landed as `ipc_send`* — a
  non-blocking post of an event packet (≤ 64 bytes) to an endpoint's bounded
  queue, delivered through `reply_wait` (badge bit `notify_message_bit`). Built
  for, and first used by, the [input service](input.md)'s keyboard-event
  broadcast, where a synchronous push would let one dead subscriber hang the
  fan-out. A full queue drops the oldest — event packets are droppable by
  design ([protocol-namespace.md](../os-development/protocol-namespace.md)'s
  wiring section states the rule).
- **A bounded reply** — half landed. The copy is still one packet
  (256 bytes) under the big kernel lock, but bulk transfer got its shared
  pages: `shared_memory_create`/`map`/`physical`, the region handle delegated as
  a capability (above) — the packets-never-fragment rule in practice.
  virtio-gpu's scanout surface is the first user
  ([display-v2.md](display-v2.md)).

## Lifecycle conventions over IPC (M17)

Three conventions from [process-lifecycle.md](../os-development/process-lifecycle.md) ride the
signal mechanism:

- **Process signals** arrive as endpoint signals on the endpoint a process
  nominated with `signal_bind` (`process.bindSignals`): badge = the signal bit
  plus the coalesced pending mask (`process.signalsFrom` decodes). Statements,
  never questions; no payload, no reply.
- **One-shot timers** (`timer_bind`, `time.timerOnce`) land as a
  timer-bit signal — the timed wait: a service arms a deadline and keeps
  serving, instead of blocking in sleep.
- **Kernel notifications go only to your own endpoint.** `signal_bind`,
  `timer_bind`, `process_subscribe`, `irq_bind`, `msi_bind`, and spawn's exit
  endpoint all *nominate where the kernel will speak*, and all of them refuse an
  endpoint the caller did not create (`-EPERM`; the check is `ipc.ownedBy`,
  normalized to the process, so any thread may nominate an endpoint a sibling
  created). Holding a handle is not enough, because holding a handle is cheap:
  `fs_resolve` installs a mounted backend's capability in *any* caller's table,
  so every process holds a handle to PID 1's mailbox. Without the rule, "bind
  init's endpoint, then signal yourself" is a genuine, kernel-stamped `terminate`
  badge in PID 1's queue — a shutdown a receiver has no way to disbelieve — and
  timers, which carry no identity at all, multiply any loop that re-arms on its
  own landing.
- **A capability that arrives belongs to the turn.** The kernel installs a sent
  capability in the receiver's table whatever the message's length or kind, so a
  receive loop must dispose of one on *every* path — the ping, the notification,
  the malformed request. The service harness (`service.run`) and PID 1 both hold
  it in an `ipc.Arrival`, released by a `defer`, and a handler that means to keep
  it says `take()`: forgetting closes, keeping is explicit. The reverse
  arrangement leaks a handle-table slot per request, and thirty-two unauthorized
  zero-length pings then end a service's ability to accept any capability —
  no subscribe, no shared-memory handover — for the rest of the boot.
- **The universal ping**: a **zero-length request is the liveness probe**,
  answered with a zero-length reply by the service harness itself
  (`service.run`). No protocol's requests start at length zero, so the
  encoding cannot collide, and a wedged service simply fails to answer — which
  is the diagnosis. Deep health ("can I reach my hardware?") stays a per-service
  protocol packet.