"Doors" in Solaris: Lightweight RPC Using File Descriptors (1996)

How Solaris Doors Work

• Doors provide synchronous RPC between processes on the same machine.
• Conceptually, a client “enters” a server’s address space via door_call, runs the service handler, then returns via door_return.
• Implementation uses a kernel “shuttle” to hand off the scheduler’s thread/time slice from client to server and back, avoiding normal run-queue scheduling.
• Server-side, a pool of user threads is created by the doors library; these wait via door_return for work and are woken by the kernel when a call arrives.
• Arguments and return values are copied or page-mapped between address spaces; descriptors can be passed too.

Performance and Concurrency Characteristics

• Main advantage: low latency and deterministic behavior compared to sockets/pipes, because the service runs immediately in the caller’s time slice.
• Hardware assists on SPARC (ASIDs, register windows, TLB behavior) were used to minimize context-switch overhead; SpringOS fast-path calls were cited as very fast historically.
• Benefits diminish when services perform slow or blocking I/O; then you’d rather use asynchronous I/O and more conventional IPC.
• Some see concurrency complexity as no worse than normal threads if code is thread-safe; others worry about harder-to-reason-about cross-process call chains.

Relationship to Other IPC Mechanisms

• Compared to classic message-passing RPC, doors are framed as a control-transfer primitive rather than a recv-based message queue.
• Conceptually similar to CPU task gates or a specialized syscall that jumps into another user process.
• Related ideas surface in Android Binder, BeOS/Palm IPC, scheduler activations, and Linux proposals like switchto/sched_ext.
• Several commenters argue you could approximate doors in user space with Unix domain sockets, SCM_RIGHTS, and mmap, but without the same scheduling optimizations.

Real-World Use & Debugging Experience

• One team implemented an in-memory ad-targeting server accessed from Apache via doors; reported it as “extremely fast,” though it never went to production.
• A SmartOS user debugging hangs had to trace door calls across multiple processes, noting that caller threads were paused while separate server threads ran, confirming the server-thread-pool model.

Debate Over Semantics and Message Passing

• Some participants initially claimed the calling thread itself continues executing in the server process.
• Others, after reading Solaris/Illumos source and assembly, clarified that:
  • The kernel transfers scheduling state/quantum, not the literal user-space stack.
  • Separate server threads execute the handler, with door data copied onto their stacks.
• There is disagreement over whether doors should be described as “message passing”; most converge on “RPC-like with direct control handoff and data copy.”

Solaris Zones, Jails, and Containers

• Thread drifts into praising Solaris as “ahead of its time” (Zones, ZFS, DTrace, Crossbow, STMF) and FreeBSD Jails.
• Linux container tech is seen as later and initially weaker, with Docker adding distinctive features (one-process-per-container, layered FS, pipeline-oriented tooling).
• Zones were designed mainly for server consolidation; some admins found them powerful, others found Solaris administration painful.
• HP-UX Vaults and older systems (CP-67, Plan 9 namespaces, chroot) are mentioned as related historical container/partitioning mechanisms.
• Question raised whether Docker-like developer UX could have been built atop Zones; some believe yes, but it wasn’t pursued.

Other Notes

• Some wish for Doors-like primitives on Linux/FreeBSD integrated with epoll/kqueue; others argue this conflicts with doors’ inherently synchronous, quantum-transfer design.
• Brief side comments note COM/RPC as a very strong IPC design, the age of “staff engineer” titles, a naming clash with SideFX’s “Solaris,” and curiosity about surviving copies of the Spring OS.