Apple’s Darwin OS and XNU Kernel Deep Dive

Original Article ↗ Hacker News Discussion ↗

Overall reception and scope of the article

  • Many readers found it the clearest high-level overview of Darwin/XNU so far, good enough to print and use for explaining OS X/macOS evolution.
  • Some corrections were suggested: VM/paging details, Mach ports vs entitlements, address space history, Secure Enclave vs Exclaves, and current status of hypervisor/virtualization on iOS.
  • The author clarifies the piece is primarily a synthesis of prior literature and reverse‑engineering work, not original RE.

Mach, BSD, and Darwin’s place in the ecosystem

  • Several comments refine the claim that Mach VM “lives on” in the BSDs:
    • FreeBSD replaced its early Mach‑derived VM with a complete rewrite; current linkage to Mach is mostly historical.
    • NetBSD/OpenBSD replaced Mach‑style VM with UVM; DragonFly took a different path.
    • Among “living” systems, macOS/iOS remain the main active Mach 3 descendant.
  • Debate on project health of DragonFlyBSD and how much Mach influence still matters in practice.

Windows NT and “personalities” comparison

  • Extended discussion of NT’s environment subsystems (Win32, OS/2, POSIX), their historical intent and practical demise.
  • Clarification that Win32 became “primary” early, so other subsystems depended on it and are now effectively gone; WSL1/2 are architecturally different from classic subsystems.
  • Historical correction around the OS/2 subsystem, its limitations (mostly 16‑bit 1.x), and its relationship to Microsoft/IBM’s split.

Darwin vs Linux as Apple’s kernel base

  • One camp laments that Apple didn’t adopt Linux, seeing a lost opportunity for the broader open‑source ecosystem.
  • Others argue strongly this was never realistic or advantageous in the late 1990s:
    • NeXTSTEP/Mach expertise in‑house, PowerPC support, and Apple’s lack of time/money for a full kernel rewrite.
    • Linux then was immature as a desktop OS and less obviously superior technically.
    • GPL would have constrained Apple’s preferred closed‑module model and control over the stack.
  • Consensus that once OS X was established, switching to Linux would have been high‑risk with little upside; also would have created a more fragile “kernel monoculture”.

Filesystems: ZFS vs APFS

  • One thread explains why ZFS is heavyweight for consumer/mobile use: deep VM integration, ARC, transactional semantics, large‑page and mmap requirements, etc.
  • Historical notes: Apple had an internal ZFS port around 10.5–10.6 but killed it; reasons debated (rumored interpersonal issue, ZFS lawsuits/patents, Oracle acquisition, resource use).
  • Several suggest APFS and similar designs (btrfs, bcachefs) are better fits for constrained and mobile devices.

Paging, swap, and Mach microkernel details

  • Clarification that:
    • Original Mach supported true user‑space pagers, but Darwin hasn’t really used that for ~20 years.
    • dynamic_pager historically just managed swap files via syscalls; actual paging stayed in kernel and is now fully kernel‑side.
    • Only explicitly “pageable” kernel allocations can be swapped; most kernel memory is wired.
  • Some discussion on why swap‑file management once lived in user space (microkernel legacy, flexibility) and later moved in‑kernel.

Security architecture and code signing

  • One long, detailed comment argues Apple is far ahead in OS security:
    • Mach‑O code signatures with per‑page hashes verified by the kernel/VMM.
    • A rich “code requirement” language and designated requirements as stable identities.
    • Entitlements as a structured capability system, backed by provisioning profiles.
    • Gatekeeper’s bundle‑integrity checks for data files, and Seatbelt’s programmable sandbox policies.
    • Recent tightening: all apps sandboxed; fine‑grained rules for whether one app can modify another’s bundles.
  • Another viewpoint stresses these mechanisms also reinforce vendor control; on macOS they’re considered somewhat user‑configurable (SIP off, Gatekeeper knobs), but the friction for non‑notarized apps is increasing.

BSD vs Linux trajectories and containers

  • Lengthy argument over why Linux “won” over the BSDs:
    • Factors cited: early lawsuit drag on BSD, friendlier contribution model and licensing dynamics for industry, faster GUI/install evolution, strong corporate backing, early SMP and device driver advantages.
    • Counter‑claims focus on BSD leadership style and strategic missteps (e.g., late or reluctant embrace of JVM, mainstream containers, and non‑x86 platforms).
  • Debate over FreeBSD jails vs Linux containers: jails are older, strong isolation primitives but originally oriented toward virtual hosting, while modern container ecosystems evolved on Linux (with Solaris zones acknowledged as important predecessors).

XNU architecture, drivers, and evolution

  • XNU is characterized as a Mach 3–derived “hybrid”: Mach primitives with a large BSD component linked in‑kernel.
  • Several clarifications:
    • All IOKit calls go over Mach IPC; IOKit’s C++ subset is for kernel‑side drivers, independent of Cocoa/Carbon decisions.
    • Apple is steadily moving many drivers back to user space via DriverKit, aligning closer to microkernel ideals for security.
  • One commenter notes Darwin’s willingness to break compatibility (syscalls, dynamic linking, driver models) to achieve performance, security, or simplicity.

Intel support timeline and architectures

  • Speculation on when x86 Macs will be dropped: educated guesses cluster around 2027–2028 based on macOS support lifecycles and the last Intel hardware sale dates.
  • Some think Intel could be dropped as soon as the next major release; others expect at least one more Intel‑compatible macOS.

Language choices: Swift vs Rust

  • Some disappointment about Apple’s apparent lack of enthusiasm for Rust despite its safety benefits.
  • Others point to official statements and talks positioning Swift as Apple’s intended “successor language” to C/C++/Objective‑C, including for low‑level and even firmware‑like use (e.g., Embedded Swift).
  • Debate continues over whether Swift can fully match C/C++/Rust performance in all domains, but Apple’s strategy is clearly to bet on Swift rather than adopt Rust.