Asynchronous IO: the next billion-dollar mistake?
Scope of the Discussion
- Debate centers on whether async I/O is a mistaken direction compared to making OS threads vastly more efficient.
- Comments split between: “async is essential and natural” vs. “threads + better runtimes are sufficient and simpler.”
Can OS Threads Be Made “Good Enough”?
- Some argue physics and CPU design limit how cheap kernel threads and context switches can get (stacks, caches, security mitigations, address-space model).
- Others question whether we are actually at the limits, suggesting claims of “threads are as fast as they can be” are unproven.
- Memory overhead of per-thread stacks is seen as a hard constraint for hundreds of thousands of threads.
What Async I/O Actually Buys You
- One view: async is mostly about efficiency and high-concurrency I/O (many idle connections, batching, fewer syscalls).
- Another: async/await exposes time as a first-class concept, making composition, cancellation, and timeouts easier.
- Counterpoint: cancellations are tricky in both models, especially with side effects (e.g., partial writes); true cancellation often degrades to “deadline hints” plus cleanup logic.
Complexity, Ergonomics, and “Function Colouring”
- Some report large async systems becoming complex, ending up mixing async, thread pools, and blocking calls—getting “the worst of both.”
- Others find thread-based concurrency far harder (deadlocks, races, lock ordering), and see async/await as easier for typical developers.
- Function colouring (async vs sync call chains) is widely seen as painful; retrofitting sync code to async is described as a multi‑year effort in real projects.
Kernel and OS Realities (Linux, Windows, io_uring, etc.)
- Several point out that most hardware and kernels are inherently event-driven; many “sync” calls are async under the hood plus a blocking wait.
- Linux historically exposed mostly synchronous POSIX APIs; newer mechanisms (AIO, io_uring, AF_XDP) provide real async I/O, including for files, though adoption in userland is still uneven.
- NT’s I/O model is cited as fully async from the start (IOCP, Registered I/O), with newer APIs (IoRing) paralleling Linux’s io_uring to reduce copies and syscalls.
- There is disagreement over how much the article underestimates existing async file I/O capabilities.
Alternative Models: Green Threads, Actors, Structured Concurrency
- Many suggest the “parallel universe” already exists in language runtimes: Go’s goroutines, Erlang’s processes, Java virtual threads, user‑space threads, actors, CSP, and structured concurrency.
- These models often hide async I/O behind a synchronous-looking API, while relying on kernel async primitives underneath.
- Some worry pushing “lightweight threads in the kernel” would freeze experimentation at the OS level and slow progress.
Hardware Nature and the “Inherently Async” Argument
- Several argue the world is fundamentally concurrent: NIC queues, DMA, storage, and even RAM distance make I/O inherently asynchronous.
- Others respond that while hardware is event- or message-driven, the choice of abstraction (threads, async/await, actors) is still a design decision, and async/await is not obviously the global optimum.