Asynchronous IO: the next billion-dollar mistake?
Asynchronous I/O and `async/await` are under scrutiny as developers weigh their complexity and “function colouring” against traditional threaded, synchronous models. Commenters debate whether decades of effort should have gone into making OS threads cheaper and more scalable instead, or whether async is simply a better fit for inherently concurrent hardware and workloads like networking and high‑latency I/O. Many see a middle path in language‑level abstractions such as green threads, virtual threads, and structured concurrency that hide low-level async details while preserving performance.
Scope of the Discussion
- Debate centers on whether async I/O is a mistaken direction compared to making OS threads vastly more efficient.
- Comments split between: “async is essential and natural” vs. “threads + better runtimes are sufficient and simpler.”
Can OS Threads Be Made “Good Enough”?
- Some argue physics and CPU design limit how cheap kernel threads and context switches can get (stacks, caches, security mitigations, address-space model).
- Others question whether we are actually at the limits, suggesting claims of “threads are as fast as they can be” are unproven.
- Memory overhead of per-thread stacks is seen as a hard constraint for hundreds of thousands of threads.
What Async I/O Actually Buys You
- One view: async is mostly about efficiency and high-concurrency I/O (many idle connections, batching, fewer syscalls).
- Another: async/await exposes time as a first-class concept, making composition, cancellation, and timeouts easier.
- Counterpoint: cancellations are tricky in both models, especially with side effects (e.g., partial writes); true cancellation often degrades to “deadline hints” plus cleanup logic.
Complexity, Ergonomics, and “Function Colouring”
- Some report large async systems becoming complex, ending up mixing async, thread pools, and blocking calls—getting “the worst of both.”
- Others find thread-based concurrency far harder (deadlocks, races, lock ordering), and see async/await as easier for typical developers.
- Function colouring (async vs sync call chains) is widely seen as painful; retrofitting sync code to async is described as a multi‑year effort in real projects.
Kernel and OS Realities (Linux, Windows, io_uring, etc.)
- Several point out that most hardware and kernels are inherently event-driven; many “sync” calls are async under the hood plus a blocking wait.
- Linux historically exposed mostly synchronous POSIX APIs; newer mechanisms (AIO, io_uring, AF_XDP) provide real async I/O, including for files, though adoption in userland is still uneven.
- NT’s I/O model is cited as fully async from the start (IOCP, Registered I/O), with newer APIs (IoRing) paralleling Linux’s io_uring to reduce copies and syscalls.
- There is disagreement over how much the article underestimates existing async file I/O capabilities.
Alternative Models: Green Threads, Actors, Structured Concurrency
- Many suggest the “parallel universe” already exists in language runtimes: Go’s goroutines, Erlang’s processes, Java virtual threads, user‑space threads, actors, CSP, and structured concurrency.
- These models often hide async I/O behind a synchronous-looking API, while relying on kernel async primitives underneath.
- Some worry pushing “lightweight threads in the kernel” would freeze experimentation at the OS level and slow progress.
Hardware Nature and the “Inherently Async” Argument
- Several argue the world is fundamentally concurrent: NIC queues, DMA, storage, and even RAM distance make I/O inherently asynchronous.
- Others respond that while hardware is event- or message-driven, the choice of abstraction (threads, async/await, actors) is still a design decision, and async/await is not obviously the global optimum.