It's always TCP_NODELAY

Original Article ↗ Hacker News Discussion ↗

TCP’s Nagle algorithm and the TCP_NODELAY option are scrutinized as sources of surprising latency in modern applications, especially chatty RPC systems, games, and interactive shells. Commenters argue that the historical goal of reducing tiny packets on low-bandwidth links no longer justifies the default behavior, since it now often penalizes latency-sensitive workloads and is hard to reason about when combined with delayed ACKs. Many advocate making TCP_NODELAY (or a similar low-latency mode) the default, pushing batching and buffering decisions into applications or higher-level protocols like QUIC, while retaining opt‑in mechanisms for throughput-optimized batching.

Nagle’s Algorithm vs. TCP_NODELAY

  • Many commenters report latency bugs repeatedly tracing back to Nagle’s algorithm and/or delayed ACKs.
  • Strong sentiment that modern defaults should favor TCP_NODELAY, with a separate opt‑in “delay” mode.
  • Some argue Nagle still has value for conserving packets and bandwidth, especially on constrained or congested links.

Application Behavior and Tiny Writes

  • Core failure mode: apps doing many small write() calls (e.g., HTTP headers line by line), or assuming one recv() equals one application message.
  • Some see this as bad application design that should be fixed with buffering or writev().
  • Others note that libraries and legacy code make it hard or impossible to fix all offenders; kernel heuristics are viewed as a pragmatic safety net.

Delayed ACKs and Pathological Interactions

  • The worst problems arise from the combination of Nagle and delayed ACKs, especially for send/send/receive patterns (e.g., RPCs, games, chatty protocols).
  • Disabling delayed ACKs (TCP_QUICKACK, route quickack 1) is suggested, but APIs are awkward and per‑platform.

Workarounds and Tuning Options

  • Techniques mentioned: TCP_NODELAY, TCP_CORK/autocorking, user‑space buffering, vectored I/O (writev), and per‑socket keepalives.
  • For closed-source apps, suggested hacks include LD_PRELOAD shims, ptrace, eBPF, and inserting socat as a local proxy.

Defaults in Languages and Stacks

  • Some modern runtimes (e.g., Go, Node.js HTTP, curl, OpenSSH) reportedly enable TCP_NODELAY by default.
  • Debate over whether the “system ABI” should be libc vs. direct syscalls; this affects how LD_PRELOAD-style interception works.

Alternatives: UDP, QUIC, and Datacenter Protocols

  • UDP is often used for real‑time video/games, with app‑level reliability/heartbeats, but people warn against reinventing TCP/SCTP badly.
  • QUIC fixes head‑of‑line blocking and lives in user space, but still faces similar batching/ACK tradeoffs; some report significant perf issues in practice.
  • Inside datacenters, there is movement toward reliable datagram protocols and custom transports for performance.

Operational Anecdotes

  • Multiple stories of dramatic latency improvements after flipping TCP_NODELAY.
  • Other “it wasn’t Nagle” tales: DNS misconfiguration, IPv6/localhost resolution quirks, hardware faults, DHCP issues, and odd router failures.