It's always TCP_NODELAY

Original Article ↗ Hacker News Discussion ↗

Nagle’s Algorithm vs. TCP_NODELAY

  • Many commenters report latency bugs repeatedly tracing back to Nagle’s algorithm and/or delayed ACKs.
  • Strong sentiment that modern defaults should favor TCP_NODELAY, with a separate opt‑in “delay” mode.
  • Some argue Nagle still has value for conserving packets and bandwidth, especially on constrained or congested links.

Application Behavior and Tiny Writes

  • Core failure mode: apps doing many small write() calls (e.g., HTTP headers line by line), or assuming one recv() equals one application message.
  • Some see this as bad application design that should be fixed with buffering or writev().
  • Others note that libraries and legacy code make it hard or impossible to fix all offenders; kernel heuristics are viewed as a pragmatic safety net.

Delayed ACKs and Pathological Interactions

  • The worst problems arise from the combination of Nagle and delayed ACKs, especially for send/send/receive patterns (e.g., RPCs, games, chatty protocols).
  • Disabling delayed ACKs (TCP_QUICKACK, route quickack 1) is suggested, but APIs are awkward and per‑platform.

Workarounds and Tuning Options

  • Techniques mentioned: TCP_NODELAY, TCP_CORK/autocorking, user‑space buffering, vectored I/O (writev), and per‑socket keepalives.
  • For closed-source apps, suggested hacks include LD_PRELOAD shims, ptrace, eBPF, and inserting socat as a local proxy.

Defaults in Languages and Stacks

  • Some modern runtimes (e.g., Go, Node.js HTTP, curl, OpenSSH) reportedly enable TCP_NODELAY by default.
  • Debate over whether the “system ABI” should be libc vs. direct syscalls; this affects how LD_PRELOAD-style interception works.

Alternatives: UDP, QUIC, and Datacenter Protocols

  • UDP is often used for real‑time video/games, with app‑level reliability/heartbeats, but people warn against reinventing TCP/SCTP badly.
  • QUIC fixes head‑of‑line blocking and lives in user space, but still faces similar batching/ACK tradeoffs; some report significant perf issues in practice.
  • Inside datacenters, there is movement toward reliable datagram protocols and custom transports for performance.

Operational Anecdotes

  • Multiple stories of dramatic latency improvements after flipping TCP_NODELAY.
  • Other “it wasn’t Nagle” tales: DNS misconfiguration, IPv6/localhost resolution quirks, hardware faults, DHCP issues, and odd router failures.