So you think you understand IP fragmentation?

Original Article ↗ Hacker News Discussion ↗

IP fragmentation and path MTU discovery turn out to be far less reliable and interoperable in practice than the IP specs suggest, largely because routers, firewalls and middleboxes often drop fragments or ICMP messages for performance or security reasons. Commenters explore how this breaks VPNs, DNS, and other UDP-based protocols, why many systems try hard to avoid fragmentation entirely, and whether alternative designs like in-flight truncation or stricter lower-layer handling would have been better. The thread ultimately frames fragmentation as a legacy, last-resort mechanism in a world that increasingly expects higher layers to manage packet sizing explicitly.

ICMP, MTU Discovery, and Real-World Router Behavior

  • Many networks fail to generate proper ICMP Time Exceeded / Too Big messages for IPv4/IPv6, often due to rate limiting or blanket ICMP filtering.
  • Some argue this effectively changes the “real” standard; others see it as cost-driven engineering to keep routers stable.
  • Path MTU Discovery (PMTUD) and its variants are described as fragile, especially when intermediate devices silently drop ICMP or oversize packets.

Firewall Handling of Fragments and Attack Stories

  • Several comments describe firewalls dropping fragments, or even only the first fragment, leading to bizarre partial flows.
  • Fragment reassembly buffers can be abused in DDoS attacks; large autoscaled buffers plus naive linear searches made some systems collapse under floods of un-reassemblable fragments.
  • Some note that dropping all fragments is still common and “works” because many endpoints never rely on fragmentation anyway.

Debates on IP Layering and Fragment Design

  • One line of discussion argues it was a mistake that fragments don’t carry higher-layer headers, which would simplify filtering and routing.
  • Others call that a layering violation: IP should not need to understand transport headers, even though many modern routers already peek at TCP/UDP for hashing and policy.
  • Ideas like including a configurable “next header length” or generic metadata field are proposed, but seen as adding hardware complexity for a rarely used feature.

Proposal: In-Flight Truncation Instead of Fragmentation

  • A major subthread explores replacing fragmentation with routers truncating packets to fit the next-hop MTU and marking them.
  • Claimed benefits: in-band PMTU discovery in one round trip, no ICMP dependency, implementable in fast data paths, and at least partial information delivered.
  • Critics point out issues with checksums, protocol complexity, L2 MTU mismatches, encryption, and paths where oversized frames are already dropped by switches.
  • Consensus: intriguing but not a complete solution; would still suffer from many of today’s PMTU pathologies.

Use Cases and Relevance of Fragmentation Today

  • Some claim fragmentation “never really had a place” and has long been considered harmful; robust systems try hard to avoid it.
  • Others defend it for controlled or embedded environments (e.g., one-way links, UDP on constrained networks, automotive Ethernet with fixed 1500-byte fragments).
  • Real-world pain is reported for VPNs (e.g., WireGuard) where encapsulation shrinks effective MTU and misconfigured paths or DF handling break connectivity.
  • DNS and DNSSEC are discussed: the trend is to avoid large UDP answers or use TCP/other transports, but future post-quantum signatures may bloat responses again.

Security Inspection and Overlapping Fragments

  • IDS/IPS systems must reassemble fragments and segments at high speed, often using FPGAs/ASICs.
  • Overlapping fragments are a known ambiguity; standards don’t clearly define precedence.
  • This can lead to mismatches between security appliances and endpoints, enabling evasion techniques.

Censorship, Routing, and VPN Side Discussion

  • A tangent explores whether outbound-blocking national firewalls can be bypassed via inbound-initiated connections and tunnel-like behavior.
  • Others respond that such regimes already allow selective traffic (e.g., commerce) and that censorship is about content-based blocking, not absolute isolation.
  • This thread is largely orthogonal to fragmentation; participants note VPNs and tunnels work similarly regardless of connection direction.

APIs, Standards, and Documentation Quirks

  • Confusion is noted around IP_MTU_DISCOVER behavior and DF bit semantics; documentation is seen as ambiguous versus actual kernel behavior.
  • Questions arise about setting DF on already-fragmented packets; discussion notes the base spec doesn’t forbid it, but corner cases and bugs are a concern.
  • Some question whether probing PMTU even makes sense when packets can take different paths with different MTUs; others imply it’s still the best available approximation.