Two's complement integers with only sign bit set should be a trap representation

Original Article ↗ Hacker News Discussion ↗

Treating the lowest two’s-complement integer value (the pattern with only the sign bit set, e.g. INT_MIN) as a trap or NaN-like sentinel could make integer overflows and optional integers safer and easier to handle, but would break existing code and require hardware or compiler support. Commenters weigh the appeal of symmetric ranges and built‑in “missing” values against the realities of performance, FFI interoperability, and decades of software that assume simple wrapping arithmetic. Many argue that better tooling (sanitizers, explicit bigints, or new integer types with reserved “niche” values) is preferable to redefining core integer semantics in C, C++ and common ISAs.

Proposal and Motivation

  • Thread centers on making the “all-sign-bit” two’s complement value (e.g., INT_MIN) a special case:
    • Either a trap representation (using it causes a fault/exception).
    • Or a NaN‑like sentinel used for “missing”/invalid integers.
  • Motivations discussed:
    • Symmetric integer ranges (−N..+N rather than −N−1..+N).
    • Cheaper optional<int> / nullable-int representations using that bit pattern.
    • Eliminating corner cases like abs(INT_MIN) overflow in C.

Hardware, Performance, and Implementation

  • Many argue this only really makes sense with hardware support; software checks after every op are seen as too costly for low-level languages.
  • Others point out dynamic or high-level languages (Python, Lisp, Swift, R) already pay similar costs (bignums, bounds checks, sentinels).
  • DSPs and some ISAs support alternative arithmetic modes (saturating arithmetic, overflow flags), which could be leveraged instead.

Correctness, UB, and Language Semantics

  • Intense debate over C/C++ signed overflow being undefined:
    • Some want wrap (-fwrapv) or traps (-ftrapv) as defaults.
    • Others stress that relying on UB for optimization has created real security bugs and difficult-to-debug behavior.
  • Distinction drawn between:
    • A trap representation (just reading/writing it is UB in C).
    • A NaN‑like value with defined propagation semantics.
  • Several note that retrofitting this into C would break existing, currently-correct code and FFI expectations.

Alternatives: Bignums, Saturation, Sentinels

  • Alternatives discussed:
    • Arbitrary-precision integers as the default (Lisp, Scheme, Python).
    • Range/interval types where out-of-range values encode “missing”.
    • Saturating arithmetic instructions or library ops.
    • Language- or library-level niche types (non-zero, 63‑bit ints, Rust “niche” optimization, OCaml’s tagged ints, R’s NA integer).

Safety vs Availability

  • Some prefer crashing/trapping on overflow to avoid silent data corruption.
  • Others, especially in safety‑ or mission‑critical domains, prioritize continued operation even with wrong values over process death.
  • Consensus: desirable behavior is highly domain‑dependent; a single hardware or language default won’t satisfy all use cases.