Floats Are Weird

Original Article ↗ Hacker News Discussion ↗

Floating-point arithmetic often surprises programmers because it approximates real numbers and can lose precision in common operations like subtracting nearly equal values, as illustrated by examples involving `exp(x) - 1` and techniques to “cancel out” numerical noise. Commenters explore numerical-analysis concepts such as catastrophic cancellation, stable reformulations (e.g., using `expm1` or alternative algebraic forms), and tools like Kahan summation or Herbie that help rewrite formulas for better accuracy. A major theme is when floats are inappropriate—especially for money and accounting—and how education, libraries, and alternative numeric types (decimal, fixed point, big integers) are crucial to using numerical computation safely.

Float behavior & catastrophic cancellation

  • Many comments stress floats aren’t “weird” so much as limited-precision approximations of reals; subtracting nearly equal values inevitably loses significant digits.
  • The article’s “magic” example is discussed: the function g(x) = (exp(x)-1)/log(exp(x)) largely cancels shared rounding noise between numerator and denominator, while f(x) = (exp(x)-1)/x does not.
  • This cancellation works only over a range; for very small x it breaks (e.g., division by zero when log(exp(x)) rounds to 0).
  • Several note this is about algorithmic stability, not just “floats being bad.”

Numerical techniques & tools

  • Recommended: use expm1(x) for exp(x)-1 near zero; Python’s math.expm1 is highlighted, including its Kahan-inspired implementation.
  • General rules of thumb:
    • Avoid subtracting almost equal numbers.
    • Avoid adding numbers of vastly different magnitudes.
    • For sums, use Kahan or pairwise summation, or add smaller-magnitude terms first.
    • Compute offsets from 1 (e.g., expm1) when values are close to 1.
  • Herbie is mentioned as a tool that suggests more stable reformulations.
  • Some libraries are noted (e.g., decimal.js), and that many languages expose IEEE-754 and C libm functions, sometimes with their own fixes for known inaccuracies.

Floats vs money and business calculations

  • Strong view: never use floats for accounting; use integers or decimal types to avoid rounding surprises on invoices, pricing, and customer-visible totals.
  • Counterview: for many financial models (interest, bonds, options, risk simulations) floats are the right tool; uncertainty in inputs dwarfs cent-level rounding, and integer/fixed-point approaches become complex and fragile.
  • Middle ground:
    • Use exact integer/decimal formats for storage, settlement, and customer-facing arithmetic.
    • Use floats in analytical/forecasting layers where exact cents are less critical.
  • Decimal floating point (IEEE decimal, IBM hardware) and fixed-point “integer cents” are discussed as alternatives, each with trade-offs.

Education, numerics as a discipline

  • Several argue that misunderstandings stem from poor education: students learn reals, then are handed floats without being taught representation, rounding, or stability.
  • Numerical analysis is highlighted as a full discipline; algorithm stability and error analysis matter as much as the choice of number format.