Should you normalize RGB values by 255 or 256?

Original Article ↗ Hacker News Discussion ↗

How to map 8‑bit color values in the range 0–255 to floating‑point [0,1] turns out to be less obvious than simply “divide by 255.” Commenters contrast dividing by 255 vs 256 and mid‑tread vs mid‑riser quantizers, weighing tiny numerical biases against the need for exact 0.0 and 1.0, compatibility with existing graphics and image‑processing pipelines, and the behavior of real‑world hardware like ADCs and displays. Many conclude that using 0–255 mapped via division by 255 (with proper rounding and often dithering) is the most practical and least error‑prone choice, even if other mappings have theoretical appeal.

Quantization choice: 255 vs 256

  • Thread revolves around two mappings between 8‑bit [0,255] and float [0,1]: “mid‑tread” (divide by 255; 0→0, 255→1) vs “mid‑riser” (effectively scale by 256 with a 0.5 offset).
  • Some argue 255 is “obviously correct” because there are 255 intervals between 0 and 255; others say both quantizers are valid, just different bin/edge interpretations.
  • Several comments stress that mixing encode/decode schemes (e.g., 256 on one side, 255 on the other) is clearly wrong.
  • A detailed subthread clarifies the “bins vs edges” confusion, showing that you can define equal‑width bins in multiple ways and that “half‑width” end intervals are a trade‑off, not a hard error.

0 and 1 as special values

  • Many insist 0 must map to 0.0 and 255 to 1.0 to preserve additive/multiplicative identities and existing image‑processing assumptions (e.g., alpha, masking thresholds at exactly 0 and 1).
  • Others argue that in real scenes “zero luminance” rarely exists, so slight shifts (e.g., +0.5 schemes, 16–235 video ranges) are acceptable or even preferable.
  • Some note that breaking exact 0/1 causes subtle bugs in pipelines that implicitly rely on them for masking or fully‑opaque alpha.

Performance and implementation

  • One camp claims using >> 8 instead of /255 is dramatically faster in hot loops; others dispute this, pointing out:
    • Modern CPUs optimize division by a constant to multiplication.
    • FP mul and SIMD mul are very fast and often not the bottleneck.
  • There is disagreement on how much compilers actually auto‑vectorize and whether shifts meaningfully help; some suspect “10x faster” claims hide compilation issues.

Color space and correctness

  • Several commenters note that all these examples assume linear color; real 8‑bit graphics commonly use non‑linear spaces (e.g., sRGB), so the simple math is physically wrong anyway.
  • Some emphasize that the correct mapping is determined by the defined color space, OETF/EOTF, and file format, not by arbitrary choice.

Analogies and related domains

  • Analogies are drawn to ADC design, digital audio ranges (e.g., 16‑bit integer vs float), HDMI/NTSC ranges, and scientific computing’s node‑vs‑cell‑centered grids.
  • Dithering (often triangular) is recommended by some when converting float → 8‑bit to reduce banding, independent of 255 vs 256 choice.