Should you normalize RGB values by 255 or 256?

Original Article ↗ Hacker News Discussion ↗

Quantization choice: 255 vs 256

  • Thread revolves around two mappings between 8‑bit [0,255] and float [0,1]: “mid‑tread” (divide by 255; 0→0, 255→1) vs “mid‑riser” (effectively scale by 256 with a 0.5 offset).
  • Some argue 255 is “obviously correct” because there are 255 intervals between 0 and 255; others say both quantizers are valid, just different bin/edge interpretations.
  • Several comments stress that mixing encode/decode schemes (e.g., 256 on one side, 255 on the other) is clearly wrong.
  • A detailed subthread clarifies the “bins vs edges” confusion, showing that you can define equal‑width bins in multiple ways and that “half‑width” end intervals are a trade‑off, not a hard error.

0 and 1 as special values

  • Many insist 0 must map to 0.0 and 255 to 1.0 to preserve additive/multiplicative identities and existing image‑processing assumptions (e.g., alpha, masking thresholds at exactly 0 and 1).
  • Others argue that in real scenes “zero luminance” rarely exists, so slight shifts (e.g., +0.5 schemes, 16–235 video ranges) are acceptable or even preferable.
  • Some note that breaking exact 0/1 causes subtle bugs in pipelines that implicitly rely on them for masking or fully‑opaque alpha.

Performance and implementation

  • One camp claims using >> 8 instead of /255 is dramatically faster in hot loops; others dispute this, pointing out:
    • Modern CPUs optimize division by a constant to multiplication.
    • FP mul and SIMD mul are very fast and often not the bottleneck.
  • There is disagreement on how much compilers actually auto‑vectorize and whether shifts meaningfully help; some suspect “10x faster” claims hide compilation issues.

Color space and correctness

  • Several commenters note that all these examples assume linear color; real 8‑bit graphics commonly use non‑linear spaces (e.g., sRGB), so the simple math is physically wrong anyway.
  • Some emphasize that the correct mapping is determined by the defined color space, OETF/EOTF, and file format, not by arbitrary choice.

Analogies and related domains

  • Analogies are drawn to ADC design, digital audio ranges (e.g., 16‑bit integer vs float), HDMI/NTSC ranges, and scientific computing’s node‑vs‑cell‑centered grids.
  • Dithering (often triangular) is recommended by some when converting float → 8‑bit to reduce banding, independent of 255 vs 256 choice.