AI engineers claim new algorithm reduces AI power consumption by 95%

Original Article ↗ Hacker News Discussion ↗

AI researchers are touting a new “L-Mul” algorithm that replaces floating‑point multiplications with cheaper integer additions, claiming up to 95% energy savings for those operations and near‑lossless accuracy in some transformer workloads. Commenters note that related ideas—logarithmic number systems, fixed‑point arithmetic, and 1‑bit/low‑bit networks—have been explored for decades, and argue the true whole‑system savings will be far lower because most energy goes into moving data, not doing math. Many see potential for more efficient hardware and cheaper inference, but remain cautious about the paper’s methodology, clickbait headlines, and the likelihood that any efficiency gains will simply drive more and larger AI workloads rather than reduce total energy use.

What the algorithm is doing

  • Many commenters relate L‑Mul to classic math tricks: using log-space (log(x) + log(y)) or approximations like (1+a)(1+b) ≈ 1 + a + b when a,b are small.
  • It operates on low‑precision formats (e.g., 8‑bit floats), approximating floating‑point multiplication via integer adds on exponent/mantissa bits, plus small correction terms.
  • Several note this is conceptually close to logarithmic number systems, fixed‑point/Q‑format arithmetic, and long‑used DSP/FPGA techniques.

Claims about precision and energy savings

  • The paper claims L‑Mul can match or beat 8‑bit FP (e4m3, e5m2) in precision and save up to ~95% energy for the multiplication operation, ~80% for dot products.
  • Multiple commenters emphasize this 95% is per multiply, not overall model power; inference is often memory‑bandwidth‑dominated, so real end‑to‑end gains would be much smaller.
  • An approximate‑computing researcher argues:
    • Much power is in data movement, not the arithmetic itself.
    • The paper’s accuracy comparison ignores standard “round to nearest even” in baseline FP, making the claimed superiority “non‑sensical.”
    • Reported attention‑accuracy results lack detail on scaling/accumulation, so are hard to trust.

Practical applicability and hardware implications

  • Consensus: this won’t remove the need for GPUs; parallelism for large models is still essential. It mainly targets more efficient inference and possibly training on suitably designed hardware.
  • Current GPUs/CPUs are not optimized for this; specialized accelerators could, in principle, exploit it. Some expect any real benefit would prompt hardware vendors to respond.
  • Debate on vendor impact:
    • Some foresee “bad news” for Nvidia; others note Nvidia could simply implement the scheme in CUDA and still win.
    • AMD’s ROCm and data‑center GPUs are discussed as partial alternatives but still trailing Nvidia in ecosystem maturity.

Experimentation and limitations

  • A hand‑written AVX‑512 L‑Mul approximation applied directly to a FP16 Llama model produced gibberish outputs, suggesting models must be trained specifically for this arithmetic and/or only some layers can use it.
  • One implementation (BitNet/bitnet.cpp) shows promising CPU speedups (≈1.4–6×) and 55–82% CPU energy reductions for certain 1‑bit/1.58‑bit models, but that is a different, though related, line of work.

Meta: hype, impact, and rebound effects

  • Multiple comments criticize clickbait headlines and stress that results are theoretical or narrow; call for real, system‑level benchmarks.
  • Some invoke Jevons paradox: more efficient AI may simply lead to far more AI usage, not less total energy.
  • There is broader side‑discussion on whether LLMs’ productivity gains justify their energy and cost, with both strong advocates and skeptics represented.