LLMs Are Not a Higher Level of Abstraction

Original Article ↗ Hacker News Discussion ↗

LLMs are often marketed as a new “higher level of abstraction” for programming, but many engineers argue this analogy breaks down because LLMs don’t preserve semantics or determinism the way compilers and traditional abstractions do. Commenters highlight that natural-language prompts are inherently ambiguous, LLM outputs can be wrong in unpredictable ways with no clear blast radius, and current systems lack the guarantees that make deeper layers of the software stack safely composable. Others counter that abstractions have always traded away control and precision, and see LLMs instead as powerful but leaky tools for delegating cognitive load and generating or transforming code, rather than a true layer in the abstraction stack.

Determinism vs. Stochasticity

  • Strong disagreement on whether LLMs are “non‑deterministic”:
    • One side: the model plus sampler is mathematically deterministic; with fixed weights, sampler, seed, hardware/config, you get the same output.
    • Other side: real deployments (GPU batching, multi‑node inference, floating‑point ordering, provider settings) make outputs practically non‑reproducible.
  • Several note that randomness is usually a design choice (sampling, temperature), not inherent to the model, but others point out that GPU math and batching make full determinism hard in practice.
  • Some argue the real distinction isn’t determinism but that LLMs are inductive/probabilistic systems rather than deductive ones.

Are LLMs a Higher-Level Abstraction?

  • Many say LLMs are not a new abstraction layer like going from machine code → C → Python:
    • Traditional abstractions have well‑defined, deterministic semantics; you can always “go down a layer” and understand the mapping.
    • Natural language specs are vague; LLMs don’t preserve semantics reliably.
  • Others argue abstraction simply means “hiding detail to manage complexity,” which LLMs do by letting you specify intent in natural language and delegating implementation.
  • Some propose LLMs are better seen as:
    • Program synthesizers or probabilistic assistants that sit outside the usual stack.
    • Organizational‑style abstractions, similar to delegating work to a coworker.

Reliability, Errors, and Blast Radius

  • A central worry is “blast radius”:
    • Network packets can fail in contained, well‑specified ways; systems can react with timeouts/retries.
    • LLM reasoning errors can silently produce plausible but harmful code or actions (e.g., deleting data, mispricing products).
  • Others counter that blast radius is controllable via permissions, guardrails, and careful system design; logic failures are not unique to LLMs.

Analogies and Misconceptions

  • Heated debate over comparing LLMs to compilers:
    • Critics: compilers have exact, fixable semantics; LLMs do not.
    • Defenders: compilers and build systems also show variation across versions, flags, and platforms; abstraction has always involved giving up some control.
  • Other analogies: TCP over noisy channels, grocery stores vs. growing food, coworkers and contractors, human non‑determinism.

Usage Patterns, Benefits, and Risks

  • Some engineers report heavy use as coding assistants to offload boilerplate and free mental bandwidth for design.
  • Others deliberately avoid deep integration to prevent skill atrophy and over‑reliance, and highlight a “gambling/dopamine” dynamic around prompting and token usage.
  • Overall split: some see LLMs as powerful high‑level aids despite leakiness; others see them as unreliable tools unsuited to being core abstractions.