I'm puzzled how anyone trusts ChatGPT for code

Hacker News Discussion ↗

Programmers are sharply divided over using ChatGPT and similar large language models to write code: some report 2–5x productivity gains from generating boilerplate, glue code, and examples, while others see brittle, syntactically broken output, especially in less common languages like Lisp. Many argue that LLM code should be treated like untrusted input from Stack Overflow or a junior developer—useful for ideas and scaffolding but always subject to human review, testing, and refactoring. Concerns center on over‑reliance by less experienced developers, the models’ opaque training limits, and the risk of degrading long‑term coding skills, even as others see these tools as an inevitable and powerful addition to the software engineer’s toolbox.

Trust vs. verification

  • Many argue “trust” is the wrong frame: you never truly trust code (from LLMs, coworkers, Stack Overflow, or your past self); you read, test, and review it.
  • Others counter that LLM code is uniquely untested and can be syntactically wrong, which they’d never accept from humans.
  • Several note that software engineering as a discipline is built on the assumption that you don’t trust code and must verify it.

How people use LLMs for code

  • Common uses: boilerplate/templates, glue code, simple scripts, API usage examples, syntax lookup, refactors, and performance tweaks.
  • Many treat LLMs like a very capable autocomplete or “well-informed intern”: good for drafts and ideas, not final code.
  • Some use them more to explain existing code (e.g., bash scripts, unfamiliar languages) than to generate new code.

Model quality, languages, and data coverage

  • Strong divide between experiences with GPT‑3.5 (often described as weak, especially for code) and GPT‑4 / specialized models (reported as much better).
  • Quality varies heavily by language and ecosystem: mainstream languages (Python, Java, C#, JS) fare better; Lisp, Autohotkey, Rust, SvelteKit, niche frameworks, and very modern APIs see more hallucinations and syntax errors.
  • LLMs struggle with highly repetitive, structurally demanding formats (e.g., Lisp parentheses), and with technologies that postdate the training cutoff.

Risks, failure modes, and misuse

  • Reported issues: hallucinated APIs, obsolete patterns, partial “framework” code with TODO comments, logical bugs, concurrency misdesigns, and mismatched brackets.
  • Concern that juniors and non-programmers may paste code they don’t understand, eroding skills and filling codebases with “AI-generated spaghetti debt.”
  • Some see LLM coding as “cargo cult programming automated,” especially when used for problems the user doesn’t grasp.
  • Others note LLMs allow much more bad code per unit time, amplifying existing problems with weak QA.

Perceived benefits

  • Many report 2–5× productivity for well-scoped tasks they already understand, mainly by reducing boilerplate and lookup time.
  • LLMs are said to be easier to “fact-check” on code than on prose because code can be run and tested.
  • Several emphasize that value comes when users spend more time reviewing, testing, and iterating than generating.