I'm puzzled how anyone trusts ChatGPT for code

Hacker News Discussion ↗

Trust vs. verification

  • Many argue “trust” is the wrong frame: you never truly trust code (from LLMs, coworkers, Stack Overflow, or your past self); you read, test, and review it.
  • Others counter that LLM code is uniquely untested and can be syntactically wrong, which they’d never accept from humans.
  • Several note that software engineering as a discipline is built on the assumption that you don’t trust code and must verify it.

How people use LLMs for code

  • Common uses: boilerplate/templates, glue code, simple scripts, API usage examples, syntax lookup, refactors, and performance tweaks.
  • Many treat LLMs like a very capable autocomplete or “well-informed intern”: good for drafts and ideas, not final code.
  • Some use them more to explain existing code (e.g., bash scripts, unfamiliar languages) than to generate new code.

Model quality, languages, and data coverage

  • Strong divide between experiences with GPT‑3.5 (often described as weak, especially for code) and GPT‑4 / specialized models (reported as much better).
  • Quality varies heavily by language and ecosystem: mainstream languages (Python, Java, C#, JS) fare better; Lisp, Autohotkey, Rust, SvelteKit, niche frameworks, and very modern APIs see more hallucinations and syntax errors.
  • LLMs struggle with highly repetitive, structurally demanding formats (e.g., Lisp parentheses), and with technologies that postdate the training cutoff.

Risks, failure modes, and misuse

  • Reported issues: hallucinated APIs, obsolete patterns, partial “framework” code with TODO comments, logical bugs, concurrency misdesigns, and mismatched brackets.
  • Concern that juniors and non-programmers may paste code they don’t understand, eroding skills and filling codebases with “AI-generated spaghetti debt.”
  • Some see LLM coding as “cargo cult programming automated,” especially when used for problems the user doesn’t grasp.
  • Others note LLMs allow much more bad code per unit time, amplifying existing problems with weak QA.

Perceived benefits

  • Many report 2–5× productivity for well-scoped tasks they already understand, mainly by reducing boilerplate and lookup time.
  • LLMs are said to be easier to “fact-check” on code than on prose because code can be run and tested.
  • Several emphasize that value comes when users spend more time reviewing, testing, and iterating than generating.