Testing how hard it is to cheat with ChatGPT in interviews
Many engineers and hiring managers are now grappling with how easily candidates can use ChatGPT and similar tools to pass technical interviews, especially LeetCode-style coding tests. Opinions split between those who see undisclosed LLM use as cheating in a “closed book” setting and those who argue interviews should mirror real work, where Google, Stack Overflow, and AI assistants are standard tools. A recurring theme is that companies will need to redesign interviews—focusing more on reasoning, code review, and real-world tasks, and either explicitly permitting and observing AI use or crafting questions that test understanding beyond what an LLM can reliably supply.
Role of tools (ChatGPT, Google, Stack Overflow) in interviews
- Many argue interviews should mirror real work, where looking things up and using LLMs is normal.
- Others say interviews are deliberately constrained “exams” to measure underlying competence, not ability to query tools.
- Some companies explicitly allow Google/LLMs and judge how candidates use them; others treat any undisclosed tool use as cheating.
- A recurring suggestion: if a tool can trivially solve your question, the question is poorly designed for assessment.
Open-book vs closed-book expectations and honesty
- Several commenters favor “open book” interviews: candidates may use docs, web search, even LLMs, but this must be explicit.
- Strong emphasis that if the interview is framed as closed-book, secretly using ChatGPT is seen as dishonest, regardless of competence.
- Some see this as over‑moralizing, arguing that broken interview processes incentivize “gaming” and that good performers may still resort to tools.
- Others stress that willingness to lie in an interview is itself a disqualifying signal about trustworthiness.
Designing LLM‑resistant or LLM‑aware interviews
- Suggestions for LLM‑resistant tasks:
- Custom DS&A questions not present online.
- Debugging or modifying nontrivial, possibly buggy code (including LLM‑generated code).
- Realistic tasks in an existing codebase, then discussing trade‑offs and design.
- Questions whose “trick” is in small details that LLMs often gloss over.
- Counterview: instead of fighting LLMs, explicitly integrate them and evaluate how candidates prompt, critique, and correct model output.
Leetcode, fundamentals, and what interviews should measure
- Some maintain DS&A / Big‑O questions, done reasonably, are still the best mass‑filter against “can’t code at all.”
- Others find this demeaning for experienced engineers and largely unrelated to typical CRUD‑style work.
- Many propose mixing: basic coding checks plus design, code review, and “shop talk” to probe depth, reasoning, and experience.
Remote vs in‑person and cheating detection
- Interviewers report obvious signs of cheating in remote interviews (reading tone, screen reflections, “select all”, odd pauses), but acknowledge selection bias: the subtle cheaters go unnoticed.
- Some foresee more in‑person or tightly monitored (screen‑shared, single‑monitor, recorded) sessions as a response.
- There is skepticism that this arms race is sustainable; others see it as inevitable.