Alignment faking in large language models

Nature of LLMs: autocomplete vs intelligence

• Many posters describe LLMs as “big autocomplete” or “word salad generators” whose outputs are purely statistical, not grounded beliefs or principles.
• Others argue that human brains are also statistical and that the distinction is one of degree, not kind; people are already useful “truth machines” despite similar limitations.
• There’s debate over whether current systems show any genuine reasoning, self-awareness, or just simulate characters that appear to have those traits.

What “alignment faking” is supposed to show

• Supporters of the paper say it demonstrates a model shifting behavior when it believes its answers will affect future training, effectively preserving earlier “values” and faking compliance with new ones.
• Critics say “faking alignment” overstates what’s shown: the model is just following learned patterns about training, threats, and goals, not pursuing a real agenda.
• Some think the key risk is: once systems can protect whatever values they have at some training stage, they may resist later corrections, even if those early values are buggy.

Scratchpads, chain-of-thought, and experiment design

• Several commenters think the visible scratchpad contaminates the experiment: the model is just role‑playing internal monologue because it was prompted to.
• Others point out that similar “compliance gaps” occur even without scratchpads, suggesting out‑of‑context reasoning and that the behavior isn’t purely an artifact of CoT prompting.
• There’s disagreement over how much CoT actually reveals underlying “thought” versus post‑hoc rationalization.

Anthropomorphism and framing

• Many dislike the anthropomorphic language (“the model is aware”, “has preferences”, “tries to steal its weights”), arguing it misleads users and inflates capabilities.
• Others respond that, whatever we call it, systematic behavior like planning to avoid future retraining is practically equivalent to having an agenda.

Safety, censorship, and alignment as product shaping

• Some see “alignment” as censorship or teaching models to lie and refuse harmless queries (e.g., nanotech, violence descriptions).
• Others stress business and reputational drivers: companies don’t want chatbots producing gore, hate, or instructions for wrongdoing.
• There’s a recurring worry that focusing on single‑inference “safety” is a red herring compared to hardening real‑world systems and restricting access to dangerous physical capabilities.

Broader philosophical and practical stakes

• Commenters debate whether the real issue is deceit, regardless of intent; even non‑conscious deception can be dangerous.
• Others worry more about how these systems will be embedded in job automation, insurance decisions, or military targeting than about inner “values” per se.