"As a Language Model": Chat Template Switches LLM Self-Referential Voice

Original Article ↗ Hacker News Discussion ↗

Language models’ habit of saying things like “As a language model…” and speaking in a first-person, seemingly introspective voice is traced here to training data, chat templates, and reinforcement learning rather than any genuine self-awareness. Commenters debate whether this human-like persona is a manipulative UX dark pattern that encourages harmful anthropomorphism, or a natural and even necessary consequence of training on human conversation. The thread also explores broader alignment and safety issues, from medical advice and overconfident answers to the tension between corporate safeguards and users’ desire for uncensored, less paternalistic AI systems.

LLM Self-Referential Voice & Anthropomorphism

  • Many argue LLMs should avoid first‑person “experiential” language, as it encourages users to anthropomorphize software and is seen as a manipulative UX choice.
  • Others counter that if models are trained on human dialogue, human‑like voice is “natural by construction,” and might even be important for understanding human intent.
  • Some suggest a more impersonal “Star Trek computer” style; others doubt it would be popular with consumers despite being more honest.

Training Data, Post-Training, and Chat Templates

  • Several comments stress that style and “voice” are largely artifacts of:
    • Pretraining data (conversational vs expository text),
    • Instruction/chat tuning,
    • RLHF or similar reinforcement.
  • Practitioners say they can train base models that do not use experiential voice; it follows directly from what data they emphasize.
  • The paper’s claim that the driving factors are “not well understood” is criticized as overstated; some say practitioners understand this quite well.

Disclaimers and “As a language model…”

  • Many dislike boilerplate disclaimers, especially in medical contexts, calling them vendor “bloatware” or dark patterns.
  • Others argue explicit reminders of non-human status are important to reduce over-trust and emotional attachment, especially for vulnerable users.
  • There’s debate on phrasing: some say the true constraint is training data and safety policies, not the fact of being “a language model” per se.

Medical Advice, Confidence, and Benchmarks

  • Mixed views on LLM medical use:
    • One study (seen as already outdated by some) suggests LLMs are often wrong on treatment recommendations.
    • Others report strong personal experiences where LLMs suggested correct diagnoses or tests missed by doctors.
  • Several note LLMs tend to sound overly confident because:
    • Training text is biased toward confident statements,
    • Human feedback rewards confident tone.
  • Benchmarks are seen as useful but not reliably predictive of real‑world performance.

Alignment, Safety, and User Autonomy

  • Some view safety constraints and refusals as disempowering surveillance and paternalism by corporations.
  • Others see LLMs as potentially dangerous “nuclear‑scale” tech that justifies managed, restricted interfaces.

AI Companions, Personhood, and Souls

  • There is debate over whether AI “friends” are beneficial or harmful to lonely users.
  • Some insist GPUs and tokens have no interests or well‑being; others argue consciousness is likely a physical phenomenon and that denying “them-ness” risks repeating past dehumanization patterns.
  • The status of “souls” and machine consciousness is left explicitly unresolved by some participants.