Political bias in AI: Where the AI models stand

Methodology and Limits of Measuring LLM Political Bias

• Several comments note that any left/right score depends heavily on how questions are written and how answers are coded.
• The site’s author (in-thread) describes using political science datasets (e.g., party and population surveys) and separate models to code stances, but concedes it is imperfect and anchored to a particular Overton window.
• Some propose higher‑dimensional or embedding-based approaches instead of fixed axes, but note these are harder to interpret for consumers.

Problems with Left/Right and the Political Compass

• Many see the political compass as oversimplified or “garbage,” arguing real politics spans many dimensions and that people often hold cross‑quadrant, internally inconsistent views.
• Others argue left/right and authoritarian/libertarian are culture- and time-dependent, so there is no objective “center” or “neutral” position.
• Attempts to locate global politicians (e.g., Macron, Xi, Putin, Albanese, Obama) are widely criticized as implausible or inconsistent with other compass-style analyses.

Model-Specific Observations

• Grok is perceived as having explicit right‑leaning tuning, with references to public statements about “fixing” it when it contradicts MAGA narratives.
• Gemini is seen as having swung from visibly overcorrected DEI-style behavior to appearing surprisingly “neutral” on this chart.
• DeepSeek is plotted as centrist, but multiple commenters say it censors or distorts sensitive Chinese topics (especially Tiananmen), so “center” is questioned.
• Some note all tested models map closest to the US Democratic Party, which itself is contested as either center-right or left depending on the comparator.

Prompting, Guardrails, and Apparent “Opinions”

• Several point out that models don’t hold stable ideologies; responses can be primed strongly by prompt framing.
• Others insist prompt engineering cannot override deeper training-set and guardrail biases, disputing how easily bias can be “created” via prompting.

Visualization and Presentation Critiques

• The main chart is accused of “chart crime”: low‑contrast dots and label offsets can visually exaggerate Grok’s position and obscure others.
• Inconsistencies between headline compass positions and more detailed breakdowns lower trust in the study.

Why Bias Matters (or Not)

• Some argue it’s important because LLMs are already influencing policy, resource allocation, and public understanding.
• Others say individuals shouldn’t—and mostly won’t—let an LLM decide their politics, so bias measurement is more academic than practical.