ChatGPT is biased against resumes with credentials that imply a disability

Scope of the Bias Finding

• Commenters see ChatGPT’s ableist behavior as unsurprising, paralleling known racial and gender biases in AI.
• Some test gender bias ad hoc (changing names/pronouns on identical CVs) and report no difference, but others point out this doesn’t test for cultural or name-based bias.
• Many emphasize that “don’t be biased” prompts may simply flip the bias or overweight disability signals rather than truly neutralize them.

AI as a Mirror of Human Bias

• Strong theme: LLMs reflect patterns in human data; if society and hiring are biased, the model will be too.
• Some argue this is “correct” in a statistical sense (it models reality), while others stress that “correct” for prediction is not “acceptable” for hiring or ethics.
• Several note this makes AI a powerful mirror of unconscious prejudice, but also a tempting way for organizations to outsource and deniably automate discrimination.

Technical and Methodological Limits

• Discussion of how instructions like “focus on disability justice” can swing rankings too far the other way, creating a new distortion.
• Multiple comments highlight that LLMs are black boxes, not transparent decision systems, and that deep learning makes detailed explanations of decisions essentially impossible.
• Others suggest limiting AI to narrow tasks (e.g., extracting structured data) and feeding that into explicit, auditable rules instead of free-form ranking.

Legal and Policy Concerns

• Debate over whether disparate impact (biased outcomes without explicit intent) is enough for legal liability; some say it’s hard to win in court, others note AI makes bias easier to empirically demonstrate.
• Concern that agencies and companies will adopt these tools quickly (e.g., child protection, housing, hiring) without understanding or controlling bias.

Disability, Productivity, and Hiring Practices

• Some question whether anti-discrimination laws “fight reality” if some disabilities affect productivity; others respond that reasonable accommodations often neutralize impact.
• Anecdotes: people omitting disability-related achievements or even “American Sign Language” from CVs due to suspected screening penalties.
• Several disabled or blind developers describe the dilemma of when to disclose and how little resumes convey about actual capability.