U-M finds students with alphabetically lower-ranked names receive lower grades

Original Article ↗ Hacker News Discussion ↗

Alphabetical sorting of student names in learning management systems may be subtly skewing grades, with a University of Michigan study finding that those with surnames early in the alphabet receive slightly higher marks on average than those later in the list. Commenters suggest the effect likely stems from grader fatigue and evolving standards as they move through a stack of work, and debate whether randomizing or anonymizing submissions is the best mitigation. While the measured gap is small (around 0.6 points on a 100-point scale), many note that systematic biases of this kind can accumulate over many courses and mirror similar order effects seen in hiring, interviews, and even election ballots.

Observed Effect & Likely Causes

  • Many commenters who grade or have graded say the result matches their experience: order affects strictness.
  • Proposed mechanisms:
    • Fatigue and reduced attention over long grading sessions.
    • Evolving standards as graders see more answers, discover common mistakes, and adjust what counts as “minor” vs “major”.
    • Desire to maintain a monotonic rank of papers, causing “ratcheting” down of scores later in the stack.
  • Some note that a minority of graders seem to get stricter earlier and more lenient later, so individual patterns differ; the published result is seen as an aggregate.

Randomization, Fairness, and Bias

  • Strong support for shuffling grading order so no fixed group (e.g., late-alphabet surnames) is systematically disadvantaged.
  • Debate:
    • One camp: “Evenly distributing an unavoidable bias is effectively fixing it” at the cohort level.
    • Another camp: it merely spreads unfairness; some individuals will still be repeatedly unlucky.
  • Several suggest more structured schemes (e.g., alternating order, variance-reduction patterns) to reduce how often the same students are at the “harsh” end.

Grading Practices & Constraints

  • Many graders describe:
    • Using detailed rubrics, problem-by-problem grading, or multiple passes to keep standards consistent.
    • Time, pay, and workload constraints make two-pass grading or multi-grader systems hard, especially with large classes.
    • Online systems (Canvas, Gradescope, similar tools) often default to alphabetic order and strongly shape grading workflows.
  • Some institutions already pseudonymize exams with candidate numbers, but posters note that anonymity alone doesn’t fix ordering bias unless submissions are also shuffled.

Magnitude, Impact, and Skepticism

  • Effect size cited (~0.6 points on a 100-point scale between early and late surnames) is viewed by some as practically minor “noise,” by others as meaningful when aggregated across many courses or when a few high-stakes exams dominate the grade.
  • Several note the huge dataset makes small effects statistically detectable but worry about overinterpreting them.
  • A minority expect this could be another non-replicating social-science result; others counter that it is highly plausible and easy to re-test.

Broader Order Effects

  • Commenters connect this to:
    • Interview and hiring order.
    • Ballot ordering in elections.
    • Classroom seating, roll call, and who gets teacher attention.
  • General consensus: order effects are real, subtle, and often baked into tools and procedures without much thought.