Ai2’s FairStress shows AI models can be swayed unevenly by identity

Across open and frontier models, including Claude and the GPT family, Ai2 found little difference when no identity was given, but growing imbalance as identity became explicit: models dropped a stronger candidate more readily when the switch favoured a minority-coded one. Their explanations differed too, stressing diversity for some candidates and evidence for others. Most debiasing methods Ai2 tested didn’t fix this; asking the model to reconsider with identities swapped came closest.

Why it matters

Fairness checks that only compare final answers can miss how easily a model’s judgement bends under pressure. Ai2 also found that a widely used older benchmark pointed the other way, partly because of how it asks questions. For anyone using AI in hiring, lending or healthcare, consistency under pressure is something to measure, not assume.