Side-by-side Comparison Amplifies Dialect Bias in Language Models
Kritee Kondapally, Claire J. Smerdon, Pooja C. Patel, Ogheneyoma Akoni, Jevon Torres, Jaspreet Ranjit, Matthew Finlayson, Swabha Swayamdipta
Why It Matters
What makes this one worth your time
Understanding and mitigating dialect bias in language models is crucial for ensuring fairness in AI applications, especially in high-stakes decision-making contexts.
Side-by-side comparisons of dialects in language models reveal amplified biases.
Summary
The paper quantifies covert dialect bias in language models by comparing the associations of stereotypical traits with tweets in Standard American English and African-American Vernacular English, revealing that side-by-side comparisons exacerbate bias.
Key contributions
- Quantification of covert dialect bias in language models through the evaluation of intent-equivalent tweets.
- Demonstration of the exacerbation of bias in contrastive settings compared to isolated evaluations.
- Evaluation of counterfactual fairness finetuning as a potential mitigation strategy.
Notable insights
- The exacerbation of bias in side-by-side comparisons suggests that existing evaluation methods may not adequately capture the severity of dialect bias.
- Counterfactual fairness finetuning shows promise but lacks consistency across different traits, highlighting the complexity of bias mitigation.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2605.24384v2 Announce Type: replace-cross Abstract: Language models (LMs) can exhibit biases based on variations in their dialects, even in the absence of a dialect label, a behavior known as covert dialect bias. In this work, we quantify covert dialect bias in online discourse by evaluating how LMs associate stereotypical traits (derived from social psychology research on racial bias) with intent-equivalent tweets in Standard American English (SAE) and African-American Vernacular English (AAVE). While prior work shows that LMs associate more negative stereotypes with AAVE when evaluating tweets in isolation, we are surprised to find that this bias is significantly exacerbated when SAE / AAVE tweet pairs are compared side by side, a setting that more closely reflects high-impact decision making contexts in which models are used to rank candidates. The bias only worsens when dialect labels are explicitly specified. This is striking, given the extensive efforts from commercial developers to mitigate bias in their LMs. Encouragingly, we show that counterfactual fairness finetuning can mitigate covert dialect bias for some stereotypical traits, reducing average disparities when evaluating tweets in isolation, however, these improvements do not consistently hold across traits when evaluating SAE / AAVE tweets side by side. Our findings show that existing evaluation settings for covert dialect bias may underestimate its severity, specifically in contrastive settings. Additionally, overt dialect bias remains pronounced even after safety aligned finetuning, indicating that it remains an unresolved problem, and motivates the need for more robust evaluation and mitigation frameworks.