no code implementations • 20 Jun 2016 • Abram Handler, Su Lin Blodgett, Brendan O'Connor
We explore two techniques which use color to make sense of statistical text models.
no code implementations • EMNLP 2016 • Su Lin Blodgett, Lisa Green, Brendan O'Connor
Though dialectal language is increasingly abundant on social media, few resources exist for developing NLP tools to handle such language.
no code implementations • 30 Jun 2017 • Su Lin Blodgett, Brendan O'Connor
We highlight an important frontier in algorithmic fairness: disparity in the quality of natural language processing algorithms when applied to language from authors of different social groups.
no code implementations • WS 2017 • Su Lin Blodgett, Johnny Wei, Brendan O{'}Connor
While language identification works well on standard texts, it performs much worse on social media language, in particular dialectal language{---}even for English.
1 code implementation • NAACL 2018 • Katherine A. Keith, Su Lin Blodgett, Brendan O'Connor
Dependency parsing research, which has made significant gains in recent years, typically focuses on improving the accuracy of single-tree predictions.
no code implementations • ACL 2018 • Su Lin Blodgett, Johnny Wei, Brendan O{'}Connor
Due to the presence of both Twitter-specific conventions and non-standard and dialectal language, Twitter presents a significant parsing challenge to current dependency parsing tools.
1 code implementation • 28 May 2020 • Su Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna Wallach
We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process.
no code implementations • ACL 2020 • Su Lin Blodgett, Solon Barocas, Hal Daum{\'e} III, Hanna Wallach
We survey 146 papers analyzing {``}bias{''} in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing {``}bias{''} is an inherently normative process.
no code implementations • 7 Apr 2021 • Christian Hardmeier, Marta R. Costa-jussà, Kellie Webster, Will Radford, Su Lin Blodgett
At the Workshop on Gender Bias in NLP (GeBNLP), we'd like to encourage authors to give explicit consideration to the wider aspects of bias and its social implications.
no code implementations • 18 May 2021 • Michael Madaio, Su Lin Blodgett, Elijah Mayfield, Ezekiel Dixon-Román
Educational technologies, and the systems of schooling in which they are deployed, enact particular ideologies about what is important to know and how learners should learn.
no code implementations • ACL 2021 • Anjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia Tsvetkov
Despite inextricable ties between race and language, little work has considered race in NLP research and development.
no code implementations • ACL 2021 • Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
Auditing NLP systems for computational harms like surfacing stereotypes is an elusive goal.
no code implementations • 19 Oct 2021 • Su Lin Blodgett, Michael Madaio
If the authors of a recent Stanford report (Bommasani et al., 2021) on the opportunities and risks of "foundation models" are to be believed, these models represent a paradigm shift for AI and for the domains in which they will supposedly be used, including education.
no code implementations • NAACL 2022 • Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu
There are many ways to express similar things in text, which makes evaluating natural language generation (NLG) systems difficult.
1 code implementation • 29 Dec 2022 • Ankita Gupta, Su Lin Blodgett, Justin H Gross, Brendan O'Connor
Participants in political discourse employ rhetorical strategies -- such as hedging, attributions, or denials -- to display varying degrees of belief commitments to claims proposed by themselves or others.
no code implementations • 13 Jan 2023 • Samer B. Nashed, Justin Svegliato, Su Lin Blodgett
As automated decision making and decision assistance systems become common in everyday life, research on the prevention or mitigation of potential harms that arise from decisions made by these systems has proliferated.
no code implementations • 15 May 2023 • Arjun Subramonian, Xingdi Yuan, Hal Daumé III, Su Lin Blodgett
Progress in NLP is increasingly measured through benchmarks; hence, contextualizing progress requires understanding when and why practitioners may disagree about the validity of benchmarks.
no code implementations • 22 May 2023 • Seraphina Goldfarb-Tarrant, Eddie Ungless, Esma Balkir, Su Lin Blodgett
Bias research in NLP seeks to analyse models for social biases, thus helping NLP practitioners uncover, measure, and mitigate social harms.
no code implementations • 9 Jun 2023 • Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, Apostol Vassilev
We move toward a standard approach in evaluating a generative AI system for any modality, in two overarching categories: what is able to be evaluated in a base system that has no predetermined application and what is able to be evaluated in society.
1 code implementation • Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics 2023 • Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach
It is critical to measure and mitigate fairness- related harms caused by AI text generation systems, including stereotyping and demeaning harms.
no code implementations • 23 Oct 2023 • Li Lucy, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, Alexandra Olteanu
Fairness-related assumptions about what constitute appropriate NLG system behaviors range from invariance, where systems are expected to behave identically for social groups, to adaptation, where behaviors should instead vary across them.
no code implementations • 18 Nov 2023 • Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
We focus on how, which, and when responsible AI issues are covered, which relevant stakeholders are considered, and mismatches between stated and realized research goals.
no code implementations • 6 Feb 2024 • Angelina Wang, Xuechunzi Bai, Solon Barocas, Su Lin Blodgett
However, certain stereotype-violating errors are more experientially harmful for men, potentially due to perceived threats to masculinity.