Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effects

no code implementations • 24 May 2025 • Reva Schwartz, Rumman Chowdhury, Akash Kundu, Heather Frase, Marzieh Fadaee, Tom David, Gabriella Waters, Afaf Taik, Morgan Briggs, Patrick Hall, Shomik Jain, Kyra Yee, Spencer Thomas, Sundeep Bhandari, Paul Duncan, Andrew Thompson, Maya Carlyle, Qinghua Lu, Matthew Holmes, Theodora Skeadas

Conventional AI evaluation approaches concentrated within the AI stack exhibit systemic limitations for exploring, navigating and resolving the human and societal factors that play out in real world deployment such as in education, finance, healthcare, and employment sectors.

Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition

1 code implementation • 29 Oct 2023 • Isaac Slaughter, Craig Greenberg, Reva Schwartz, Aylin Caliskan

We compare biases found in pre-trained models to biases in downstream models adapted to the task of Speech Emotion Recognition (SER) and find that in 66 of the 96 tests performed (69%), the group that is more associated with positive valence as indicated by the SpEAT also tends to be predicted as speaking with higher valence by the downstream model.

Speech Emotion Recognition

The Role of Individual User Differences in Interpretable and Explainable Machine Learning Systems

no code implementations • 14 Sep 2020 • Lydia P. Gleaves, Reva Schwartz, David A. Broniatowski

There is increased interest in assisting non-expert audiences to effectively interact with machine learning (ML) tools and understand the complex output such systems produce.

BIG-bench Machine Learning