Search Results for author:
Found 14 papers, 7 papers with code
Toward an Evaluation Science for Generative AI Systems
There is an increasing imperative to anticipate and understand the performance and safety of generative AI systems in real-world deployment contexts.
Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs
Algorithmic fairness has conventionally adopted a perspective of racial color-blindness (i. e., difference unaware treatment).
Measuring Implicit Bias in Explicitly Unbiased Large Language Models
Large language models (LLMs) can pass explicit social bias tests but still harbor implicit biases, similar to humans who endorse egalitarian beliefs yet exhibit subtle biases.
Measuring machine learning harms from stereotypes: requires understanding who is being harmed by which errors in what ways
However, certain stereotype-violating errors are more experientially harmful for men, potentially due to perceived threats to masculinity.
Overwriting Pretrained Bias with Finetuning Data
Transfer learning is beneficial by allowing the expressive features of models pretrained on large-scale datasets to be finetuned for the target task of smaller, more domain-specific datasets.
Gender Artifacts in Visual Datasets
Gender biases are known to exist within large-scale visual datasets and can be reflected or even amplified in downstream models.
Measuring Representational Harms in Image Captioning
We propose multiple measurement techniques for each type of harm.
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation
In this work, we grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: (1) which demographic attributes to include as dataset labels, (2) how to handle the progressively smaller size of subgroups during model training, and (3) how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups.
Understanding and Evaluating Racial Biases in Image Captioning
Image captioning is an important task for benchmarking visual reasoning and for enabling accessibility for people with vision impairments.
Directional Bias Amplification
We introduce and analyze a new, decoupled metric for measuring bias amplification, $\text{BiasAmp}_{\rightarrow}$ (Directional Bias Amplification).
A Technical and Normative Investigation of Social Bias Amplification
The conversation around the fairness of machine learning models is growing and evolving.
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Machine learning models are known to perpetuate and even amplify the biases present in the data.
Learning Robotic Manipulation through Visual Planning and Acting
We further demonstrate our approach on learning to imagine and execute in 3 environments, the final of which is deformable rope manipulation on a PR2 robot.
Safer Classification by Synthesis
We posit that a generative approach is the natural remedy for this problem, and propose a method for classification using generative models.
