Search Results for author:
Found 33 papers, 17 papers with code
A Taxonomy of Stereotype Content in Large Language Models
Finally, the taxonomy predicted the LLMs' internal evaluations of social categories (e. g., how positively/negatively the categories were represented), supporting the relevance of a multidimensional taxonomy for characterizing LLM stereotypes.
Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval
We find that the MTEs are biased, significantly favoring White-associated names in 85. 1\% of cases and female-associated names in only 11. 1\% of cases, with a minority of cases showing no statistically significant differences.
Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approach
Our research investigates the impact of Generative Artificial Intelligence (GAI) models, specifically text-to-image generators (T2Is), on the representation of non-Western cultures, with a focus on Indian contexts.
Breaking Bias, Building Bridges: Evaluation and Mitigation of Social Biases in LLMs via Contact Hypothesis
We propose a unique debiasing technique, Social Contact Debiasing (SCD), that instruction-tunes these models with unbiased responses to prompts.
BiasDora: Exploring Hidden Biased Associations in Vision-Language Models
Existing works examining Vision-Language Models (VLMs) for social biases predominantly focus on a limited set of documented bias associations, such as gender:profession or race:crime.
ChatGPT as Research Scientist: Probing GPT's Capabilities as a Research Librarian, Research Ethicist, Data Generator and Data Predictor
How good a research scientist is ChatGPT?
Label-Efficient Group Robustness via Out-of-Distribution Concept Curation
We propose Concept Correction a framework that represents a concept as a curated set of images from any source then labels each training sample by its similarity to the concept set to control spurious correlations.
'Person' == Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion
We observe how Stable Diffusion outputs of `a person' without any additional gender/nationality information correspond closest to images of men and least with persons of nonbinary gender, and to persons from Europe/North America over Africa/Asia, pointing towards Stable Diffusion having a concerning representation of personhood to be a European/North American man.
Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition
We compare biases found in pre-trained models to biases in downstream models adapted to the task of Speech Emotion Recognition (SER) and find that in 66 of the 96 tests performed (69%), the group that is more associated with positive valence as indicated by the SpEAT also tends to be predicted as speaking with higher valence by the downstream model.
Is the U.S. Legal System Ready for AI's Challenges to Human Values?
Our interdisciplinary study investigates how effectively U. S. laws confront the challenges posed by Generative AI to human values.
Evaluating Biased Attitude Associations of Language Models in an Intersectional Context
Adapting the projection-based approach to embedding association tests that quantify bias, we find that language models exhibit the most biased attitudes against gender identity, social class, and sexual orientation signals in language.
Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Tasks
We investigate bias against these groups in English pre-trained Masked Language Models (MLMs) and their downstream sentiment classification tasks.
ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages
In this multicultural age, language translation is one of the most performed tasks, and it is becoming increasingly AI-moderated and automated.
Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias
A first experiment uses standardized images of women from the Sexual OBjectification and EMotion Database, and finds that human characteristics are disassociated from images of objectified women: the model's recognition of emotional state is mediated by whether the subject is fully or partially clothed.
Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale
For example, we find cases of prompting for basic traits or social roles resulting in images reinforcing whiteness as ideal, prompting for occupations resulting in amplification of racial and gender disparities, and prompting for objects resulting in reification of American norms.
American == White in Multimodal Language-and-Image AI
In an image captioning task, BLIP remarks upon the race of Asian individuals as much as 36% of the time, but never remarks upon race for White individuals.
Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics
Using the Single-Category Word Embedding Association Test, we demonstrate the widespread prevalence of gender biases that also show differences in: (1) frequencies of words associated with men versus women; (b) part-of-speech tags in gender-associated words; (c) semantic categories in gender-associated words; and (d) valence, arousal, and dominance in gender-associated words.
Measuring Gender Bias in Word Embeddings of Gendered Languages Requires Disentangling Grammatical Gender Signals
We demonstrate that word embeddings learn the association between a noun and its grammatical gender in grammatically gendered languages, which can skew social gender bias measurements.
Markedness in Visual Semantic AI
The model is more likely to rank the unmarked "person" label higher than labels denoting gender for Male individuals (26. 7% of the time) vs.
Evidence for Hypodescent in Visual Semantic AI
We examine the state-of-the-art multimodal "visual semantic" model CLIP ("Contrastive Language Image Pretraining") for the rule of hypodescent, or one-drop rule, whereby multiracial people are more likely to be assigned a racial or ethnic label corresponding to a minority or disadvantaged racial or ethnic group than to the equivalent majority or advantaged group.
Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations
We find that contrastive visual semantic pretraining significantly mitigates the anisotropy found in contextualized word embeddings from GPT-2, such that the intra-layer self-similarity (mean pairwise cosine similarity) of CLIP word embeddings is under . 25 in all layers, compared to greater than . 95 in the top layer of GPT-2.
VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models
VAST, the Valence-Assessing Semantics Test, is a novel intrinsic evaluation task for contextualized word embeddings (CWEs).
Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models
Moreover, we find Spearman's r between racial bias and name frequency in BERT of . 492, indicating that lower-frequency minority group names are more associated with unpleasantness.
Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases
Recent advances in machine learning leverage massive datasets of unlabeled images from the web to learn general-purpose image representations for tasks from image classification to face recognition.
Disparate Impact of Artificial Intelligence Bias in Ridehailing Economy's Price Discrimination Algorithms
An analysis of 100 million ridehailing samples from the city of Chicago indicates a significant disparate impact in fare pricing of neighborhoods due to AI bias learned from ridehailing utilization patterns associated with demographic attributes.
ValNorm Quantifies Semantics to Reveal Consistent Valence Biases Across Languages and Over Centuries
Word embeddings learn implicit biases from linguistic regularities captured by word co-occurrence statistics.
Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases
Furthermore, we develop two methods, Intersectional Bias Detection (IBD) and Emergent Intersectional Bias Detection (EIBD), to automatically identify the intersectional biases and emergent intersectional biases from static word embeddings in addition to measuring them in contextualized word embeddings.
Automatically Characterizing Targeted Information Operations Through Biases Present in Discourse on Twitter
This paper considers the problem of automatically characterizing overall attitudes and biases that may be associated with emerging information operations via artificial intelligence.
A Set of Distinct Facial Traits Learned by Machines Is Not Predictive of Appearance Bias in the Wild
We shed light on the ways in which appearance bias could be embedded in face processing technology and cast further doubt on the practice of predicting subjective traits based on appearances.
Git Blame Who?: Stylistic Authorship Attribution of Small, Incomplete Source Code Fragments
We show that we can also use these calibration curves in the case that we do not have linking information and thus are forced to classify individual samples directly.
Semantics derived automatically from language corpora contain human-like biases
Here we show for the first time that human-like semantic biases result from the application of standard machine learning to ordinary language---the same sort of language humans are exposed to every day.
When Coding Style Survives Compilation: De-anonymizing Programmers from Executable Binaries
Many distinguishing features present in source code, e. g. variable names, are removed in the compilation process, and compiler optimization may alter the structure of a program, further obscuring features that are known to be useful in determining authorship.
Cryptography and Security
