Search Results for author:
Found 36 papers, 23 papers with code
Using Item Response Theory to Measure Gender and Racial Bias of a BERT-based Automated English Speech Assessment System
This study addresses the need to examine potential biases of transformer-based models in the context of automated English speech assessment.
MUSE: Machine Unlearning Six-Way Evaluation for Language Models
Data owners may request the removal of their data from a trained model due to privacy or copyright concerns.
CLIMB: A Benchmark of Clinical Bias in Large Language Models
We introduce CLIMB (shorthand for A Benchmark of Clinical Bias in Large Language Models), a pioneering comprehensive benchmark to evaluate both intrinsic (within LLMs) and extrinsic (on downstream tasks) bias in LLMs for clinical decision tasks.
Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective
Consequently, focusing on blurring gender representations within the image encoder, which contributes most to the model bias, reduces bias efficiently by 22. 03% and 9. 04% in the MSCOCO and PASCAL-SENTENCE datasets, respectively, with minimal performance loss or increased computational demands.
InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context
In this paper, we developed a novel framework, InterIntent, to assess LLMs' social intelligence by mapping their ability to understand and manage intentions in a game setting.
"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations
In this study, we utilize GPT-3. 5-Turbo and Llama 3-70B-Instruct to simulate hiring decisions and salary recommendations for candidates with 320 first names that strongly signal their race and gender, across over 750, 000 prompts.
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
Multilingual large language models leak human stereotypes across language boundaries
This raises the question: do stereotypes present in one social context leak across languages within the model?
Self-Contradictory Reasoning Evaluation and Detection
In a plethora of recent work, large language models (LLMs) demonstrated impressive reasoning ability, but many proposed downstream reasoning tasks focus on performance-wise evaluation.
Safer-Instruct: Aligning Language Models with Automated Preference Data
To verify the effectiveness of Safer-Instruct, we apply the pipeline to construct a safety preference dataset as a case study.
Fair Abstractive Summarization of Diverse Perspectives
However, current work in summarization metrics and Large Language Models (LLMs) evaluation has not explored fair abstractive summarization.
Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems
Recent advancements in Large Language Models empower them to follow freeform instructions, including imitating generic or specific demographic personas in conversations.
Adapting Static Fairness to Sequential Decision-Making: Bias Mitigation Strategies towards Equal Long-term Benefit Rate
To address biases in sequential decision-making, we introduce a long-term fairness concept named Equal Long-term Benefit Rate (ELBERT).
TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning
Despite recent progress in reinforcement learning (RL) from raw pixel data, sample inefficiency continues to present a substantial obstacle.
Auditing Algorithmic Fairness in Machine Learning for Health with Severity-Based LOGAN
Therefore, ML auditing tools must be (1) better aligned with ML4H auditing principles and (2) able to illuminate and characterize communities vulnerable to the most harm.
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers
Large pre-trained language models have shown remarkable performance over the past few years.
SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models
A common limitation of diagnostic tests for detecting social biases in NLP models is that they may only detect stereotypic associations that are pre-specified by the designer of the test.
DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation
Disinformation has become a serious problem on social media.
On Measures of Biases and Harms in NLP
Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality.
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?
We investigate the effectiveness of natural language interventions for reading-comprehension systems, studying this in the context of social stereotypes.
Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation
Our method is able to reveal the hidden model biases not directly shown in the test dataset.
LOGAN: Local Group Bias Detection by Clustering
Machine learning techniques have been widely used in natural language processing (NLP).
``The Boating Store Had Its Best Sail Ever'': Pronunciation-attentive Contextualized Pun Recognition
In this paper, we propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor, detect if a sentence contains puns and locate them in the sentence.
Fairness-Aware Explainable Recommendation over Knowledge Graphs
There has been growing attention on fairness considerations recently, especially in the context of intelligent decision making systems.
Mitigating Gender Bias Amplification in Distribution by Posterior Regularization
With little performance loss, our method can almost remove the bias amplification in the distribution.
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer
In this paper, we study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications.
"The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognition
In this paper, we propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor, detect if a sentence contains puns and locate them in the sentence.
Short-Term Temporal Convolutional Networks for Dynamic Hand Gesture Recognition
Subsequently, in temporal analysis, we use TCNs to extract temporal features and employ improved Squeeze-and-Excitation Networks (SENets) to strengthen the representational power of temporal features from each TCNs' layers.
Towards Understanding Gender Bias in Relation Extraction
We use WikiGenderBias to evaluate systems for bias and find that NRE systems exhibit gender biased predictions and lay groundwork for future evaluation of bias in NRE.
Examining Gender Bias in Languages with Grammatical Gender
Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora.
Mitigating Gender Bias in Natural Language Processing: Literature Review
In this paper, we review contemporary studies on recognizing and mitigating gender bias in NLP.
Gender Bias in Contextualized Word Embeddings
In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo's contextualized word vectors.
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
In this work, we present a framework to measure and mitigate intrinsic biases with respect to protected variables --such as gender-- in visual recognition tasks.
Learning Gender-Neutral Word Embeddings
Word embedding models have become a fundamental component in a wide range of Natural Language Processing (NLP) applications.
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias.
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Language is increasingly being used to define rich visual recognition problems with supporting image collections sourced from the web.
