Bias Detection
54 papers with code • 5 benchmarks • 8 datasets
Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)
Benchmarks
These leaderboards are used to track progress in Bias Detection
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
StereoSet
CI-MNIST
Chicago Face Database (CFD)
WEATHub
Latest papers
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations
In this research, we used OpenAI GPT as point of comparison since it excels at all levels of safety.
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set.
RuBia: A Russian Language Bias Detection Dataset
To illustrate the dataset's purpose, we conduct a diagnostic evaluation of state-of-the-art or near-state-of-the-art LLMs and discuss the LLMs' predisposition to social biases.
The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias
However, we have identified a lack of interdisciplinarity in existing projects, and a need for more awareness of the various types of media bias to support methodologically thorough performance evaluations of media bias detection systems.
Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models
LLMs are increasingly powerful and widely used to assist users in a variety of tasks.
LUCID-GAN: Conditional Generative Models to Locate Unfairness
integrated-intelligence-lab/canonical_sets
•
•
Most group fairness notions detect unethical biases by computing statistical parity metrics on a model's output.
The Hidden Language of Diffusion Models
In this work, we present Conceptor, a novel method to interpret the internal representation of a textual concept by a diffusion model.
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
The development of large language models (LLMs) such as ChatGPT has brought a lot of attention recently.
Trade-Offs Between Fairness and Privacy in Language Modeling
cleolotta/fair-and-private-lm
•
•
Protecting privacy in contemporary NLP models is gaining in importance.
BiasAsker: Measuring the Bias in Conversational AI System
Particularly, it is hard to generate inputs that can comprehensively trigger potential bias due to the lack of data containing both social groups as well as biased properties.
