About

Bias detection is the task of detecting and measuring racism, sexism and otherwise discriminatory behavior in a model (Source: https://stereoset.mit.edu/)

Benchmarks

TREND DATASET BEST METHOD PAPER TITLE PAPER CODE COMPARE

Subtasks

Datasets

Greatest papers with code

fairmodels: A Flexible Tool For Bias Detection, Visualization, And Mitigation

1 Apr 2021ModelOriented/FairModels

The package includes a series of methods for bias mitigation that aim to diminish the discrimination in the model.

BIAS DETECTION FAIRNESS

StereoSet: Measuring stereotypical bias in pretrained language models

20 Apr 2020moinnadeem/StereoSet

Since pretrained language models are trained on large real world data, they are known to capture stereotypical biases.

BIAS DETECTION

Towards explainable classifiers using the counterfactual approach -- global explanations for discovering bias in data

Preprint 2020 agamiko/gebi

The paper proposes summarized attribution-based post-hoc explanations for the detection and identification of bias in data.

BIAS DETECTION

LOGAN: Local Group Bias Detection by Clustering

EMNLP 2020 uclanlp/clusters

Machine learning techniques have been widely used in natural language processing (NLP).

BIAS DETECTION OBJECT CLASSIFICATION

Measuring Gender Bias in Word Embeddings across Domains and Discovering New Gender Bias Word Categories

WS 2019 alfredomg/GeBNLP2019

We find that some domains are definitely more prone to gender bias than others, and that the categories of gender bias present also vary for each set of word embeddings.

BIAS DETECTION GENDER BIAS DETECTION TWO-SAMPLE TESTING WORD EMBEDDINGS

Towards Detection of Subjective Bias using Contextualized Word Embeddings

16 Feb 2020tanvidadu/Subjective-Bias-Detection

Subjective bias detection is critical for applications like propaganda detection, content recommendation, sentiment analysis, and bias neutralization.

BIAS DETECTION PROPAGANDA DETECTION SENTIMENT ANALYSIS WORD EMBEDDINGS

Context in Informational Bias Detection

3 Dec 2020vdenberg/context-in-informational-bias-detection

We find that the best-performing context-inclusive model outperforms the baseline on longer sentences, and sentences from politically centrist articles.

BIAS DETECTION

Automated Dependence Plots

2 Dec 2019davidinouye/automated-dependence-plots

To address these drawbacks, we formalize a method for automating the selection of interesting PDPs and extend PDPs beyond showing single features to show the model response along arbitrary directions, for example in raw feature space or a latent space arising from some generative model.

MODEL SELECTION SELECTION BIAS

Predicting the Leading Political Ideology of YouTube Channels Using Acoustic, Textual, and Metadata Information

20 Oct 2019yoandinkov/interspeech-2019

Our analysis shows that the use of acoustic signal helped to improve bias detection by more than 6% absolute over using text and metadata only.

BIAS DETECTION MULTIMODAL DEEP LEARNING

Multilingual sentence-level bias detection in Wikipedia

RANLP 2019 crim-ca/wiki-bias

We propose a multilingual method for the extraction of biased sentences from Wikipedia, and use it to create corpora in Bulgarian, French and English.

BIAS DETECTION