Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets

1 code implementation • EMNLP 2020 • Nedjma Ousidhoum, Yangqiu Song, Dit-yan Yeung

Work on bias in hate speech typically aims to improve classification performance while relatively overlooking the quality of the data.

Hate Speech Detection Selection bias +3

BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages

1 code implementation • 14 Jun 2024 • Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Víctor Gutiérrez-Basulto, Yazmín Ibáñez-García, Hwaran Lee, Shamsuddeen Hassan Muhammad, Kiwoong Park, Anar Sabuhi Rzayev, Nina White, Seid Muhie Yimam, Mohammad Taher Pilehvar, Nedjma Ousidhoum, Jose Camacho-Collados, Alice Oh

To address this issue, we introduce BLEnD, a hand-crafted benchmark designed to evaluate LLMs' everyday knowledge across diverse cultures and languages.

Multiple-choice

Words as Trigger Points in Social Media Discussions

no code implementations • 16 May 2024 • Dimosthenis Antypas, Christian Arnold, Jose Camacho-Collados, Nedjma Ousidhoum, Carla Perez Almendros

Our work is the first to introduce trigger points to computational studies of online communication.

SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages

1 code implementation • 27 Mar 2024 • Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Meriem Beloucif, Christine de Kock, Oumaima Hourrane, Manish Shrivastava, Thamar Solorio, Nirmal Surange, Krishnapriya Vishnubhotla, Seid Muhie Yimam, Saif M. Mohammad

We present the first shared task on Semantic Textual Relatedness (STR).

Semantic Similarity Semantic Textual Similarity +1

SemRel2024: A Collection of Semantic Textual Relatedness Datasets for 13 Languages

2 code implementations • 13 Feb 2024 • Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Abinew Ali Ayele, Pavan Baswani, Meriem Beloucif, Chris Biemann, Sofia Bourhim, Christine de Kock, Genet Shanko Dekebo, Oumaima Hourrane, Gopichand Kanumolu, Lokesh Madasu, Samuel Rutunda, Manish Shrivastava, Thamar Solorio, Nirmal Surange, Hailegnaw Getaneh Tilaye, Krishnapriya Vishnubhotla, Genta Winata, Seid Muhie Yimam, Saif M. Mohammad

Exploring and quantifying semantic relatedness is central to representing language and holds significant implications across various NLP tasks.

Semantic Similarity Semantic Textual Similarity +1

The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Who

1 code implementation • 27 Apr 2023 • Michael Schlichtkrull, Nedjma Ousidhoum, Andreas Vlachos

Automated fact-checking is often presented as an epistemic tool that fact-checkers, social media consumers, and other stakeholders can use to fight misinformation.

Fact Checking Misinformation

SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)

1 code implementation • 13 Apr 2023 • Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Seid Muhie Yimam, David Ifeoluwa Adelani, Ibrahim Sa'id Ahmad, Nedjma Ousidhoum, Abinew Ayele, Saif M. Mohammad, Meriem Beloucif, Sebastian Ruder

We present the first Africentric SemEval Shared task, Sentiment Analysis for African Languages (AfriSenti-SemEval) - The dataset is available at https://github. com/afrisenti-semeval/afrisent-semeval-2023.

Classification Sentiment Analysis +2

AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages

3 code implementations • 17 Feb 2023 • Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, Nedjma Ousidhoum, David Ifeoluwa Adelani, Seid Muhie Yimam, Ibrahim Sa'id Ahmad, Meriem Beloucif, Saif M. Mohammad, Sebastian Ruder, Oumaima Hourrane, Pavel Brazdil, Felermino Dário Mário António Ali, Davis David, Salomey Osei, Bello Shehu Bello, Falalu Ibrahim, Tajuddeen Gwadabe, Samuel Rutunda, Tadesse Belay, Wendimu Baye Messelle, Hailu Beshada Balcha, Sisay Adugna Chala, Hagos Tesfahun Gebremichael, Bernard Opoku, Steven Arthur

These include 75 languages with at least one million speakers each.

Diversity Sentiment Classification +1

Varifocal Question Generation for Fact-checking

1 code implementation • 22 Oct 2022 • Nedjma Ousidhoum, Zhangdie Yuan, Andreas Vlachos

Our method outperforms previous work on a fact-checking question generation dataset on a wide range of automatic evaluation metrics.

Fact Checking Question Answering +2

Probing Toxic Content in Large Pre-Trained Language Models

1 code implementation • ACL 2021 • Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, Dit-yan Yeung

Large pre-trained language models (PTLMs) have been shown to carry biases towards different social groups which leads to the reproduction of stereotypical and toxic content by major NLP systems.

Probing Language Models Sentence

Multilingual and Multi-Aspect Hate Speech Analysis

1 code implementation • IJCNLP 2019 • Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, Dit-yan Yeung

Current research on hate speech analysis is typically oriented towards monolingual and single classification tasks.

Classification General Classification +1