Search Results for author:
Found 28 papers, 7 papers with code
The PRISM Alignment Project: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
Human feedback plays a central role in the alignment of Large Language Models (LLMs).
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
While some of the models do not give a single unsafe response, most give unsafe responses to more than 20% of the prompts, with over 50% unsafe responses in the extreme.
Lost in Translation -- Multilingual Misinformation and its Evolution
Using fact-checks as a proxy for the spread of misinformation, we find 33% of repeated claims cross linguistic boundaries, suggesting that some misinformation permeates language barriers.
The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values
Human feedback is increasingly used to steer the behaviours of Large Language Models (LLMs).
The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models
In this paper, we address the concept of "alignment" in large language models (LLMs) through the lens of post-structuralist socio-political theory, specifically examining its parallels to empty signifiers.
Casteist but Not Racist? Quantifying Disparities in Large Language Model Bias between India and the West
In this paper, we quantify stereotypical bias in popular LLMs according to an Indian-centric frame and compare bias levels between the Indian and Western contexts.
DoDo Learning: DOmain-DemOgraphic Transfer in Language Models for Detecting Abuse Targeted at Public Figures
We find that (i) small amounts of diverse data are hugely beneficial to generalisation and model adaptation; (ii) models transfer more easily across demographics but models trained on cross-domain data are more generalisable; (iii) some groups contribute more to generalisability than others; and (iv) dataset similarity is a signal of transferability.
Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback
Large language models (LLMs) are used to generate content for a wide range of tasks, and are set to reach a growing audience in coming years due to integration in product interfaces like ChatGPT or search engines like Bing.
Query Rewriting for Effective Misinformation Discovery
We propose a novel system to help fact-checkers formulate search queries for known misinformation claims and effectively search across multiple social media platforms.
Is More Data Better? Re-thinking the Importance of Efficiency in Abusive Language Detection with Transformers-Based Active Learning
Annotating abusive language is expensive, logistically complex and creates a risk of psychological harm.
Top Gear or Black Mirror: Inferring Political Leaning From Non-Political Content
Polarization and echo chambers are often studied in the context of explicitly political events such as elections, and little scholarship has examined the mixing of political groups in non-political contexts.
Matching Tweets With Applicable Fact-Checks Across Languages
We conduct both classification and retrieval experiments, in monolingual (English only), multilingual (Spanish, Portuguese), and cross-lingual (Hindi-English) settings using multilingual transformer models such as XLM-RoBERTa and multilingual embeddings such as LaBSE and SBERT.
Fairness via AI: Bias Reduction in Medical Information
While part (3) of this work specifically focuses on the health domain, the fundamental computer science advances and contributions stemming from research efforts in bias reduction and Fairness via AI have broad implications in all areas of society.
Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate
Using the test suite, we expose weaknesses in existing hate detection models.
Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate
We show that both text- and visual- enrichment improves model performance, with the multimodal model (0. 771) outperforming other models' F1 scores (0. 544, 0. 737, and 0. 754).
Tiplines to Combat Misinformation on Encrypted Platforms: A Case Study of the 2019 Indian Election on WhatsApp
There is currently no easy way to fact-check content on WhatsApp and other end-to-end encrypted platforms at scale.
Claim Matching Beyond English to Scale Global Fact-Checking
We train our own embedding model using knowledge distillation and a high-quality "teacher" model in order to address the imbalance in embedding quality between the low- and high-resource languages in our dataset.
Semantic Journeys: Quantifying Change in Emoji Meaning from 2012-2018
The semantics of emoji has, to date, been considered from a static perspective.
Tackling Racial Bias in Automated Online Hate Detection: Towards Fair and Accurate Classification of Hateful Online Users Using Geometric Deep Learning
Yet, most research in online hate detection to date has focused on hateful content.
Demographic Inference and Representative Population Estimates from Multilingual Social Media Data
In a large experiment over multilingual heterogeneous European regions, we show that our demographic inference and bias correction together allow for more accurate estimates of populations and make a significant step towards representative social sensing in downstream applications with multilingual social media.
Measuring the Volatility of the Political agenda in Public Opinion and News Media
Recent election surprises, regime changes, and political shocks indicate that political agendas have become more fast-moving and volatile.
Foreign-language Reviews: Help or Hindrance?
The number and quality of user reviews greatly affects consumer purchasing decisions.
User Reviews and Language: How Language Influences Ratings
The number of user reviews of tourist attractions, restaurants, mobile apps, etc.
Understanding Editing Behaviors in Multilingual Wikipedia
We study multilingualism by collecting and analyzing a large dataset of the content written by multilingual editors of the English, German, and Spanish editions of Wikipedia.
How much is said in a microblog? A multilingual inquiry based on Weibo and Twitter
This paper presents a multilingual study on, per single post of microblog text, (a) how much can be said, (b) how much is written in terms of characters and bytes, and (c) how much is said in terms of information content in posts by different organizations in different languages.
Cross-language Wikipedia Editing of Okinawa, Japan
This article analyzes users who edit Wikipedia articles about Okinawa, Japan, in English and Japanese.
Multilinguals and Wikipedia Editing
This article analyzes one month of edits to Wikipedia in order to examine the role of users editing multiple language editions (referred to as multilingual users).
