Search Results for author:
Found 19 papers, 10 papers with code
SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
This is a critical safety risk for businesses and developers.
The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values
Human feedback is increasingly used to steer the behaviours of Large Language Models (LLMs).
The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models
In this paper, we address the concept of "alignment" in large language models (LLMs) through the lens of post-structuralist socio-political theory, specifically examining its parallels to empty signifiers.
Casteist but Not Racist? Quantifying Disparities in Large Language Model Bias between India and the West
In this paper, we quantify stereotypical bias in popular LLMs according to an Indian-centric frame and compare bias levels between the Indian and Western contexts.
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
In this paper, we introduce a new test suite called XSTest to identify such eXaggerated Safety behaviours in a systematic way.
DoDo Learning: DOmain-DemOgraphic Transfer in Language Models for Detecting Abuse Targeted at Public Figures
We find that (i) small amounts of diverse data are hugely beneficial to generalisation and model adaptation; (ii) models transfer more easily across demographics but models trained on cross-domain data are more generalisable; (iii) some groups contribute more to generalisability than others; and (iv) dataset similarity is a signal of transferability.
VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution
We introduce VisoGender, a novel dataset for benchmarking gender bias in vision-language models.
Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Sets
To address this issue, we propose a novel dataset debiasing pipeline to augment the COCO dataset with synthetic, gender-balanced contrast sets, where only the gender of the subject is edited and the background is fixed.
Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
To address this need, we introduce the Adversarial Nibbler challenge.
Assessing Language Model Deployment with Risk Cards
However, there is no risk-centric framework for documenting the complexity of a landscape in which some risks are shared across models and contexts, while others are specific, and where certain conditions may be required for risks to manifest as harms.
Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback
Large language models (LLMs) are used to generate content for a wide range of tasks, and are set to reach a growing audience in coming years due to integration in product interfaces like ChatGPT or search engines like Bing.
SemEval-2023 Task 10: Explainable Detection of Online Sexism
Online sexism is a widespread and harmful phenomenon.
Auditing large language models: a three-layered approach
However, existing auditing procedures fail to address the governance challenges posed by LLMs, which display emergent capabilities and are adaptable to a wide range of downstream tasks.
Is More Data Better? Re-thinking the Importance of Efficiency in Abusive Language Detection with Transformers-Based Active Learning
Annotating abusive language is expensive, logistically complex and creates a risk of psychological harm.
Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements
The growing capability and availability of generative language models has enabled a wide range of new downstream tasks.
A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning
Vision-language models can encode societal biases and stereotypes, but there are challenges to measuring and mitigating these multimodal harms due to lacking measurement robustness and feature degradation.
Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate
Using the test suite, we expose weaknesses in existing hate detection models.
Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset
In this paper, we collect hateful and non-hateful memes from Pinterest to evaluate out-of-sample performance on models pre-trained on the Facebook dataset.
