Search Results for author:
Found 25 papers, 3 papers with code
ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Detection in Conversational AI
We find that the distribution of abuse is vastly different compared to other commonly used datasets, with more sexually tinted aggression towards the virtual persona of these systems.
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems
We then empirically assess the extent to which current tools can measure these effects and current systems display them.
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design
Over the last several years, end-to-end neural conversational agents have vastly improved their ability to carry unrestricted, open-domain conversations with humans.
NLP for Counterspeech against Hate: A Survey and How-To Guide
In recent years, counterspeech has emerged as one of the most promising strategies to fight online hate.
Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution
We then analyze the emotions generated by the models in relation to the gender-event pairs.
Subjective $\textit{Isms}$? On the Danger of Conflating Hate and Offence in Abusive Language Detection
Natural language processing research has begun to embrace the notion of annotator subjectivity, motivated by variations in labelling.
Understanding Counterspeech for Online Harm Mitigation
Counterspeech offers direct rebuttals to hateful speech by challenging perpetrators of hate and showing support to targets of abuse.
Mirages: On Anthropomorphism in Dialogue Systems
In this paper, we discuss the linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise, including reinforcing gender stereotypes and notions of acceptable language.
On the Origins of Bias in NLP through the Lens of the Jim Code
In this paper, we trace the biases in current natural language processing (NLP) models back to their origins in racism, sexism, and homophobia over the last 500 years.
iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives?
There are two competing approaches for modelling annotator disagreement: distributional soft-labelling approaches (which aim to capture the level of disagreement) or modelling perspectives of individual annotators or groups thereof.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)
We report on the second LeWiDi shared task, which differs from the first edition in three crucial respects: (i) it focuses entirely on NLP, instead of both NLP and computer vision tasks in its first edition; (ii) it focuses on subjective tasks, instead of covering different types of disagreements-as training with aggregated labels for subjective NLP tasks is a particularly obvious misrepresentation of the data; and (iii) for the evaluation, we concentrate on soft approaches to evaluation.
Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement
We commonly use agreement measures to assess the utility of judgements made by human annotators in Natural Language Processing (NLP) tasks.
Risk-graded Safety for Handling Medical Queries in Conversational AI
While individual crowdworkers may be unreliable at grading the seriousness of the prompts, their aggregated labels tend to agree with professional opinion to a greater extent on identifying the medical queries and recognising the risk types posed by the responses.
ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI
We find that the distribution of abuse is vastly different compared to other commonly used datasets, with more sexually tinted aggression towards the virtual persona of these systems.
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans.
Alexa, Google, Siri: What are Your Pronouns? Gender and Anthropomorphism in the Design and Perception of Conversational Assistants
Technology companies have produced varied responses to concerns about the effects of the design of their conversational AI systems.
ParlVote: A Corpus for Sentiment Analysis of Political Debates
These include a linear classifier as well as a neural network trained using a transformer word embedding model (BERT), and fine-tuned on the parliamentary speeches.
Policy Preference Detection in Parliamentary Debate Motions
Debate motions (proposals) tabled in the UK Parliament contain information about the stated policy preferences of the Members of Parliament who propose them, and are key to the analysis of all subsequent speeches given in response to them.
Semantic Change in the Language of UK Parliamentary Debates
We investigate changes in the meanings of words used in the UK Parliament across two different epochs.
Sentiment and position-taking analysis of parliamentary debates: A systematic literature review
In this article we present the results of a systematic literature review of 61 studies, all of which address the automatic analysis of the sentiment and opinions expressed and positions taken by speakers in parliamentary (and other legislative) debates.
Identifying Opinion-Topics and Polarity of Parliamentary Debate Motions
Analysis of the topics mentioned and opinions expressed in parliamentary debate motions{--}or proposals{--}is difficult for human readers, but necessary for understanding and automatic processing of the content of the subsequent speeches.
A Rule-based Shallow-transfer Machine Translation System for Scots and English
By concentrating on translation for assimilation (gist comprehension) from Scots to English, it is proposed that the development of dictionaries designed to be used with in the Apertium platform will be sufficient to produce translations that improve non-Scots speakers understanding of the language.
