Search Results for author:
Found 10 papers, 2 papers with code
D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation
While human annotations play a crucial role in language technologies, annotator subjectivity has long been overlooked in data collection.
Discipline and Label: A WEIRD Genealogy and Social Theory of Data Annotation
Data annotation remains the sine qua non of machine learning and AI.
Disentangling Perceptions of Offensiveness: Cultural and Moral Correlates
More importantly, we find that individual moral values play a crucial role in shaping these variations: moral concerns about Care and Purity are significant mediating factors driving cross-cultural differences.
SoUnD Framework: Analyzing (So)cial Representation in (Un)structured (D)ata
The unstructured nature of data used in foundation model development is a challenge to systematic analyses for making data use and documentation decisions.
A Framework to Assess (Dis)agreement Among Diverse Rater Groups
Recent advancements in conversational AI have created an urgent need for safety guardrails that prevent users from being exposed to offensive and dangerous content.
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Ranked #1 on
Question Answering
on StrategyQA
(Re)Defining Expertise in Machine Learning Development
Domain experts are often engaged in the development of machine learning systems in a variety of ways, such as in data collection and evaluation of system performance.
Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation
Human annotations play a crucial role in machine learning (ML) research and development.
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
Majority voting and averaging are common approaches employed to resolve annotator disagreements and derive single ground truth labels from multiple annotations.
On Releasing Annotator-Level Labels and Information in Datasets
A common practice in building NLP datasets, especially using crowd-sourced annotations, involves obtaining multiple annotator judgements on the same data instances, which are then flattened to produce a single "ground truth" label or score, through majority voting, averaging, or adjudication.
