Search Results for author:
Found 16 papers, 2 papers with code
Annotator Response Distributions as a Sampling Frame
Annotator disagreement is often dismissed as noise or the result of poor annotation process quality.
Findings of the 2021 Conference on Machine Translation (WMT21)
This paper presents the results of the newstranslation task, the multilingual low-resourcetranslation for Indo-European languages, thetriangular translation task, and the automaticpost-editing task organised as part of the Con-ference on Machine Translation (WMT) 2021. In the news task, participants were asked tobuild machine translation systems for any of10 language pairs, to be evaluated on test setsconsisting mainly of news stories.
How Many Ratings per Item are Necessary for Reliable Significance Testing?
We introduce methods for determining whether an (existing or planned) evaluation dataset has enough responses per item to reliably compare the performance of one model to another.
Grammatical Error Correction for Low-Resource Languages: The Case of Zarma
Our experiments show that the MT-based approach using the M2M100 model outperforms others, achieving a detection rate of 95. 82% and a suggestion accuracy of 78. 90% in automatic evaluations, and scoring 3. 0 out of 5. 0 in logical/grammar error correction during MEs by native speakers.
Feriji: A French-Zarma Parallel Corpus, Glossary & Translator
Machine translation (MT) is a rapidly expanding field that has experienced significant advancements in recent years with the development of models capable of translating multiple languages with remarkable accuracy.
GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives
Human annotation plays a core role in machine learning -- annotations for supervised models, safety guardrails for generative models, and human feedback for reinforcement learning, to cite a few avenues.
Disagreement Matters: Preserving Label Diversity by Jointly Modeling Item and Annotator Label Distributions with DisCo
Annotator disagreement is common whenever human judgment is needed for supervised learning.
Neural Machine Translation for Extremely Low-Resource African Languages: A Case Study on Bambara
Low-resource languages present unique challenges to (neural) machine translation.
Sensing and Learning Human Annotators Engaged in Narrative Sensemaking
While labor issues and quality assurance in crowdwork are increasingly studied, how annotators make sense of texts and how they are personally impacted by doing so are not.
Understanding the Semantics of Narratives of Interpersonal Violence through Reader Annotations and Physiological Reactions
Interpersonal violence (IPV) is a prominent sociological problem that affects people of all demographic backgrounds.
Analyzing Gender Bias in Student Evaluations
University students in the United States are routinely asked to provide feedback on the quality of the instruction they have received.
