Search Results for author:
Found 13 papers, 1 papers with code
Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts
Although several datasets annotated for anaphoric reference/coreference exist, even the largest such datasets have limitations in terms of size, range of domains, coverage of anaphoric phenomena, and size of documents included.
SemEval-2021 Task 12: Learning with Disagreements
Disagreement between coders is ubiquitous in virtually all datasets annotated with human judgements in both natural language processing and computer vision.
Overview of MediaEval 2020 Predicting Media Memorability Task: What Makes a Video Memorable?
This paper describes the MediaEval 2020 \textit{Predicting Media Memorability} task.
Cipher: A Prototype Game-with-a-Purpose for Detecting Errors in Text
Errors commonly exist in machine-generated documents and publication materials; however, some correction algorithms do not perform well for complex errors and it is costly to employ humans to do the task.
Speaking Outside the Box: Exploring the Benefits of Unconstrained Input in Crowdsourcing and Citizen Science Platforms
Crowdsourcing approaches provide a difficult design challenge for developers.
Aggregation Driven Progression System for GWAPs
As the uses of Games-With-A-Purpose (GWAPs) broadens, the systems that incorporate its usages have expanded in complexity.
A Mention-Pair Model of Annotation with Nonparametric User Communities
The model is also flexible enough to be used in standard annotation tasks for classification where it registers on par performance with the state of the art.
Crowdsourcing and Aggregating Nested Markable Annotations
One of the key steps in language resource creation is the identification of the text segments to be annotated, or markables, which depending on the task may vary from nominal chunks for named entity resolution to (potentially nested) noun phrases in coreference resolution (or mentions) to larger text segments in text segmentation.
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
The corpus, containing annotations for about 108, 000 markables, is one of the largest corpora for coreference for English, and one of the largest crowdsourced NLP corpora, but its main feature is the large number of judgments per markable: 20 on average, and over 2. 2M in total.
A Probabilistic Annotation Model for Crowdsourcing Coreference
The availability of large scale annotated corpora for coreference is essential to the development of the field.
Comparing Bayesian Models of Annotation
We evaluate these models along four aspects: comparison to gold labels, predictive accuracy for new annotations, annotator characterization, and item difficulty, using four datasets with varying degrees of noise in the form of random (spammy) annotators.
Phrase Detectives Corpus 1.0 Crowdsourced Anaphoric Coreference.
Corpora are typically annotated by several experts to create a gold standard; however, there are now compelling reasons to use a non-expert crowd to annotate text, driven by cost, speed and scalability.
