Search Results for author:
Found 14 papers, 4 papers with code
We Need to Consider Disagreement in Evaluation
Instead, we suggest that we need to better capture the sources of disagreement to improve today’s evaluation practice.
The Universal Anaphora Scorer
It also supports the evaluation of split antecedent anaphora and discourse deixis, for which no tools existed.
Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts
Although several datasets annotated for anaphoric reference/coreference exist, even the largest such datasets have limitations in terms of size, range of domains, coverage of anaphoric phenomena, and size of documents included.
Scoring Coreference Chains with Split-Antecedent Anaphors
Anaphoric reference is an aspect of language interpretation covering a variety of types of interpretation beyond the simple case of identity reference to entities introduced via nominal expressions covered by the traditional coreference task in its most recent incarnation in ONTONOTES and similar datasets.
Beyond Black \& White: Leveraging Annotator Disagreement via Soft-Label Multi-Task Learning
Supervised learning assumes that a ground truth label exists.
Stay Together: A System for Single and Split-antecedent Anaphora Resolution
Split-antecedent anaphora is rarer and more complex to resolve than single-antecedent anaphora; as a result, it is not annotated in many datasets designed to test coreference, and previous work on resolving this type of anaphora was carried out in unrealistic conditions that assume gold mentions and/or gold split-antecedent anaphors are available.
Aggregating and Learning from Multiple Annotators
There is also a growing body of recent work arguing that following the convention and training with adjudicated labels ignores any uncertainty the labellers had in their classifications, which results in models with poorer generalisation capabilities.
Free the Plural: Unrestricted Split-Antecedent Anaphora Resolution
One limitation of virtually all coreference resolution models is the focus on single-antecedent anaphors.
Aggregation Driven Progression System for GWAPs
As the uses of Games-With-A-Purpose (GWAPs) broadens, the systems that incorporate its usages have expanded in complexity.
A Mention-Pair Model of Annotation with Nonparametric User Communities
The model is also flexible enough to be used in standard annotation tasks for classification where it registers on par performance with the state of the art.
Crowdsourcing and Aggregating Nested Markable Annotations
One of the key steps in language resource creation is the identification of the text segments to be annotated, or markables, which depending on the task may vary from nominal chunks for named entity resolution to (potentially nested) noun phrases in coreference resolution (or mentions) to larger text segments in text segmentation.
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
The corpus, containing annotations for about 108, 000 markables, is one of the largest corpora for coreference for English, and one of the largest crowdsourced NLP corpora, but its main feature is the large number of judgments per markable: 20 on average, and over 2. 2M in total.
A Probabilistic Annotation Model for Crowdsourcing Coreference
The availability of large scale annotated corpora for coreference is essential to the development of the field.
Comparing Bayesian Models of Annotation
We evaluate these models along four aspects: comparison to gold labels, predictive accuracy for new annotations, annotator characterization, and item difficulty, using four datasets with varying degrees of noise in the form of random (spammy) annotators.
