Applying Citizen Science to Gene, Drug, Disease Relationship Extraction from Biomedical Abstracts

no code implementations • 17 Nov 2018 • Ginger Tsueng, Max Nanis, Jennifer T. Fouquier, Michael Mayers, Benjamin M. Good, Andrew I Su

Here, we used the web-based application Mark2Cure (https://mark2cure. org) to demonstrate that citizen scientists can perform relationship extraction and confirm the importance of accurate named entity recognition on this task.

named-entity-recognition Named Entity Recognition +1

Exposing ambiguities in a relation-extraction gold standard with crowdsourcing

1 code implementation • 23 May 2015 • Tong Shu Li, Benjamin M. Good, Andrew I. Su

This work rein-forces the power of crowdsourcing in the process of assembling gold standards for relation extraction.

Relation Relation Extraction

OntoLoki: an automatic, instance-based method for the evaluation of biological ontologies on the Semantic Web

no code implementations • 20 Feb 2015 • Benjamin M. Good, Gavin Ha, Chi K. Ho, Mark D. Wilkinson

The delineation of logical definitions for each class in an ontology and the consistent application of these definitions to the assignment of instances to classes are important criteria for ontology evaluation.

Microtask crowdsourcing for disease mention annotation in PubMed abstracts

no code implementations • 8 Aug 2014 • Benjamin M. Good, Max Nanis, Andrew I. Su

After several iterations, we arrived at a protocol that reproduced the annotations of the 593 documents in the training set of this gold standard with an overall F measure of 0. 872 (precision 0. 862, recall 0. 883).

Benchmarking