Ontology-driven weak supervision for clinical entity classification in electronic health records

In the electronic health record, using clinical notes to identify entities such as disorders and their temporality (e.g. the order of an event relative to a time index) can inform many important analyses. However, creating training data for clinical entity tasks is time consuming and sharing labeled data is challenging due to privacy concerns. The information needs of the COVID-19 pandemic highlight the need for agile methods of training machine learning models for clinical notes. We present Trove, a framework for weakly supervised entity classification using medical ontologies and expert-generated rules. Our approach, unlike hand-labeled notes, is easy to share and modify, while offering performance comparable to learning from manually labeled training data. In this work, we validate our framework on six benchmark tasks and demonstrate Trove's ability to analyze the records of patients visiting the emergency department at Stanford Health Care for COVID-19 presenting symptoms and risk factors.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Weakly-Supervised Named Entity Recognition BC5CDR Trove Precision 85.5 # 2
Recall 86.8 # 1
F1 86.1 # 1
Weakly-Supervised Named Entity Recognition BC5CDR-chemical Trove F1 91 # 1
Weakly-Supervised Named Entity Recognition BC5CDR-disease Trove F1 80.1 # 1
Weakly Supervised Classification ShARe/CLEF 2014: Task 2 Disorders Trove F1 92.7 # 1
Weakly-Supervised Named Entity Recognition ShARe/CLEF 2014: Task 2 Disorders Trove F1 76.3 # 1
Weakly Supervised Classification THYME-2016 Trove F1 72.9 # 1