BioMegatron: Larger Biomedical Domain Language Model

There has been an influx of biomedical domain-specific language models, showing language models pre-trained on biomedical text perform better on biomedical domain benchmarks than those trained on general domain text corpora such as Wikipedia and Books. Yet, most works do not study the factors affecting each domain language application deeply... (read more)

PDF Abstract EMNLP 2020 PDF EMNLP 2020 Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Named Entity Recognition BC5CDR-chemical BioMegatron F1 92.9 # 3
Named Entity Recognition BC5CDR-disease BioMegatron F1 88.5 # 1
Relation Extraction ChemProt BioMegatron F1 77.0 # 2
Named Entity Recognition NCBI-disease BioMegatron BERT-cased F1 87.8 # 6

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet