Search Results for author:
Found 5 papers, 2 papers with code
Re-train or Train from Scratch? Comparing Pre-training Strategies of BERT in the Medical Domain
BERT models used in specialized domains all seem to be the result of a simple strategy: initializing with the original BERT and then resuming pre-training on a specialized corpus.
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters
Due to the compelling improvements brought by BERT, many recent representation models adopted the Transformer architecture as their main building block, consequently inheriting the wordpiece tokenization system despite it not being intrinsically linked to the notion of Transformers.
Ranked #1 on
Semantic Similarity
on ClinicalSTS
Clinical Concept Extraction
Drug–drug Interaction Extraction
+3
R\'e-entra\^\iner ou entra\^\iner soi-m\^eme ? Strat\'egies de pr\'e-entra\^\inement de BERT en domaine m\'edical (Re-train or train from scratch ? Pre-training strategies for BERT in the medical domain )
Les mod{\`e}les BERT employ{\'e}s en domaine sp{\'e}cialis{\'e} semblent tous d{\'e}couler d{'}une strat{\'e}gie assez simple : utiliser le mod{\`e}le BERT originel comme initialisation puis poursuivre l{'}entra{\^\i}nement de celuici sur un corpus sp{\'e}cialis{\'e}.
Embedding Strategies for Specialized Domains: Application to Clinical Entity Recognition
Using pre-trained word embeddings in conjunction with Deep Learning models has become the {``}de facto{''} approach in Natural Language Processing (NLP).
Ranked #4 on
Clinical Concept Extraction
on 2010 i2b2/VA
