Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning

30 Aug 2022  ·  Sheng Zhang, Hao Cheng, Jianfeng Gao, Hoifung Poon ·

We present a bi-encoder framework for named entity recognition (NER), which applies contrastive learning to map candidate text spans and entity types into the same vector representation space. Prior work predominantly approaches NER as sequence labeling or span classification. We instead frame NER as a representation learning problem that maximizes the similarity between the vector representations of an entity mention and its type. This makes it easy to handle nested and flat NER alike, and can better leverage noisy self-supervision signals. A major challenge to this bi-encoder formulation for NER lies in separating non-entity spans from entity mentions. Instead of explicitly labeling all non-entity spans as the same class $\texttt{Outside}$ ($\texttt{O}$) as in most prior methods, we introduce a novel dynamic thresholding loss. Experiments show that our method performs well in both supervised and distantly supervised settings, for nested and flat NER alike, establishing new state of the art across standard datasets in the general domain (e.g., ACE2004, ACE2005) and high-value verticals such as biomedicine (e.g., GENIA, NCBI, BC5CDR, JNLPBA). We release the code at github.com/microsoft/binder.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Nested Named Entity Recognition ACE 2004 BINDER F1 88.7 # 2
Nested Named Entity Recognition ACE 2005 BINDER F1 89.5 # 1
Named Entity Recognition (NER) BC5CDR BINDER F1 91.9 # 1
Nested Named Entity Recognition GENIA BINDER F1 80.5 # 7
Named Entity Recognition (NER) JNLPBA BINDER F1 80.3 # 4