Dice Loss for Data-imbalanced NLP Tasks

7 Nov 2019Xiaoya LiXiaofei SunYuxian MengJunjun LiangFei WuJiwei Li

Many NLP tasks such as tagging and machine reading comprehension are faced with the severe data imbalance issue: negative examples significantly outnumber positive examples, and the huge number of background examples (or easy-negative examples) overwhelms the training. The most commonly used cross entropy (CE) criteria is actually an accuracy-oriented objective, and thus creates a discrepancy between training and test: at training time, each training instance contributes equally to the objective function, while at test time F1 score concerns more about positive examples... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT LEADERBOARD
Named Entity Recognition CoNLL 2003 (English) BERT-MRC+DSC F1 93.33 # 5
Chinese Named Entity Recognition MSRA BERT-MRC+DSC F1 96.72 # 1
Chinese Named Entity Recognition OntoNotes 4 BERT-MRC+DSC F1 84.47 # 1
Named Entity Recognition Ontonotes v5 (English) BERT-MRC+DSC F1 92.07 # 1
Question Answering SQuAD1.1 dev XLNet+DSC EM 89.79 # 2
F1 95.77 # 1
Question Answering SQuAD2.0 dev XLNet+DSC F1 89.51 # 2
EM 87.65 # 2