Dice Loss for Data-imbalanced NLP Tasks

7 Nov 2019Xiaoya LiXiaofei SunYuxian MengJunjun LiangFei WuJiwei Li

Many NLP tasks such as tagging and machine reading comprehension are faced with the severe data imbalance issue: negative examples significantly outnumber positive examples, and the huge number of background examples (or easy-negative examples) overwhelms the training. The most commonly used cross entropy (CE) criteria is actually an accuracy-oriented objective, and thus creates a discrepancy between training and test: at training time, each training instance contributes equally to the objective function, while at test time F1 score concerns more about positive examples... (read more)

PDF Abstract

Evaluation Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
COMPARE
Named Entity Recognition CoNLL 2003 (English) BERT-MRC+DSC F1 93.33 # 5
Named Entity Recognition Ontonotes v5 (English) BERT-MRC+DSC F1 92.07 # 1