Multi-Task Deep Neural Networks for Natural Language Understanding

ACL 2019 Xiaodong LiuPengcheng HeWeizhu ChenJianfeng Gao

In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks. MT-DNN not only leverages large amounts of cross-task data, but also benefits from a regularization effect that leads to more general representations in order to adapt to new tasks and domains... (read more)

PDF Abstract

Evaluation results from the paper


Task Dataset Model Metric name Metric value Global rank Compare
Linguistic Acceptability CoLA MT-DNN Accuracy 68.4% # 1
Natural Language Inference MultiNLI MT-DNN Matched 86.7 # 4
Natural Language Inference MultiNLI MT-DNN Mismatched 86.0 # 4
Paraphrase Identification Quora Question Pairs MT-DNN Accuracy 89.6 # 1
Natural Language Inference SciTail MT-DNN Accuracy 94.1 # 1
Natural Language Inference SNLI MT-DNN % Test Accuracy 91.6 # 1
Natural Language Inference SNLI MT-DNN % Train Accuracy 97.2 # 3
Natural Language Inference SNLI MT-DNN Parameters 330m # 1
Sentiment Analysis SST-2 Binary classification MT-DNN Accuracy 95.6 # 4