Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition

In this paper, we study differentiable neural architecture search (NAS) methods for natural language processing. In particular, we improve differentiable architecture search by removing the softmax-local constraint. Also, we apply differentiable NAS to named entity recognition (NER). It is the first time that differentiable NAS methods are adopted in NLP tasks other than language modeling. On both the PTB language modeling and CoNLL-2003 English NER data, our method outperforms strong baselines. It achieves a new state-of-the-art on the NER task.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Benchmark
Named Entity Recognition (NER) CoNLL 2003 (English) I-DARTS + Flair F1 93.47 # 16
Language Modelling PTB Diagnostic ECG Database I-DARTS PPL 56.0 # 1

Methods