Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

Obstacles hindering the development of capsule networks for challenging NLP applications include poor scalability to large output spaces and less reliable routing processes. In this paper, we introduce: 1) an agreement score to evaluate the performance of routing processes at instance level; 2) an adaptive optimizer to enhance the reliability of routing; 3) capsule compression and partial routing to improve the scalability of capsule networks. We validate our approach on two NLP tasks, namely: multi-label text classification and question answering. Experimental results show that our approach considerably improves over strong competitors on both tasks. In addition, we gain the best results in low-resource settings with few training instances.

PDF Abstract ACL 2019 PDF ACL 2019 Abstract

Datasets


Results from the Paper


 Ranked #1 on Text Classification on RCV1 (P@1 metric)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Multi-Label Text Classification EUR-Lex NLP-Cap nDCG@5 68.8 # 2
P@1 80.2 # 1
P@3 65.48 # 1
P@5 52.83 # 2
nDCG@3 71.11 # 1
nDCG@1 80.2 # 1
Text Classification RCV1 NLP-Cap P@1 97.05 # 1
P@3 81.27 # 1
P@5 56.33 # 1
nDCG@1 97.05 # 1
nDCG@3 92.47 # 1
nDCG@5 93.11 # 1
Question Answering TrecQA NLP-Capsule MAP 0.7773 # 3
MRR 0.7416 # 7

Methods


No methods listed for this paper. Add relevant methods here