SqueezeBERT: What can computer vision teach NLP about efficient neural networks?

Humans read and write hundreds of billions of messages every day. Further, due to the availability of large datasets, large computing systems, and better neural network models, natural language processing (NLP) technology has made significant strides in understanding, proofreading, and organizing these messages... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Linguistic Acceptability CoLA SqueezeBERT Accuracy 46.5% # 16
Natural Language Inference MultiNLI SqueezeBERT Matched 82.0 # 16
Mismatched 81.1 # 13
Natural Language Inference RTE SqueezeBERT Accuracy 73.2% # 13
Sentiment Analysis SST-2 Binary classification SqueezeBERT Accuracy 91.4 # 19

Methods used in the Paper