Transformers
SqueezeBERT
Introduced by Iandola et al. in SqueezeBERT: What can computer vision teach NLP about efficient neural networks?SqueezeBERT is an efficient architectural variant of BERT for natural language processing that uses grouped convolutions. It is much like BERT-base, but with positional feedforward connection layers implemented as convolutions, and grouped convolution for many of the layers.
Source: SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Linguistic Acceptability | 1 | 16.67% |
| Natural Language Inference | 1 | 16.67% |
| Question Answering | 1 | 16.67% |
| Semantic Textual Similarity | 1 | 16.67% |
| Sentiment Analysis | 1 | 16.67% |
| Text Classification | 1 | 16.67% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
Convolution
|
Convolutions | |
Grouped Convolution
|
Convolutions | |
Multi-Head Attention
|
Attention Modules |
