Transformers
Inverted Bottleneck BERT
Introduced by Sun et al. in MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited DevicesIB-BERT, or Inverted Bottleneck BERT, is a BERT variant that uses an inverted bottleneck structure. It is used as a teacher network to train the MobileBERT models.
Source: MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Natural Language Inference | 1 | 33.33% |
| Question Answering | 1 | 33.33% |
| Semantic Textual Similarity | 1 | 33.33% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
