Vision Transformers
Data-efficient Image Transformer
Introduced by Touvron et al. in Training data-efficient image transformers & distillation through attentionA Data-Efficient Image Transformer is a type of Vision Transformer for image classification tasks. The model is trained using a teacher-student strategy specific to transformers. It relies on a distillation token ensuring that the student learns from the teacher through attention.
Source: Training data-efficient image transformers & distillation through attention
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Image Classification | 17 | 29.31% |
| Object Detection | 7 | 12.07% |
| Semantic Segmentation | 6 | 10.34% |
| Document Image Classification | 3 | 5.17% |
| Document Layout Analysis | 3 | 5.17% |
| Self-Supervised Learning | 3 | 5.17% |
| Fine-Grained Image Classification | 3 | 5.17% |
| Instance Segmentation | 2 | 3.45% |
| Self-Supervised Image Classification | 2 | 3.45% |
Attention Dropout
Dropout
Feedforward Network
Multi-Head Attention
Scaled Dot-Product Attention
