Normalization
Root Mean Square Layer Normalization
Introduced by Zhang et al. in Root Mean Square Layer NormalizationRMSNorm regularizes the summed inputs to a neuron in one layer according to root mean square (RMS), giving the model re-scaling invariance property and implicit learning rate adaptation ability. RMSNorm is computationally simpler and thus more efficient than LayerNorm.
Source: Root Mean Square Layer Normalization
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Model Compression | 1 | 5.88% |
| HellaSwag | 1 | 5.88% |
| Beat Tracking | 1 | 5.88% |
| Downbeat Tracking | 1 | 5.88% |
| Drum Transcription | 1 | 5.88% |
| Multi-instrument Music Transcription | 1 | 5.88% |
| Multi-Task Learning | 1 | 5.88% |
| Music Transcription | 1 | 5.88% |
| Quantization | 1 | 5.88% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
