Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the likelihood of $\log{p}\left(y\mid{x}\right)$ directly can be harmful. Assume for a small constant $\epsilon$, the training set label $y$ is correct with probability $1-\epsilon$ and incorrect otherwise. Label Smoothing regularizes a model based on a softmax with $k$ output values by replacing the hard $0$ and $1$ classification targets with targets of $\frac{\epsilon}{k}$ and $1-\frac{k-1}{k}\epsilon$ respectively.
Source: Deep Learning, Goodfellow et al
Image Source: When Does Label Smoothing Help?
Paper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Large Language Model | 23 | 2.79% |
Computational Efficiency | 23 | 2.79% |
Language Modeling | 21 | 2.55% |
Language Modelling | 21 | 2.55% |
Decoder | 19 | 2.31% |
Question Answering | 15 | 1.82% |
Semantic Segmentation | 14 | 1.70% |
Image Generation | 13 | 1.58% |
Image Classification | 12 | 1.46% |
Component | Type |
|
---|---|---|
🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |