Output Functions
Sparsemax
Introduced by Martins et al. in From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label ClassificationSparsemax is a type of activation/output function similar to the traditional softmax, but able to output sparse probabilities.
$$ \text{sparsemax}\left(z\right) = \arg_{p∈\Delta^{K−1}}\min||\mathbf{p} - \mathbf{z}||^{2} $$
Source: From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Machine Translation | 4 | 7.02% |
| Translation | 4 | 7.02% |
| Language Modeling | 3 | 5.26% |
| Language Modelling | 3 | 5.26% |
| General Classification | 3 | 5.26% |
| Question Answering | 2 | 3.51% |
| Visual Question Answering | 2 | 3.51% |
| Visual Question Answering (VQA) | 2 | 3.51% |
| Decoder | 2 | 3.51% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
