Output Functions
Adaptive Softmax
Introduced by Grave et al. in Efficient softmax approximation for GPUsAdaptive Softmax is a speedup technique for the computation of probability distributions over words. The adaptive softmax is inspired by the class-based hierarchical softmax, where the word classes are built to minimize the computation time. Adaptive softmax achieves efficiency by explicitly taking into account the computation time of matrix-multiplication on parallel systems and combining it with a few important observations, namely keeping a shortlist of frequent words in the root node and reducing the capacity of rare words.
Source: Efficient softmax approximation for GPUs
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Language Modelling | 36 | 40.91% |
| Machine Translation | 7 | 7.95% |
| Speech Recognition | 5 | 5.68% |
| Text Generation | 3 | 3.41% |
| Automatic Speech Recognition | 3 | 3.41% |
| Abstractive Text Summarization | 2 | 2.27% |
| Paraphrase Identification | 2 | 2.27% |
| Question Answering | 2 | 2.27% |
| Sentiment Analysis | 2 | 2.27% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
