Feedforward Networks
Position-Wise Feed-Forward Layer
Introduced by Vaswani et al. in Attention Is All You NeedPosition-Wise Feed-Forward Layer is a type of feedforward layer consisting of two dense layers that applies to the last dimension, which means the same dense layers are used for each position item in the sequence, so called position-wise.
Source: Attention Is All You NeedPapers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Language Modelling | 45 | 5.60% |
| Semantic Segmentation | 34 | 4.23% |
| Large Language Model | 30 | 3.73% |
| Object Detection | 22 | 2.74% |
| Decision Making | 19 | 2.36% |
| Question Answering | 13 | 1.62% |
| Retrieval | 12 | 1.49% |
| Image Segmentation | 12 | 1.49% |
| Denoising | 11 | 1.37% |
Dense Connections
