Feedforward Networks

Position-Wise Feed-Forward Layer

Introduced by Vaswani et al. in Attention Is All You Need

Position-Wise Feed-Forward Layer is a type of feedforward layer consisting of two dense layers that applies to the last dimension, which means the same dense layers are used for each position item in the sequence, so called position-wise.

Source: Attention Is All You Need


Paper Code Results Date Stars


Task Papers Share
Machine Translation 49 6.96%
Language Modelling 46 6.53%
Image Classification 27 3.84%
Object Detection 27 3.84%
Semantic Segmentation 24 3.41%
Time Series 18 2.56%
Text Generation 16 2.27%
Speech Recognition 14 1.99%
Question Answering 14 1.99%