Feedforward Networks

Edit

General • 16 methods

Feedforward Networks are a type of neural network architecture which rely primarily on dense-like connections. Below you can find a continuously updating list of feedforward network components.

Methods

Add a Method

Method	Year	Papers
Dense Connections	2000	25672
Linear Layer	2000	22235
Position-Wise Feed-Forward Layer Attention Is All You Need	2017	12194
Feedforward Network	2000	1261
Adapter Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing	2021	850
HyperNetwork HyperNetworks	2016	201
Highway Network Highway Networks	2015	90
Mix-FFN SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers	2021	41
Switch FFN Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity	2021	17
Affine Operator ResMLP: Feedforward networks for image classification with data-efficient training	2021	12
Spatial Gating Unit Pay Attention to MLPs	2021	9
Boom Layer Single Headed Attention RNN: Stop Thinking With Your Head	2019	2
ALCN An Adaptive Locally Connected Neuron Model: Focusing Neuron	2018	2
DExTra DeLighT: Deep and Light-weight Transformer	2020	1
VarImpVIANN Variance-Based Feature Importance in Neural Networks	2019	1
NLSIG	2000	1