Feedforward Networks

Edit

General • 16 methods

Feedforward Networks are a type of neural network architecture which rely primarily on dense-like connections. Below you can find a continuously updating list of feedforward network components.

Methods

Add a Method

Method	Year	Papers
Dense Connections	2000	18079
Linear Layer	2000	15006
Position-Wise Feed-Forward Layer Attention Is All You Need	2017	7709
Feedforward Network	2000	1025
Adapter Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing	2021	366
HyperNetwork HyperNetworks	2016	143
Highway Network Highway Networks	2015	78
Mix-FFN SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers	2021	27
Switch FFN Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity	2021	16
Affine Operator ResMLP: Feedforward networks for image classification with data-efficient training	2021	10
Spatial Gating Unit Pay Attention to MLPs	2021	8
Boom Layer Single Headed Attention RNN: Stop Thinking With Your Head	2019	2
ALCN An Adaptive Locally Connected Neuron Model: Focusing Neuron	2018	2
DExTra DeLighT: Deep and Light-weight Transformer	2020	1
VarImpVIANN Variance-Based Feature Importance in Neural Networks	2019	1
NLSIG	2000	1