Feedforward Networks

Dense Connections

Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output by a weight. This means there are $n_{\text{inputs}}*n_{\text{outputs}}$ parameters, which can lead to a lot of parameters for a sizeable network.

$$h_{l} = g\left(\textbf{W}^{T}h_{l-1}\right)$$

where $g$ is an activation function.

Image Source: Deep Learning by Goodfellow, Bengio and Courville


Paper Code Results Date Stars


Task Papers Share
Language Modelling 66 8.51%
Retrieval 33 4.25%
Question Answering 30 3.87%
Large Language Model 26 3.35%
Sentence 17 2.19%
In-Context Learning 16 2.06%
Text Generation 15 1.93%
Semantic Segmentation 13 1.68%
Machine Translation 11 1.42%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign