Activation Functions

Sigmoid Activation

Sigmoid Activations are a type of activation function for neural networks:

$$f\left(x\right) = \frac{1}{\left(1+\exp\left(-x\right)\right)}$$

Some drawbacks of this activation that have been noted in the literature are: sharp damp gradients during backpropagation from deeper hidden layers to inputs, gradient saturation, and slow convergence.


Paper Code Results Date Stars


Task Papers Share
Language Modelling 22 3.16%
Sentence 19 2.73%
Classification 18 2.59%
Image Classification 16 2.30%
Management 16 2.30%
Time Series Forecasting 15 2.16%
Translation 14 2.01%
Object Detection 13 1.87%
Image Generation 13 1.87%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign