Activation Functions

Leaky ReLU

Leaky Rectified Linear Unit, or Leaky ReLU, is a type of activation function based on a ReLU, but it has a small slope for negative values instead of a flat slope. The slope coefficient is determined before training, i.e. it is not learnt during training. This type of activation function is popular in tasks where we we may suffer from sparse gradients, for example training generative adversarial networks.


Paper Code Results Date Stars


Task Papers Share
Image Generation 78 11.96%
Image-to-Image Translation 57 8.74%
Super-Resolution 33 5.06%
Domain Adaptation 26 3.99%
Semantic Segmentation 22 3.37%
Image Super-Resolution 19 2.91%
Object Detection 16 2.45%
Style Transfer 14 2.15%
Voice Conversion 13 1.99%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign