Activation Functions

Rectified Linear Units

Rectified Linear Units, or ReLUs, are a type of activation function that are linear in the positive dimension, but zero in the negative dimension. The kink in the function is the source of the non-linearity. Linearity in the positive dimension has the attractive property that it prevents non-saturation of gradients (contrast with sigmoid activations), although for half of the real line its gradient is zero.

$$ f\left(x\right) = \max\left(0, x\right) $$


Paper Code Results Date Stars


Task Papers Share
Semantic Segmentation 42 7.84%
Image Classification 24 4.48%
Object Detection 23 4.29%
General Classification 17 3.17%
Image Generation 16 2.99%
Super-Resolution 14 2.61%
Medical Image Segmentation 13 2.43%
Denoising 13 2.43%
Domain Adaptation 12 2.24%


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign