Activation Functions

Rectified Linear Units

Rectified Linear Units, or ReLUs, are a type of activation function that are linear in the positive dimension, but zero in the negative dimension. The kink in the function is the source of the non-linearity. Linearity in the positive dimension has the attractive property that it prevents non-saturation of gradients (contrast with sigmoid activations), although for half of the real line its gradient is zero.

$$ f\left(x\right) = \max\left(0, x\right) $$

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Semantic Segmentation 50 7.96%
Image Generation 33 5.25%
Image Segmentation 24 3.82%
Image Classification 23 3.66%
Denoising 19 3.03%
Classification 15 2.39%
Self-Supervised Learning 13 2.07%
Object Detection 13 2.07%
Tumor Segmentation 11 1.75%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories