Activation Functions

Hard Swish

Introduced by Howard et al. in Searching for MobileNetV3

Hard Swish is a type of activation function based on Swish, but replaces the computationally expensive sigmoid with a piecewise linear analogue:

$$\text{h-swish}\left(x\right) = x\frac{\text{ReLU6}\left(x+3\right)}{6} $$

Source: Searching for MobileNetV3

Papers


Paper Code Results Date Stars

Components

Component Type
ReLU6
Activation Functions

Categories