Activation Functions


Introduced by Goodfellow et al. in Maxout Networks

The Maxout Unit is a generalization of the ReLU and the leaky ReLU functions. It is a piecewise linear function that returns the maximum of the inputs, designed to be used in conjunction with dropout. Both ReLU and leaky ReLU are special cases of Maxout.

$$f\left(x\right) = \max\left(w^{T}_{1}x + b_{1}, w^{T}_{2}x + b_{2}\right)$$

The main drawback of Maxout is that it is computationally expensive as it doubles the number of parameters for each neuron.

Source: Maxout Networks


Paper Code Results Date Stars


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign