Learning Activation Functions to Improve Deep Neural Networks

21 Dec 2014Forest AgostinelliMatthew HoffmanPeter SadowskiPierre Baldi

Artificial neural networks typically have a fixed, non-linear activation function at each neuron. We have designed a novel form of piecewise linear activation function that is learned independently for each neuron using gradient descent... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Image Classification CIFAR-10 NiN+APL Percentage correct 92.5 # 52
Image Classification CIFAR-100 NiN+APL Percentage correct 69.2 # 49

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet