Activation Functions

Tanh Activation

Tanh Activation is an activation function used for neural networks:

$$f\left(x\right) = \frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}$$

Historically, the tanh function became preferred over the sigmoid function as it gave better performance for multi-layer neural networks. But it did not solve the vanishing gradient problem that sigmoids suffered, which was tackled more effectively with the introduction of ReLU activations.

Image Source: Junxi Feng

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Time Series 52 8.18%
Object Detection 20 3.14%
Language Modelling 19 2.99%
Sentiment Analysis 15 2.36%
Time Series Forecasting 13 2.04%
Speech Recognition 13 2.04%
Machine Translation 13 2.04%
Text Classification 12 1.89%
Anomaly Detection 10 1.57%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories