no code implementations • 26 Jul 2020 • Renlong Jie, Junbin Gao, Andrey Vasnev, Min-ngoc Tran
Based on this, a novel family of flexible activation functions that can replace sigmoid or tanh in LSTM cells are implemented, as well as a new family by combining ReLU and ELUs.