Search Results for author: Kai-Xin Gao

Found 3 papers, 1 papers with code

Eigenvalue-corrected Natural Gradient Based on a New Approximation

no code implementations27 Nov 2020 Kai-Xin Gao, Xiao-Lei Liu, Zheng-Hai Huang, Min Wang, Shuangling Wang, Zidong Wang, Dachuan Xu, Fan Yu

Using second-order optimization methods for training deep neural networks (DNNs) has attracted many researchers.

A Trace-restricted Kronecker-Factored Approximation to Natural Gradient

no code implementations21 Nov 2020 Kai-Xin Gao, Xiao-Lei Liu, Zheng-Hai Huang, Min Wang, Zidong Wang, Dachuan Xu, Fan Yu

There have been many attempts to use second-order optimization methods for training deep neural networks.

EAdam Optimizer: How $ε$ Impact Adam

2 code implementations4 Nov 2020 Wei Yuan, Kai-Xin Gao

Recently, many variants of Adam, such as Adabound, RAdam and Adabelief, have been proposed and show better performance than Adam.

Cannot find the paper you are looking for? You can Submit a new open access paper.