Search Results for author:
Found 8 papers, 1 papers with code
Effective Subspace Indexing via Interpolation on Stiefel and Grassmann manifolds
We propose a novel local Subspace Indexing Model with Interpolation (SIM-I) for low-dimensional embedding of image datasets.
Efficient Smooth Non-Convex Stochastic Compositional Optimization via Stochastic Recursive Gradient Descent
Stochastic compositional optimization arises in many important machine learning tasks such as reinforcement learning and portfolio management.
On the Noisy Gradient Descent that Generalizes as SGD
The gradient noise of SGD is considered to play a central role in the observed strong generalization abilities of deep learning.
SHE2: Stochastic Hamiltonian Exploration and Exploitation for Derivative-Free Optimization
Derivative-free optimization (DFO) using trust region methods is frequently used for machine learning applications, such as (hyper-)parameter optimization without the derivatives of objective functions known.
Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent
We show in this case that the quasi-potential function is related to the noise covariance structure of SGD via a partial differential equation of Hamilton-Jacobi type.
A convergence analysis of the perturbed compositional gradient flow: averaging principle and normal deviations
By introducing a separation of fast and slow scales of the two equations, we show that the limit of the slow motion is given by an averaged ordinary differential equation.
On the diffusion approximation of nonconvex stochastic gradient descent
In addition, we discuss the effects of batch size for the deep neural networks, and we find that small batch size is helpful for SGD algorithms to escape unstable stationary points and sharp minimizers.
FWDA: a Fast Wishart Discriminant Analysis with its Application to Electronic Health Records Data Classification
Classical LDA for EHR data classification, however, suffers from two handicaps: the ill-posed estimation of LDA parameters (e. g., covariance matrix), and the "linear inseparability" of EHR data.
