Search Results for author:
Found 9 papers, 0 papers with code
Convergence of variational Monte Carlo simulation and scale-invariant pre-training
We provide theoretical convergence bounds for the variational Monte Carlo (VMC) method as applied to optimize neural network wave functions for the electronic structure problem.
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime
Finding the optimal configuration of parameters in ResNet is a nonconvex minimization problem, but first-order methods nevertheless find the global optimum in the overparameterized regime.
Overparameterization of deep ResNet: zero loss and mean-field analysis
Finding parameters in a deep neural network (NN) that fit training data is a nonconvex optimization problem, but a basic first-order optimization method (gradient descent) finds a global optimizer with perfect fit (zero-loss) in many practical situations.
Constrained Ensemble Langevin Monte Carlo
In particular, we find that if one directly surrogates the gradient using the ensemble approximation, the algorithm, termed Ensemble Langevin Monte Carlo, is unstable due to a high variance term.
Random Coordinate Underdamped Langevin Monte Carlo
We investigate the computational complexity of RC-ULMC and compare it with the classical ULMC for strongly log-concave probability distributions.
Random Coordinate Langevin Monte Carlo
We investigate the total complexity of RC-LMC and compare it with the classical LMC for log-concave probability distributions.
Langevin Monte Carlo: random coordinate descent and variance reduction
However, the method requires the evaluation of a full gradient in each iteration, and for a problem on $\mathbb{R}^d$, this amounts to $d$ times partial derivative evaluations per iteration.
Variance reduction for Random Coordinate Descent-Langevin Monte Carlo
The high variance induced by the randomness means a larger number of iterations are needed, and this balances out the saving in each iteration.
Error Lower Bounds of Constant Step-size Stochastic Gradient Descent
To our knowledge, this is the first analysis for SGD error lower bound without the strong convexity assumption.
