Search Results for author:
Found 22 papers, 8 papers with code
Linear Recursive Feature Machines provably recover low-rank matrices
A possible explanation is that common training algorithms for neural networks implicitly perform dimensionality reduction - a process called feature learning.
Asymptotic normality and optimality in nonsmooth stochastic approximation
In their seminal work, Polyak and Juditsky showed that stochastic approximation algorithms for solving smooth equations enjoy a central limit theorem.
Stochastic Approximation with Decision-Dependent Distributions: Asymptotic Normality and Optimality
We show that under mild assumptions, the deviation between the average iterate of the algorithm and the solution is asymptotically normal, with a covariance that clearly decouples the effects of the gradient noise and the distributional shift.
Decision-Dependent Risk Minimization in Geometrically Decaying Dynamic Environments
This paper studies the problem of expected loss minimization given a data distribution that is dependent on the decision-maker's action and evolves dynamically in time according to a geometric decay process.
Flat minima generalize for low-rank matrix recovery
Empirical evidence suggests that for a variety of overparameterized nonlinear models, most notably in neural network training, the growth of the loss around a minimizer strongly impacts its performance.
Multiplayer Performative Prediction: Learning in Decision-Dependent Games
We show that under mild assumptions, the performatively stable equilibria can be found efficiently by a variety of algorithms, including repeated retraining and the repeated (stochastic) gradient method.
Active manifolds, stratifications, and convergence to local minima in nonsmooth optimization
We show that the subgradient method converges only to local minimizers when applied to generic Lipschitz continuous and subdifferentially regular functions that are definable in an o-minimal structure.
Stochastic Optimization under Distributional Drift
We consider the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself.
Escaping strict saddle points of the Moreau envelope in nonsmooth optimization
The main conclusion is that a variety of algorithms for nonsmooth optimization can escape strict saddle points of the Moreau envelope at a controlled rate.
Stochastic optimization under time drift: iterate averaging, step-decay schedules, and high probability guarantees
We consider the problem of minimizing a convex function that is evolving in time according to unknown and possibly stochastic dynamics.
Proximal methods avoid active strict saddles of weakly convex functions
We introduce a geometrically transparent strict saddle property for nonsmooth functions.
From low probability to high confidence in stochastic convex optimization
Standard results in stochastic convex optimization bound the number of samples that an algorithm needs to generate a point with small function value in expectation.
Stochastic algorithms with geometric step decay converge linearly on sharp functions
In this work, we ask whether geometric step decay similarly improves stochastic algorithms for the class of sharp nonconvex problems.
Low-rank matrix recovery with composite optimization: good conditioning and rapid convergence
The task of recovering a low-rank matrix from its noisy linear measurements plays a central role in computational science.
Composite optimization for robust blind deconvolution
The blind deconvolution problem seeks to recover a pair of vectors from a set of rank one bilinear measurements.
Graphical Convergence of Subgradients in Nonconvex Optimization and Learning
We investigate the stochastic optimization problem of minimizing population risk, where the loss defining the risk is assumed to be weakly convex.
Stochastic model-based minimization under high-order growth
Given a nonsmooth, nonconvex minimization problem, we consider algorithms that iteratively sample and minimize stochastic convex models of the objective function.
Stochastic subgradient method converges on tame functions
This work considers the question: what convergence guarantees does the stochastic subgradient method have in the absence of smoothness and convexity?
Stochastic model-based minimization of weakly convex functions
We consider a family of algorithms that successively sample and minimize simple stochastic models of the objective function.
Stochastic subgradient method converges at the rate $O(k^{-1/4})$ on weakly convex functions
We prove that the proximal stochastic subgradient method, applied to a weakly convex problem, drives the gradient of the Moreau envelope to zero at the rate $O(k^{-1/4})$.
The many faces of degeneracy in conic optimization
Slater's condition -- existence of a "strictly feasible solution" -- is a common assumption in conic optimization.
Optimization and Control
Catalyst Acceleration for Gradient-Based Non-Convex Optimization
We introduce a generic scheme to solve nonconvex optimization problems using gradient-based algorithms originally designed for minimizing convex functions.
