Search Results for author:
Found 24 papers, 4 papers with code
Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensing
We introduce a general framework for efficiently finding an approximate SOSP with \emph{dimension-independent} accuracy guarantees, using $\widetilde{O}({D^2}/{\epsilon})$ samples where $D$ is the ambient dimension and $\epsilon$ is the fraction of corrupted datapoints.
How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization
We use this framework to obtain improved, and sometimes optimal, rates for several classes of non-convex loss functions.
Extending the Reach of First-Order Algorithms for Nonconvex Min-Max Problems with Cohypomonotonicity
With a simple argument, we obtain optimal or best-known complexity guarantees with cohypomonotonicity or weak MVI conditions for $\rho < \frac{1}{L}$.
Complexity of Single Loop Algorithms for Nonlinear Programming with Stochastic Objective and Constraints
To find a point that satisfies $\varepsilon$-approximate first-order conditions, we require $\widetilde{O}(\varepsilon^{-3})$ complexity in the first case, $\widetilde{O}(\varepsilon^{-4})$ in the second case, and $\widetilde{O}(\varepsilon^{-5})$ in the third case.
A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees
We consider minimization of a smooth nonconvex function with inexact oracle access to gradient and Hessian (without assuming access to the function value) to achieve approximate second-order optimality.
Accelerating optimization over the space of probability measures
Most research has focused on optimization over Euclidean spaces, but given the need to optimize over spaces of probability measures in many machine learning problems, it is of interest to investigate accelerated gradient methods in this context too.
Correcting auto-differentiation in neural-ODE training
Does the use of auto-differentiation yield reasonable updates to deep neural networks that represent neural ODEs?
Differentially Private Optimization for Smooth Nonconvex ERM
We develop simple differentially private optimization algorithms that move along directions of (expected) descent to find an approximate second-order solution for nonconvex ERM.
Cyclic Block Coordinate Descent With Variance Reduction for Composite Nonconvex Optimization
Our convergence analysis is based on a gradient Lipschitz condition with respect to a Mahalanobis norm, inspired by a recent progress on cyclic block coordinate methods.
On the Complexity of a Practical Primal-Dual Coordinate Method
We prove complexity bounds for the primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD), which has been shown to obtain good practical performance for solving convex-concave min-max problems with bilinear coupling.
Coordinate Linear Variance Reduction for Generalized Linear Programming
\textsc{clvr} yields improved complexity results for (GLP) that depend on the max row norm of the linear constraint matrix in (GLP) rather than the spectral norm.
Randomized Algorithms for Scientific Computing (RASC)
Randomized algorithms have propelled advances in artificial intelligence and represent a foundational research area in advancing AI for Science.
Variance Reduction via Primal-Dual Accelerated Dual Averaging for Nonsmooth Convex Finite-Sums
We study structured nonsmooth convex finite-sum optimization that appears widely in machine learning applications, including support vector machines and least absolute deviation.
Random Coordinate Underdamped Langevin Monte Carlo
We investigate the computational complexity of RC-ULMC and compare it with the classical ULMC for strongly log-concave probability distributions.
Random Coordinate Langevin Monte Carlo
We investigate the total complexity of RC-LMC and compare it with the classical LMC for log-concave probability distributions.
Adversarial Classification via Distributional Robustness with Wasserstein Ambiguity
Inspired by this observation, we show that, for a certain class of distributions, the only stationary point of the regularized ramp loss minimization problem is the global minimizer.
Interleaved Composite Quantization for High-Dimensional Similarity Search
A method often used to reduce this computational cost is quantization of the vector space and location-based encoding of the dataset vectors.
A Distributed Quasi-Newton Algorithm for Primal and Dual Regularized Empirical Risk Minimization
When applied to the distributed dual ERM problem, unlike state of the art that takes only the block-diagonal part of the Hessian, our approach is able to utilize global curvature information and is thus magnitudes faster.
A Distributed Quasi-Newton Algorithm for Empirical Risk Minimization with Nonsmooth Regularization
Initial computational results on convex problems demonstrate that our method significantly improves on communication cost and running time over the current state-of-the-art methods.
Training Set Debugging Using Trusted Items
The set of trusted items may not by itself be adequate for learning, so we propose an algorithm that uses these items to identify bugs in the training set and thus im- proves learning.
Online Algorithms for Factorization-Based Structure from Motion
We present a family of online algorithms for real-time factorization-based structure from motion, leveraging a relationship between incremental singular value decomposition and recently proposed methods for online matrix completion.
On GROUSE and Incremental SVD
GROUSE (Grassmannian Rank-One Update Subspace Estimation) is an incremental algorithm for identifying a subspace of Rn from a sequence of vectors in this subspace, where only a subset of components of each vector is revealed at each iteration.
Robust Dequantized Compressive Sensing
We consider the reconstruction problem in compressed sensing in which the observations are recorded in a finite number of bits.
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks.
