Search Results for author:
Found 14 papers, 1 papers with code
Double-Loop Unadjusted Langevin Algorithm
A well-known first-order method for sampling from log-concave probability distributions is the Unadjusted Langevin Algorithm (ULA).
Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
We theoretically analyze the impact of fine-tuning with our method in a linear setting, showing that it stalls learning in a certain subspace which inhibits overfitting to the target task.
Understanding Contrastive Learning via Gaussian Mixture Models
Contrastive learning attempts to learn representations from un-labeled data; it does so via a loss function that encourages the embedding of a point to be close to that of its augmentations, and far from the embeddings of random other points.
Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization
We propose adaptive, line search-free second-order methods with optimal rate of convergence for solving convex-concave min-max problems.
Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization
We propose an adaptive variance-reduction method, called AdaSpider, for minimization of $L$-smooth, non-convex functions with a finite-sum structure.
Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods
This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions.
High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize
We present our analysis in a modular way and obtain a complementary $\mathcal O (1 / T)$ convergence rate in the deterministic setting.
Sifting through the noise: Universal first-order methods for stochastic variational inequalities
Our first result is that the algorithm achieves the optimal rates of convergence for cocoercive problems when the profile of the randomness is known to the optimizer: $\mathcal{O}(1/\sqrt{T})$ for absolute noise profiles, and $\mathcal{O}(1/T)$ for relative ones.
STORM+: Fully Adaptive SGD with Recursive Momentum for Nonconvex Optimization
In this work we propose $\rm{STORM}^{+}$, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.
STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization
In this work we propose STORM+, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems
This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's convergence properties in non-convex problems.
UniXGrad: A Universal, Adaptive Algorithm with Optimal Guarantees for Constrained Optimization
To the best of our knowledge, this is the first adaptive, unified algorithm that achieves the optimal rates in the constrained setting.
Efficient learning of smooth probability functions from Bernoulli tests with guarantees
We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests.
Mirrored Langevin Dynamics
We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design.
