Search Results for author:
Found 22 papers, 2 papers with code
Mitigating Catastrophic Forgetting in Long Short-Term Memory Networks
Compared to state-of-the-art weight regularization methods to mitigate catastrophic forgetting, our approach is simple, effective, and enables faster learning.
Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization
Quasar convexity is a condition that allows some first-order methods to efficiently minimize a function even when the optimization landscape is non-convex.
Convergence of the Inexact Langevin Algorithm and Score-based Generative Models in KL Divergence
We study the Inexact Langevin Dynamics (ILD), Inexact Langevin Algorithm (ILA), and Score-based Generative Modeling (SGM) when utilizing estimated score functions for sampling.
Aggregation in the Mirror Space (AIMS): Fast, Accurate Distributed Machine Learning in Military Settings
Distributed machine learning (DML) can be an important capability for modern military to take advantage of data and devices distributed at multiple vantage points to adapt and learn.
Towards Understanding GD with Hard and Conjugate Pseudo-labels for Test-Time Adaptation
We consider a setting that a model needs to adapt to a new domain under distribution shifts, given that only unlabeled test samples from the new domain are accessible at test time.
Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time
When the potential $f$ is $L$-smooth and $m$-strongly convex, i. e.\ for sampling from a log-smooth and strongly log-concave target distribution $\pi$, it is known that under a constant integration time, the number of iterations that ideal HMC takes to get an $\epsilon$ Wasserstein-2 distance to the target $\pi$ is $O( \kappa \log \frac{1}{\epsilon} )$, where $\kappa := \frac{L}{m}$ is the condition number.
Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-Łojasiewicz Functions when the Non-Convexity is Averaged-Out
An additional condition needs to be satisfied for the acceleration result of HB beyond quadratics in this work, which naturally holds when the dimension is one or, more broadly, when the Hessian is diagonal.
Alternating Mirror Descent for Constrained Min-Max Games
In this paper we study two-player bilinear zero-sum games with constrained strategy spaces.
Improved analysis for a proximal algorithm for sampling
We study the proximal sampler of Lee, Shen, and Tian (2021) and obtain new convergence guarantees under weaker assumptions than strong log-concavity: namely, our results hold for (1) weakly log-concave targets, and (2) targets satisfying isoperimetric assumptions which allow for non-log-concavity.
Achieving Efficient Distributed Machine Learning Using a Novel Non-Linear Class of Aggregation Functions
In this paper, we analyze properties of the WPM and rigorously prove convergence properties of our aggregation mechanism.
The Mirror Langevin Algorithm Converges with Vanishing Bias
The Mirror Langevin Diffusion (MLD) is a sampling analogue of mirror flow in continuous time, and it has nice convergence properties under log-Sobolev or Poincare inequalities relative to the Hessian metric, as shown by Chewi et al. (2020).
Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry
We study the Proximal Langevin Algorithm (PLA) for sampling from a probability distribution $\nu = e^{-f}$ on $\mathbb{R}^n$ under isoperimetry.
Last-iterate convergence rates for min-max optimization
While classic work in convex-concave min-max optimization relies on average-iterate convergence results, the emergence of nonconvex applications such as training Generative Adversarial Networks has led to renewed interest in last-iterate convergence guarantees.
Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices
We also prove convergence guarantees in R\'enyi divergence of order $q > 1$ assuming the limit of ULA satisfies either the log-Sobolev or Poincar\'e inequality.
Accelerating Rescaled Gradient Descent: Fast Optimization of Smooth Functions
We also introduce a new first-order algorithm, called rescaled gradient descent (RGD), and show that RGD achieves a faster convergence rate than gradient descent provided the function is strongly smooth -- a natural generalization of the standard smoothness assumption on the objective function.
Optimization and Control
Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem
We show SLA is in fact consistent for Gaussian target measure, whereas ULA is not.
A Variational Perspective on Accelerated Methods in Optimization
We show that there is a Lagrangian functional that we call the \emph{Bregman Lagrangian} which generates a large class of accelerated methods in continuous time, including (but not limited to) accelerated gradient descent, its non-Euclidean extension, and accelerated higher-order gradient methods.
Concavity of reweighted Kikuchi approximation
We establish sufficient conditions for the concavity of our reweighted objective function in terms of weight assignments in the Kikuchi expansion, and show that a reweighted version of the sum product algorithm applied to the Kikuchi region graph will produce global optima of the Kikuchi approximation whenever the algorithm converges.
Optimal rates for zero-order convex optimization: the power of two function evaluations
We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients.
How to Hedge an Option Against an Adversary: Black-Scholes Pricing is Minimax Optimal
We consider a popular problem in finance, option pricing, through the lens of an online learning game between Nature and an Investor.
Streaming Variational Bayes
We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior.
Finite Sample Convergence Rates of Zero-Order Stochastic Optimization Methods
We consider derivative-free algorithms for stochastic optimization problems that use only noisy function values rather than gradients, analyzing their finite-sample convergence rates.
