Search Results for author:
Found 32 papers, 14 papers with code
Armijo Line-search Makes (Stochastic) Gradient Descent Go Fast
For smooth functions, Armijo-LS alleviates the need to know the global smoothness constant $L$ and adapts to the local smoothness, enabling GD to converge faster.
Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates
The proofs are based on novel findings about action sampling rates and the relationship between cumulative progress and noise, and extend the current understanding of how simple stochastic gradient methods behave in bandit settings.
Improving OOD Generalization of Pre-trained Encoders via Aligned Embedding-Space Ensembles
The quality of self-supervised pre-trained embeddings on out-of-distribution (OOD) data is poor without fine-tuning.
Fast Convergence of Softmax Policy Mirror Ascent
Natural policy gradient (NPG) is a common policy optimization algorithm and can be viewed as mirror ascent in the space of probabilities.
Towards Principled, Practical Policy Gradient for Bandits and Tabular MDPs
We show that the proposed algorithm offers similar theoretical guarantees as the state-of-the art results, but does not require the knowledge of oracle-like quantities.
From Inverse Optimization to Feasibility to ERM
Inverse optimization involves inferring unknown parameters of an optimization problem from known solutions and is widely used in fields such as transportation, power systems, and healthcare.
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
Specifically, we prove that with the same step-size and momentum parameters as in the deterministic setting, SHB with a sufficiently large mini-batch size results in an $O\left(\exp(-\frac{T}{\sqrt{\kappa}}) + \sigma \right)$ convergence, where $T$ is the number of iterations and $\sigma^2$ is the variance in the stochastic gradients.
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Instantiating the generic algorithm results in an actor that involves maximizing a sequence of surrogate functions (similar to TRPO, PPO) and a critic that involves minimizing a closely connected objective.
Target-based Surrogates for Stochastic Optimization
Our target optimization framework uses the (expensive) gradient computation to construct surrogate functions in a \emph{target space} (e. g. the logits output by a linear model for classification) that can be minimized efficiently.
Improved Policy Optimization for Online Imitation Learning
Specifically, if the class of policies is sufficiently expressive to contain the expert policy, we prove that DAGGER achieves constant regret.
Near-Optimal Sample Complexity Bounds for Constrained MDPs
In particular, we design a model-based algorithm that addresses two settings: (i) relaxed feasibility, where small constraint violations are allowed, and (ii) strict feasibility, where the output policy is required to satisfy the constraint.
Towards Painless Policy Optimization for Constrained MDPs
We propose a generic primal-dual framework that allows us to bound the reward sub-optimality and constraint violation for arbitrary algorithms in terms of their primal and dual regret on online linear optimization problems.
Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
In order to be adaptive to the smoothness, we use a stochastic line-search (SLS) and show (via upper and lower-bounds) that SGD with SLS converges at the desired rate, but only to a neighbourhood of the solution.
A general class of surrogate functions for stable and efficient reinforcement learning
Common policy gradient methods rely on the maximization of a sequence of surrogate functions.
SVRG Meets AdaGrad: Painless Variance Reduction
Variance reduction (VR) methods for finite-sum minimization typically require the knowledge of problem-dependent constants that are often unknown and difficult to estimate.
Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search)
Under an interpolation assumption, we prove that AMSGrad with a constant step-size and momentum can converge to the minimizer at the faster $O(1/T)$ rate for smooth, convex functions.
Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)
In this setting, we prove that AMSGrad with constant step-size and momentum converges to the minimizer at a faster $O(1/T)$ rate.
To Each Optimizer a Norm, To Each Norm its Generalization
For under-parameterized linear classification, we prove that for any linear classifier separating the data, there exists a family of quadratic norms ||.||_P such that the classifier's direction is the same as that of the maximum P-margin solution.
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence
Consequently, the proposed stochastic Polyak step-size (SPS) is an attractive choice for setting the learning rate for stochastic gradient descent (SGD).
Old Dog Learns New Tricks: Randomized UCB for Bandit Problems
We propose $\tt RandUCB$, a bandit strategy that builds on theoretically derived confidence intervals similar to upper confidence bound (UCB) algorithms, but akin to Thompson sampling (TS), it uses randomization to trade off exploration and exploitation.
Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation
Under this condition, we show that the regularized subsampled Newton method (R-SSN) achieves global linear convergence with an adaptive step-size and a constant batch-size.
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
To improve the proposed methods' practical performance, we give heuristics to use larger step-sizes and acceleration.
Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Bandits
Specifically, it pulls the arm with the highest mean reward in a non-parametric bootstrap sample of its history with pseudo rewards.
Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron
Under this condition, we prove that constant step-size stochastic gradient descent (SGD) with Nesterov acceleration matches the convergence rate of the deterministic accelerated method for both convex and strongly-convex functions.
Combining Bayesian Optimization and Lipschitz Optimization
Indeed, in a particular setting, we prove that using the Lipschitz information yields the same or a better bound on the regret compared to using Bayesian optimization on its own.
New Insights into Bootstrapping for Bandits
We investigate the use of bootstrapping in the bandit setting.
Horde of Bandits using Gaussian Markov Random Fields
The gang of bandits (GOB) model \cite{cesa2013gang} is a recent contextual bandits framework that shares information between a set of bandit problems, related by a known (possibly noisy) graph.
Model-Independent Online Learning for Influence Maximization
We consider influence maximization (IM) in social networks, which is the problem of maximizing the number of users that become aware of a product by selecting a set of "seed" users to expose the product to.
Online Influence Maximization under Independent Cascade Model with Semi-Bandit Feedback
Specifically, we aim to learn the set of "best influencers" in a social network online while repeatedly interacting with it.
Adaptive Influence Maximization in Social Networks: Why Commit when You can Adapt?
A disadvantage of this setting is that the marketer is forced to select all the seeds based solely on a diffusion model.
Social and Information Networks
Influence Maximization with Bandits
We consider the problem of \emph{influence maximization}, the problem of maximizing the number of people that become aware of a product by finding the `best' set of `seed' users to expose the product to.
Fast 3D Salient Region Detection in Medical Images using GPUs
This allows us to initialize a sparse seed-point grid as the set of tentative salient region centers and iteratively converge to the local entropy maxima, thereby reducing the computation complexity compared to the Kadir Brady approach of performing this computation at every point in the image.
