Search Results for author:
Found 16 papers, 6 papers with code
Noise-adaptive (Accelerated) Stochastic Heavy-Ball Momentum
In particular, for strongly-convex quadratics with condition number $\kappa$, we prove that SHB with the standard step-size and momentum parameters results in an $O\left(\exp(-\frac{T}{\sqrt{\kappa}}) + \sigma \right)$ convergence rate, where $T$ is the number of iterations and $\sigma^2$ is the variance in the stochastic gradients.
Fast Online Node Labeling for Very Large Graphs
This paper studies the online node classification problem under a transductive learning setting.
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Instantiating the generic algorithm results in an actor that involves maximizing a sequence of surrogate functions (similar to TRPO, PPO) and a critic that involves minimizing a closely connected objective.
Target-based Surrogates for Stochastic Optimization
Our target optimization framework uses the (expensive) gradient computation to construct surrogate functions in a \emph{target space} (e. g. the logits output by a linear model for classification) that can be minimized efficiently.
Towards Painless Policy Optimization for Constrained MDPs
We propose a generic primal-dual framework that allows us to bound the reward sub-optimality and constraint violation for arbitrary algorithms in terms of their primal and dual regret on online linear optimization problems.
Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
In order to be adaptive to the smoothness, we use a stochastic line-search (SLS) and show (via upper and lower-bounds) that SGD with SLS converges at the desired rate, but only to a neighbourhood of the solution.
SVRG Meets AdaGrad: Painless Variance Reduction
Variance reduction (VR) methods for finite-sum minimization typically require the knowledge of problem-dependent constants that are often unknown and difficult to estimate.
Geometry-Aware Universal Mirror-Prox
Mirror-prox (MP) is a well-known algorithm to solve variational inequality (VI) problems.
To Each Optimizer a Norm, To Each Norm its Generalization
For under-parameterized linear classification, we prove that for any linear classifier separating the data, there exists a family of quadratic norms ||.||_P such that the classifier's direction is the same as that of the maximum P-margin solution.
Semantics Preserving Adversarial Attacks
While progress has been made in crafting visually imperceptible adversarial examples, constructing semantically meaningful ones remains a challenge.
Reducing the variance in online optimization by transporting past gradients
While variance reduction methods have shown that reusing past gradients can be beneficial when there is a finite number of datapoints, they do not easily extend to the online setting.
Semantics Preserving Adversarial Learning
While progress has been made in crafting visually imperceptible adversarial examples, constructing semantically meaningful ones remains a challenge.
M-ADDA: Unsupervised Domain Adaptation with Deep Metric Learning
Unsupervised domain adaptation techniques have been successful for a wide range of problems where supervised labels are limited.
Stop Wasting My Gradients: Practical SVRG
We present and analyze several strategies for improving the performance of stochastic variance-reduced gradient (SVRG) methods.
Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions
We also give a convergence-rate analysis of our method and many other previous methods which exploit the geometry of the space.
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs).
