Search Results for author:
Found 82 papers, 4 papers with code
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
The activations of the RNN store the state of the "fast" RL algorithm on the current (previously unseen) MDP.
Sharpness-Aware Minimization and the Edge of Stability
Recent experiments have shown that, often, when training a neural network with gradient descent (GD) with a step size $\eta$, the operator norm of the Hessian of the loss grows until it approximately reaches $2/\eta$, after which it fluctuates around this value.
Greedy Convex Ensemble
Theoretically, we first consider whether we can use linear, instead of convex, combinations, and obtain generalization results similar to existing ones for learning from a convex hull.
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks
We provide polynomial bounds on the number of iterations for gradient descent to approximate the least squares matrix $\Phi$, in the case where the initial hypothesis $\Theta_1 = ... = \Theta_L = I$ has excess loss bounded by a small enough constant.
Best of many worlds: Robust model selection for online supervised learning
We introduce algorithms for online, full-information prediction that are competitive with contextual tree experts of unknown complexity, in both probabilistic and adversarial settings.
Sharp convergence rates for Langevin dynamics in the nonconvex setting
We study the problem of sampling from a distribution $p^*(x) \propto \exp\left(-U(x)\right)$, where the function $U$ is $L$-smooth everywhere and $m$-strongly convex outside a ball of radius $R$, but potentially nonconvex inside this ball.
Representing smooth functions as compositions of near-identity functions with implications for deep network optimization
This implies that $h$ can be represented to any accuracy by a deep residual network whose nonlinear layers compute functions with a small Lipschitz constant.
Online learning with kernel losses
We also study the full information setting when the underlying losses are kernel functions and present an adapted exponential weights algorithm and a conditional gradient descent algorithm.
On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
We provide convergence guarantees in Wasserstein distance for a variety of variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion and control-variate underdamped Langevin diffusion.
Alternating minimization for dictionary learning: Local Convergence Guarantees
We present theoretical guarantees for an alternating minimization algorithm for the dictionary learning/sparse coding problem.
Underdamped Langevin MCMC: A non-asymptotic analysis
We study the underdamped Langevin diffusion when the log of the target distribution is smooth and strongly concave.
FLAG n' FLARE: Fast Linearly-Coupled Adaptive Gradient Methods
We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions.
Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks
We prove new upper and lower bounds on the VC-dimension of deep neural networks with the ReLU activation function.
Acceleration and Averaging in Stochastic Mirror Descent Dynamics
We discuss the interaction between the parameters of the dynamics (learning rate and averaging weights) and the covariation of the noise process, and show, in particular, how the asymptotic rate of covariation affects the choice of parameters and, ultimately, the convergence rate.
Recovery Guarantees for One-hidden-layer Neural Networks
For activation functions that are also smooth, we show $\mathit{local~linear~convergence}$ guarantees of gradient descent under a resampling rule.
Hit-and-Run for Sampling and Planning in Non-Convex Spaces
We propose the Hit-and-Run algorithm for planning and sampling problems in non-convex spaces.
Linear Programming for Large-Scale Markov Decision Problems
We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost.
Bounding Embeddings of VC Classes into Maximum Classes
The most promising approach to positively resolving the conjecture is by embedding general VC classes into maximum classes without super-linear increase to their VC dimensions, as such embeddings would extend the known compression schemes to all VC classes.
A simple parameter-free and adaptive approach to optimization under a minimal local smoothness assumption
The difficulty of optimization is measured in terms of 1) the amount of \emph{noise} $b$ of the function evaluation and 2) the local smoothness, $d$, of the function.
Gen-Oja: A Two-time-scale approach for Streaming CCA
In this paper, we study the problems of principal Generalized Eigenvector computation and Canonical Correlation Analysis in the stochastic setting.
Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems
We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback.
Horizon-Independent Minimax Linear Regression
We consider online linear regression: at each round, an adversary reveals a covariate vector, the learner predicts a real value, the adversary reveals a label, and the learner suffers the squared prediction error.
Gen-Oja: Simple & Efficient Algorithm for Streaming Generalized Eigenvector Computation
In this paper, we study the problems of principle Generalized Eigenvector computation and Canonical Correlation Analysis in the stochastic setting.
Alternating minimization for dictionary learning with random initialization
However, in contrast to previous theoretical analyses for this problem, we replace a condition on the operator norm (that is, the largest magnitude singular value) of the true underlying dictionary $A^*$ with a condition on the matrix infinity norm (that is, the largest magnitude term).
Near Minimax Optimal Players for the Finite-Time 3-Expert Prediction Problem
We study minimax strategies for the online prediction problem with expert advice.
Acceleration and Averaging in Stochastic Descent Dynamics
We formulate and study a general family of (continuous-time) stochastic dynamics for accelerated first-order minimization of smooth convex functions.
Adaptive Averaging in Accelerated Descent Dynamics
This dynamics can be described naturally as a coupling of a dual variable accumulating gradients at a given rate $\eta(t)$, and a primal variable obtained as the weighted average of the mirrored dual trajectory, with weights $w(t)$.
Minimax Time Series Prediction
We consider an adversarial formulation of the problem ofpredicting a time series with square loss.
Accelerated Mirror Descent in Continuous and Discrete Time
We study accelerated mirror descent dynamics in continuous and discrete time.
Efficient Minimax Strategies for Square Loss Games
We consider online prediction problems where the loss between the prediction and the outcome is measured by the squared Euclidean distance and its generalization, the squared Mahalanobis distance.
Large-Margin Convex Polytope Machine
We present the Convex Polytope Machine (CPM), a novel non-linear learning algorithm for large-scale binary classification tasks.
How to Hedge an Option Against an Adversary: Black-Scholes Pricing is Minimax Optimal
We consider a popular problem in finance, option pricing, through the lens of an online learning game between Nature and an Investor.
Online Learning in Markov Decision Processes with Adversarially Chosen Transition Probability Distributions
The goal of the learning algorithm is to choose a path that minimizes the loss while traversing from the start to finish node.
Information-theoretic lower bounds on the oracle complexity of convex optimization
The extensive use of convex optimization in machine learning and statistics makes such an understanding critical to understand fundamental computational limits of learning and estimation.
Optimistic Linear Programming gives Logarithmic Regret for Irreducible MDPs
OLP is closely related to an algorithm proposed by Burnetas and Katehakis with four key differences: OLP is simpler, it does not require knowledge of the supports of transition probabilities and the proof of the regret bound is simpler, but our regret bound is a constant factor larger than the regret of their algorithm.
Large-Scale Markov Decision Problems via the Linear Programming Dual
Moreover, we propose an efficient algorithm, scaling with the size of the subspace but not the state space, that is able to find a policy with low excess loss relative to the best policy in this class.
Quantitative Weak Convergence for Discrete Stochastic Processes
In this paper, we quantitative convergence in $W_2$ for a family of Langevin-like stochastic processes that includes stochastic gradient descent and related gradient-based algorithms.
Fast Mean Estimation with Sub-Gaussian Rates
We propose an estimator for the mean of a random vector in $\mathbb{R}^d$ that can be computed in time $O(n^4+n^2d)$ for $n$ i. i. d.~samples and that has error bounds matching the sub-Gaussian case.
OSOM: A simultaneously optimal algorithm for multi-armed and linear contextual bandits
We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information.
Langevin Monte Carlo without smoothness
Langevin Monte Carlo (LMC) is an iterative algorithm used to generate samples from a distribution that is known only up to a normalizing constant.
Benign Overfitting in Linear Regression
Motivated by this phenomenon, we consider when a perfect fit to training data in linear regression is compatible with accurate prediction.
Stochastic Gradient and Langevin Processes
We prove quantitative convergence rates at which discrete Langevin-like processes converge to the invariant distribution of a related stochastic differential equation.
Improved Bounds for Discretization of Langevin Diffusions: Near-Optimal Rates without Convexity
We present an improved analysis of the Euler-Maruyama discretization of the Langevin diffusion.
Bayesian Robustness: A Nonasymptotic Viewpoint
We study the problem of robustly estimating the posterior distribution for the setting where observed data can be contaminated with potentially adversarial outliers.
High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm
We propose a Markov chain Monte Carlo (MCMC) algorithm based on third-order Langevin dynamics for sampling from distributions with log-concave and smooth densities.
An Efficient Sampling Algorithm for Non-smooth Composite Potentials
We consider the problem of sampling from a density of the form $p(x) \propto \exp(-f(x)- g(x))$, where $f: \mathbb{R}^d \rightarrow \mathbb{R}$ is a smooth and strongly convex function and $g: \mathbb{R}^d \rightarrow \mathbb{R}$ is a convex and Lipschitz function.
Infinite-Horizon Policy-Gradient Estimation
In this paper we introduce GPOMDP, a simulation-based algorithm for generating a {\em biased} estimate of the gradient of the {\em average reward} in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies.
Hebbian Synaptic Modifications in Spiking Neurons that Learn
In this paper, we derive a new model of synaptic plasticity, based on recent algorithms for reinforcement learning (in which an agent attempts to learn appropriate actions to maximize its long-term average reward).
Sampling for Bayesian Mixture Models: MCMC with Polynomial-Time Mixing
We study the problem of sampling from the power posterior distribution in Bayesian Gaussian mixture models, a robust version of the classical posterior.
Oracle Lower Bounds for Stochastic Gradient Sampling Algorithms
We consider the problem of sampling from a strongly log-concave density in $\mathbb{R}^d$, and prove an information theoretic lower bound on the number of stochastic gradient queries of the log density needed.
Self-Distillation Amplifies Regularization in Hilbert Space
Knowledge distillation introduced in the deep learning context is a method to transfer knowledge from one architecture to another.
On Thompson Sampling with Langevin Algorithms
The resulting approximate Thompson sampling algorithm has logarithmic regret and its computational complexity does not scale with the time horizon of the algorithm.
On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentration
When the matrix $\bar{A}$ is Hurwitz, we prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity.
Optimal Robust Linear Regression in Nearly Linear Time
We study the problem of high-dimensional robust linear regression where a learner is given access to $n$ samples from the generative model $Y = \langle X, w^* \rangle + \epsilon$ (with $X \in \mathbb{R}^d$ and $\epsilon$ independent), in which an $\eta$ fraction of the samples have been adversarially corrupted.
Failures of model-dependent generalization bounds for least-norm interpolation
We consider bounds on the generalization performance of the least-norm linear regressor, in the over-parameterized regime where it can interpolate the data.
Optimal Mean Estimation without a Variance
Concretely, given a sample $\mathbf{X} = \{X_i\}_{i = 1}^n$ from a distribution $\mathcal{D}$ over $\mathbb{R}^d$ with mean $\mu$ which satisfies the following \emph{weak-moment} assumption for some ${\alpha \in [0, 1]}$: \begin{equation*} \forall \|v\| = 1: \mathbb{E}_{X \thicksim \mathcal{D}}[\lvert \langle X - \mu, v\rangle \rvert^{1 + \alpha}] \leq 1, \end{equation*} and given a target failure probability, $\delta$, our goal is to design an estimator which attains the smallest possible confidence interval as a function of $n, d,\delta$.
When does gradient descent with logistic loss find interpolating two-layer networks?
We study the training of finite-width two-layer smoothed ReLU networks for binary classification using the logistic loss.
When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations?
We establish conditions under which gradient descent applied to fixed-width deep networks drives the logistic loss to zero, and prove bounds on the rate of convergence.
Deep learning: a statistical viewpoint
We conjecture that specific principles underlie these phenomena: that overparametrization allows gradient methods to find interpolating solutions, that these methods implicitly impose regularization, and that overparametrization leads to benign overfitting.
Infinite-Horizon Offline Reinforcement Learning with Linear Function Approximation: Curse of Dimensionality and Algorithm
In this regime, for any $q\in[\gamma^{2}, 1]$, we can construct a hard instance such that the smallest eigenvalue of its feature covariance matrix is $q/d$ and it requires $\Omega\left(\frac{d}{\gamma^{2}\left(q-\gamma^{2}\right)\varepsilon^{2}}\exp\left(\Theta\left(d\gamma^{2}\right)\right)\right)$ samples to approximate the value function up to an additive error $\varepsilon$.
Agnostic learning with unknown utilities
This raises an interesting question whether learning is even possible in our setup, given that obtaining a generalizable estimate of utility $u^*$ might not be possible from finitely many samples.
Preference learning along multiple criteria: A game-theoretic perspective
Finally, we showcase the practical utility of our framework in a user study on autonomous driving, where we find that the Blackwell winner outperforms the von Neumann winner for the overall preferences.
On the Theory of Reinforcement Learning with Once-per-Episode Feedback
We study a theory of reinforcement learning (RL) in which the learner receives binary feedback only once at the end of an episode.
Adversarial Examples in Multi-Layer Random ReLU Networks
We consider the phenomenon of adversarial examples in ReLU networks with independent gaussian parameters.
The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks
The recent success of neural network models has shone light on a rather surprising statistical phenomenon: statistical models that perfectly fit noisy data can generalize well to unseen test data.
Optimal and instance-dependent guarantees for Markovian linear stochastic approximation
We then prove a non-asymptotic instance-dependent bound on a suitably averaged sequence of iterates, with a leading term that matches the local asymptotic minimax limit, including sharp dependence on the parameters $(d, t_{\mathrm{mix}})$ in the higher order terms.
Optimal variance-reduced stochastic approximation in Banach spaces
We study the problem of estimating the fixed point of a contractive operator defined on a separable Banach space.
Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data
Benign overfitting, the phenomenon where interpolating models generalize well in the presence of noisy data, was first observed in neural network models trained with gradient descent.
Random Feature Amplification: Feature Learning and Generalization in Neural Networks
We consider data with binary labels that are generated by an XOR-like function of the input features.
Off-policy estimation of linear functionals: Non-asymptotic theory for semi-parametric efficiency
The problem of estimating a linear functional based on observational data is canonical in both the causal inference and bandit literatures.
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima
We consider Sharpness-Aware Minimization (SAM), a gradient-based optimization method for deep networks that has exhibited performance improvements on image and language prediction problems.
Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
In this work, we investigate the implicit bias of gradient flow and gradient descent in two-layer fully-connected neural networks with leaky ReLU activations when the training data are nearly-orthogonal, a common property of high-dimensional data.
Kernel-based off-policy estimation without overlap: Instance optimality beyond semiparametric efficiency
When it is violated, the classical semi-parametric efficiency bound can easily become infinite, so that the instance-optimal risk depends on the function class used to model the regression function.
Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization
Linear classifiers and leaky ReLU networks trained by gradient flow on the logistic loss have an implicit bias towards solutions which satisfy the Karush--Kuhn--Tucker (KKT) conditions for margin maximization.
Prediction, Learning, Uniform Convergence, and Scale-sensitive Dimensions
We apply this result, together with techniques due to Haussler and to Benedek and Itai, to obtain new upper bounds on packing numbers in terms of this scale-sensitive notion of dimension.
Trained Transformers Learn Linear Models In-Context
We show that although gradient flow succeeds at finding a global minimum in this setting, the trained transformer is still brittle under mild covariate shifts.
How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
Transformers pretrained on diverse tasks exhibit remarkable in-context learning (ICL) capabilities, enabling them to solve unseen tasks solely based on input contexts without adjusting model parameters.
On the Statistical Properties of Generative Adversarial Models for Low Intrinsic Data Dimension
In this paper, we attempt to bridge the gap between the theory and practice of GANs and their bidirectional variant, Bi-directional GANs (BiGANs), by deriving statistical guarantees on the estimated densities in terms of the intrinsic dimension of the data and the latent space.
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization
We study the \emph{in-context learning} (ICL) ability of a \emph{Linear Transformer Block} (LTB) that combines a linear attention component and a linear multi-layer perceptron (MLP) component.
Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiency
We consider gradient descent (GD) with a constant stepsize applied to logistic regression with linearly separable data, where the constant stepsize $\eta$ is so large that the loss initially oscillates.
A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data
To bridge the gap between the theory and practice of WAEs, in this paper, we show that WAEs can learn the data distributions when the network architectures are properly chosen.
