Search Results for author:
Found 20 papers, 3 papers with code
Deep Linear Networks can Benignly Overfit when Shallow Ones Do
We bound the excess risk of interpolating deep linear networks trained using gradient flow.
Undersampling is a Minimax Optimal Robustness Intervention in Nonparametric Classification
While a broad range of techniques have been proposed to tackle distribution shift, the simple baseline of training on an $\textit{undersampled}$ balanced dataset often achieves close to state-of-the-art-accuracy across several popular benchmarks.
Random Feature Amplification: Feature Learning and Generalization in Neural Networks
We consider data with binary labels that are generated by an XOR-like function of the input features.
Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data
Benign overfitting, the phenomenon where interpolating models generalize well in the presence of noisy data, was first observed in neural network models trained with gradient descent.
Is Importance Weighting Incompatible with Interpolating Classifiers?
As a remedy, we show that polynomially-tailed losses restore the effects of importance reweighting in correcting distribution shift in overparameterized models.
Foolish Crowds Support Benign Overfitting
We prove a lower bound on the excess risk of sparse interpolating procedures for linear regression with Gaussian data in the overparameterized regime.
The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks
The recent success of neural network models has shone light on a rather surprising statistical phenomenon: statistical models that perfectly fit noisy data can generalize well to unseen test data.
On the Theory of Reinforcement Learning with Once-per-Episode Feedback
We study a theory of reinforcement learning (RL) in which the learner receives binary feedback only once at the end of an episode.
When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations?
We establish conditions under which gradient descent applied to fixed-width deep networks drives the logistic loss to zero, and prove bounds on the rate of convergence.
When does gradient descent with logistic loss find interpolating two-layer networks?
We study the training of finite-width two-layer smoothed ReLU networks for binary classification using the logistic loss.
Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime
We prove bounds on the population risk of the maximum margin algorithm for two-class linear classification.
Oracle Lower Bounds for Stochastic Gradient Sampling Algorithms
We consider the problem of sampling from a strongly log-concave density in $\mathbb{R}^d$, and prove an information theoretic lower bound on the number of stochastic gradient queries of the log density needed.
The intriguing role of module criticality in the generalization of deep networks
We study the phenomenon that some modules of deep neural networks (DNNs) are more critical than others.
Langevin Monte Carlo without smoothness
Langevin Monte Carlo (LMC) is an iterative algorithm used to generate samples from a distribution that is known only up to a normalizing constant.
OSOM: A simultaneously optimal algorithm for multi-armed and linear contextual bandits
We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information.
Sharp convergence rates for Langevin dynamics in the nonconvex setting
We study the problem of sampling from a distribution $p^*(x) \propto \exp\left(-U(x)\right)$, where the function $U$ is $L$-smooth everywhere and $m$-strongly convex outside a ball of radius $R$, but potentially nonconvex inside this ball.
Online learning with kernel losses
We also study the full information setting when the underlying losses are kernel functions and present an adapted exponential weights algorithm and a conditional gradient descent algorithm.
On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
We provide convergence guarantees in Wasserstein distance for a variety of variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion and control-variate underdamped Langevin diffusion.
Alternating minimization for dictionary learning: Local Convergence Guarantees
We present theoretical guarantees for an alternating minimization algorithm for the dictionary learning/sparse coding problem.
Underdamped Langevin MCMC: A non-asymptotic analysis
We study the underdamped Langevin diffusion when the log of the target distribution is smooth and strongly concave.
