Search Results for author:
Found 18 papers, 10 papers with code
Criterion collapse and loss distribution control
In this work, we consider the notion of "criterion collapse," in which optimization of one metric implies optimality in another, with a particular focus on conditions for collapse into error probability minimizers under a wide variety of learning criteria, ranging from DRO and OCE risks (CVaR, tilted ERM) to non-monotonic criteria underlying recent ascent-descent algorithms explored in the literature (Flooding, SoftAD).
Implicit regularization via soft ascent-descent
As models grow larger and more complex, achieving better off-sample generalization with minimal trial-and-error is critical to the reliability and economy of machine learning workflows.
Robust variance-regularized risk minimization with concomitant scaling
Under losses which are potentially heavy-tailed, we consider the task of minimizing sums of the loss mean and standard deviation, without trying to accurately estimate the variance.
Flexible risk design using bi-directional dispersion
Many novel notions of "risk" (e. g., CVaR, tilted risk, DRO risk) have been proposed and studied, but these risks are all at least as sensitive as the mean to loss tails on the upside, and tend to ignore deviations on the downside.
A Survey of Learning Criteria Going Beyond the Usual Risk
Virtually all machine learning tasks are characterized using some form of loss function, and "good performance" is typically stated in terms of a sufficiently small average loss, taken over the random draw of test data.
Robust learning with anytime-guaranteed feedback
Under data distributions which may be heavy-tailed, many stochastic gradient-based learning algorithms are driven by feedback queried at points with almost no performance guarantees on their own.
Spectral risk-based learning using unbounded losses
In this work, we consider the setting of learning problems under a wide class of spectral risk (or "L-risk") functions, where a Lipschitz-continuous spectral density is used to flexibly assign weight to extreme loss values.
Better scalability under potentially heavy-tailed feedback
We study scalable alternatives to robust gradient descent (RGD) techniques that can be used when the losses and/or gradients can be heavy-tailed, though this will be unknown to the learner.
Learning with risks based on M-location
In this work, we study a new class of risks defined in terms of the location and deviation of the loss distribution, generalizing far beyond classical mean-variance risk functions.
Making learning more transparent using conformalized performance prediction
In this work, we study some novel applications of conformal inference techniques to the problem of providing machine learning procedures with more transparent, accurate, and practical performance guarantees.
Learning with CVaR-based feedback under potentially heavy tails
We study learning algorithms that seek to minimize the conditional value-at-risk (CVaR), when all the learner knows is that the losses incurred may be heavy-tailed.
Improved scalability under heavy tails, without strong convexity
Empirically, we also show that under heavy-tailed losses, the proposed procedure cannot simply be replaced with naive cross-validation.
Better scalability under potentially heavy-tailed gradients
We study a scalable alternative to robust gradient descent (RGD) techniques that can be used when the gradients can be heavy-tailed, though this will be unknown to the learner.
Distribution-robust mean estimation via smoothed random perturbations
We consider the problem of mean estimation assuming only finite variance.
Statistics Theory Statistics Theory
PAC-Bayes under potentially heavy tails
We derive PAC-Bayesian learning guarantees for heavy-tailed losses, and obtain a novel optimal Gibbs posterior which enjoys finite-sample excess risk bounds at logarithmic confidence.
Robust descent using smoothed multiplicative noise
To improve the off-sample generalization of classical procedures minimizing the empirical risk under potentially heavy-tailed data, new robust learning algorithms have been proposed in recent years, with generalized median-of-means strategies being particularly salient.
Classification using margin pursuit
In this work, we study a new approach to optimizing the margin distribution realized by binary classifiers.
Efficient learning with robust gradient descent
Minimizing the empirical risk is a popular training strategy, but for learning tasks where the data may be noisy or heavy-tailed, one may require many observations in order to generalize well.
