Search Results for author:
Found 34 papers, 4 papers with code
Geometry and Stability of Supervised Learning Problems
We introduce a notion of distance between supervised learning problems, which we call the Risk distance.
Four Facets of Forecast Felicity: Calibration, Predictiveness, Randomness and Regret
Machine learning has traditionally focused on types of losses and their corresponding regret.
A General Framework for Learning under Corruption: Label Noise, Attribute Noise, and Beyond
Corruption is frequently observed in collected data and has been extensively studied in machine learning under different corruption models.
Insights From Insurance for Fair Machine Learning
Tracing the interaction of uncertainty, fairness and responsibility in insurance provides a fresh perspective on fairness in machine learning.
The Geometry of Mixability
Mixable loss functions are of fundamental importance in the context of prediction with expert advice in the online setting since they characterize fast learning rates.
The Geometry and Calculus of Losses
In this paper we systematically develop the theory of loss functions for such problems from a novel perspective whose basic ingredients are convex sets with a particular structure.
Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity
As a concrete example, we focus on divergence risk measures based on f-divergence ambiguity sets, which are a widespread tool used to foster distributional robustness of machine learning systems.
Fairness and Randomness in Machine Learning: Statistical Independence and Relativization
In this paper, we dissect the role of statistical independence in fairness and randomness notions regularly used in machine learning.
Information Processing Equalities and the Information-Risk Bridge
We introduce two new classes of measures of information for statistical experiments which generalise and subsume $\phi$-divergences, integral probability metrics, $\mathfrak{N}$-distances (MMD), and $(f,\Gamma)$ divergences between two or more distributions.
Risk Measures and Upper Probabilities: Coherence and Stratification
Machine learning typically presupposes classical probability theory which implies that aggregation is built upon expectation.
What killed the Convex Booster ?
A landmark negative result of Long and Servedio established a worst-case spectacular failure of a supervised learning trio (loss, algorithm, model) otherwise praised for its high precision machinery.
PAC-Bayesian Bound for the Conditional Value at Risk
Conditional Value at Risk (CVaR) is a family of "coherent risk measures" which generalize the traditional mathematical expectation.
A Primal-Dual link between GANs and Autoencoders
First, we find that the $f$-GAN and WAE objectives partake in a primal-dual relationship and are equivalent under some assumptions, which then allows us to explicate the success of WAE.
Proper-Composite Loss Functions in Arbitrary Dimensions
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used.
Adversarial Networks and Autoencoders: The Primal-Dual Relationship and Generalization Bounds
First, we find that the $f$-GAN and WAE objectives partake in a primal-dual relationship and are equivalent under some assumptions, which then allows us to explicate the success of WAE.
Fairness risk measures
In this paper, we propose a new definition of fairness that generalises some existing proposals, while allowing for generic sensitive features and resulting in a convex objective.
Minimax Lower Bounds for Cost Sensitive Classification
The cost-sensitive classification problem plays a crucial role in mission-critical machine learning applications, and differs with traditional classification by taking the misclassification costs into consideration.
Exp-Concavity of Proper Composite Losses
In special cases like the Aggregating Algorithm (\cite{vovk1995game}) with mixable losses and the Weighted Average Algorithm (\cite{kivinen1999averaging}) with exp-concave losses, it is possible to achieve $O(1)$ regret bounds.
Constant Regret, Generalized Mixability, and Mirror Descent
For a given entropy $\Phi$, losses for which a constant regret is possible using the \textsc{GAA} are called $\Phi$-mixable.
Provably Fair Representations
These provable properties can be used in a governance model involving a data producer, a data user and a data regulator, where there is a separation of concerns between fairness and target task utility to ensure transparency and prevent perverse incentives.
f-GANs in an Information Geometric Nutshell
In this paper, we unveil a broad class of distributions for which such convergence happens --- namely, deformed exponential families, a wide superset of exponential families --- and show tight connections with the three other key GAN parameters: loss, game and architecture.
The cost of fairness in classification
We study the problem of learning classifiers with a fairness constraint, with three main contributions towards the goal of quantifying the problem's inherent tradeoffs.
A Modular Theory of Feature Learning
We propose the idea of a risk gap induced by representation learning for a given prediction context, which measures the difference in the risk of some learner using the learned features as compared to the original inputs.
Fast rates in statistical and online learning
For bounded losses, we show how the central condition enables a direct proof of fast rates and we prove its equivalence to the Bernstein condition, itself a generalization of the Tsybakov margin condition, both of which have played a central role in obtaining fast rates in statistical learning.
An Average Classification Algorithm
When working with a high or infinite dimensional kernel, it is imperative for speed of evaluation and storage issues that as few training samples as possible are used in the kernel expansion.
Learning with Symmetric Label Noise: The Importance of Being Unhinged
However, Long and Servedio [2010] proved that under symmetric label noise (SLN), minimisation of any convex potential over a linear function class can result in classification performance equivalent to random guessing.
A Theory of Feature Learning
Feature Learning aims to extract relevant information contained in data sets in an automated fashion.
Learning in the Presence of Corruption
In this paper we develop a general framework for tackling such problems as well as introducing upper and lower bounds on the risk for learning in the presence of corruption.
Generalized Mixability via Entropic Duality
Mixability is a property of a loss which characterizes when fast convergence is possible in the game of prediction with expert advice.
From Stochastic Mixability to Fast Rates
In the non-statistical prediction with expert advice setting, there is an analogous slow and fast rate phenomenon, and it is entirely characterized in terms of the mixability of the loss $\ell$ (there being no role there for $\mathcal{F}$ or $\mathsf{P}$).
Generalised Mixability, Constant Regret, and Bayesian Updating
Mixability of a loss is known to characterise when constant regret bounds are achievable in games of prediction with expert advice through the use of Vovk's aggregating algorithm.
Le Cam meets LeCun: Deficiency and Generic Feature Learning
"Deep Learning" methods attempt to learn generic features in an unsupervised fashion from a large unlabelled data set.
Mixability in Statistical Learning
We show that, in the special case of log-loss, stochastic mixability reduces to a well-known (but usually unnamed) martingale condition, which is used in existing convergence theorems for minimum description length and Bayesian inference.
Composite Multiclass Losses
We also show that the integral representation for binary proper losses can not be extended to multiclass losses.
