Search Results for author:
Found 61 papers, 12 papers with code
On Modulating the Gradient for Meta-Learning
Inspired by optimization techniques, we propose a novel meta-learning algorithm with gradient modulation to encourage fast-adaptation of neural networks in the absence of abundant data.
Boosting gets full Attention for Relational Learning
Second, what has been learned progresses back bottom-up via attention and aggregation mechanisms, progressively crafting new features that complete at the end the set of observation features over which a single tree is learned, boosting's iteration clock is incremented and new class residuals are computed.
Tempered Calculus for ML: Application to Hyperbolic Model Embedding
Most mathematical distortions used in ML are fundamentally integral in nature: $f$-divergences, Bregman divergences, (regularized) optimal transport distances, integral probability metrics, geodesic distances, etc.
The Tempered Hilbert Simplex Distance and Its Application To Non-linear Embeddings of TEMs
Tempered Exponential Measures (TEMs) are a parametric generalization of the exponential family of distributions maximizing the tempered entropy function among positive measures subject to a probability normalization of their power densities.
Optimal Transport with Tempered Exponential Measures
In the field of optimal transport, two prominent subfields face each other: (i) unregularized optimal transport, "\`a-la-Kantorovich", which leads to extremely sparse plans but with algorithms that scale poorly, and (ii) entropic-regularized optimal transport, "\`a-la-Sinkhorn-Cuturi", which gets near-linear approximation algorithms but leads to maximally un-sparse plans.
Smoothly Giving up: Robustness for Simple Models
There is a growing need for models that are interpretable and have reduced energy and computational cost (e. g., in health care analytics and federated learning).
LegendreTron: Uprising Proper Multiclass Loss Learning
Existing methods do this by fitting an inverse canonical link function which monotonically maps $\mathbb{R}$ to $[0, 1]$ to estimate probabilities for binary problems.
Clustering above Exponential Families with Tempered Exponential Measures
The link with exponential families has allowed $k$-means clustering to be generalized to a wide variety of data generating distributions in exponential families and clustering distortions among Bregman divergences.
What killed the Convex Booster ?
A landmark negative result of Long and Servedio established a worst-case spectacular failure of a supervised learning trio (loss, algorithm, model) otherwise praised for its high precision machinery.
Fair Wrapping for Black-box Predictions
We introduce a new family of techniques to post-process ("wrap") a black-box classifier in order to reduce its bias.
Generative Trees: Adversarial and Copycat
While Generative Adversarial Networks (GANs) achieve spectacular results on unstructured data like images, there is still a gap on tabular data, data for which state of the art supervised learning still favours to a large extent decision tree (DT)-based models.
Manifold Learning Benefits GANs
In our design, the manifold learning and coding steps are intertwined with layers of the discriminator, with the goal of attracting intermediate feature representations onto manifolds.
Being Properly Improper
Hence, optimizing a proper loss function on twisted data could perilously lead the learning algorithm towards the twisted posterior, rather than to the desired clean posterior.
Fair Densities via Boosting the Sufficient Statistics of Exponential Families
Furthermore, when the weak learners are specified to be decision trees, the sufficient statistics of the learned distribution can be examined to provide clues on sources of (un)fairness.
All your loss are belong to Bayes
Loss functions are a cornerstone of machine learning and the starting point of most algorithms.
Cumulant-free closed-form formulas for some common (dis)similarities between densities of an exponential family
It is well-known that the Bhattacharyya, Hellinger, Kullback-Leibler, $\alpha$-divergences, and Jeffreys' divergences between densities belonging to a same exponential family have generic closed-form formulas relying on the strictly convex and real-analytic cumulant function characterizing the exponential family.
Generalised Lipschitz Regularisation Equals Distributional Robustness
The problem of adversarial examples has highlighted the need for a theory of regularisation that is general enough to apply to exotic function classes, such as universal approximators.
Supervised Learning: No Loss No Cry
In detail, we cast {\sc SLIsotron} as learning a loss from a family of composite square losses.
Boosted and Differentially Private Ensembles of Decision Trees
To address this, we craft a new parametererized proper loss, called the M$\alpha$-loss, which, as we show, allows to finely tune the tradeoff in the complete spectrum of sensitivity vs boosting guarantees.
Advances and Open Problems in Federated Learning
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
Disentangled behavioural representations
Individual characteristics in human decision-making are often quantified by fitting a parametric cognitive model to subjects' behavior and then studying differences between them in the associated parameter space.
A Primal-Dual link between GANs and Autoencoders
First, we find that the $f$-GAN and WAE objectives partake in a primal-dual relationship and are equivalent under some assumptions, which then allows us to explicate the success of WAE.
Certifying Distributional Robustness using Lipschitz Regularisation
Distributional robust risk (DRR) minimisation has arisen as a flexible and effective framework for machine learning.
Proper-Composite Loss Functions in Arbitrary Dimensions
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used.
Adversarial Networks and Autoencoders: The Primal-Dual Relationship and Generalization Bounds
First, we find that the $f$-GAN and WAE objectives partake in a primal-dual relationship and are equivalent under some assumptions, which then allows us to explicate the success of WAE.
New Tricks for Estimating Gradients of Expectations
We introduce a family of pairwise stochastic gradient estimators for gradients of expectations, which are related to the log-derivative trick, but involve pairwise interactions between samples.
Representation Learning of Compositional Data
Our approach combines the benefits of the log-ratio transformation from compositional data analysis and exponential family PCA.
The Bregman chord divergence
Distances are fundamental primitives whose choice significantly impacts the performances of algorithms in machine learning and signal processing.
Lipschitz Networks and Distributional Robustness
Robust risk minimisation has several advantages: it has been studied with regards to improving the generalisation properties of models and robustness to adversarial perturbation.
Hyperparameter Learning for Conditional Kernel Mean Embeddings with Rademacher Complexity Bounds
Conditional kernel mean embeddings are nonparametric models that encode conditional expectations in a reproducing kernel Hilbert space.
D-PAGE: Diverse Paraphrase Generation
In this paper, we investigate the diversity aspect of paraphrase generation.
Variational Network Inference: Strong and Stable with Concrete Support
Traditional methods for the discovery of latent network structures are limited in two ways: they either assume that all the signal comes from the network (i. e. there is no source of signal outside the network) or they place constraints on the network parameters to ensure model or algorithmic stability.
Private Text Classification
Confidential text corpora exist in many forms, but do not allow arbitrary sharing.
Integral Privacy for Sampling
Privacy enforces an information theoretic barrier on approximation, and we show how to reach this barrier with guarantees on the approximation of the target non private density.
Monge blunts Bayes: Hardness Results for Adversarial Training
A key feature of our result is that it holds for all proper losses, and for a popular subset of these, the optimisation of this central measure appears to be independent of the loss.
Boosted Density Estimation Remastered
There has recently been a steady increase in the number iterative approaches to density estimation.
Entity Resolution and Federated Learning get a Federated Resolution
In our experiments, we modify a simple token-based entity resolution algorithm so that it indeed aims at avoiding matching rows belonging to different classes, and perform experiments in the setting where entity resolution relies on noisy data, which is very relevant to real world domains.
Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
Our results bring a clear and strong support for federated learning: under reasonable assumptions on the number and magnitude of entity resolution's mistakes, it can be extremely beneficial to carry out federated learning in the setting where each peer's data provides a significant uplift to the other.
On $w$-mixtures: Finite convex combinations of prescribed component distributions
The information geometry induced by the Bregman generator set to the Shannon negentropy on this space yields a dually flat space called the mixture family manifold.
f-GANs in an Information Geometric Nutshell
In this paper, we unveil a broad class of distributions for which such convergence happens --- namely, deformed exponential families, a wide superset of exponential families --- and show tight connections with the three other key GAN parameters: loss, game and architecture.
Evolving a Vector Space with any Generating Set
In Valiant's model of evolution, a class of representations is evolvable iff a polynomial-time process of random mutations guided by selection converges with high probability to a representation as $\epsilon$-close as desired from the optimal one, for any required $\epsilon>0$.
Semi-parametric Network Structure Discovery Models
We propose a network structure discovery model for continuous observations that generalizes linear causal models by incorporating a Gaussian process (GP) prior on a network-independent component, and random sparsity and weight matrices as the network-dependent parameters.
Generalizing Jensen and Bregman divergences with comparative convexity and the statistical Bhattacharyya distances with comparable means
Comparative convexity is a generalization of convexity relying on abstract notions of means.
A series of maximum entropy upper bounds of the differential entropy
We present a series of closed-form maximum entropy upper bounds for the differential entropy of a continuous univariate random variable and study the properties of that series.
On Regularizing Rademacher Observation Losses
It has recently been shown that supervised learning linear classifiers with two of the most popular losses, the logistic and square loss, is equivalent to optimizing an equivalent loss over sufficient statistics about the class: Rademacher observations (rados).
Large Margin Nearest Neighbor Classification using Curved Mahalanobis Distances
We consider the supervised classification problem of machine learning in Cayley-Klein projective geometries: We show how to learn a curved Mahalanobis metric distance corresponding to either the hyperbolic geometry or the elliptic geometry using the Large Margin Nearest Neighbor (LMNN) framework.
Tsallis Regularized Optimal Transport and Ecological Inference
We also present the first application of optimal transport to the problem of ecological inference, that is, the reconstruction of joint distributions from their marginals, a problem of large interest in the social sciences.
Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise.
Ranked #2 on
Image Classification
on Clothing1M (using clean data)
(using extra training data)
A scaled Bregman theorem with applications
Experiments on each of these domains validate the analyses and suggest that the scaled Bregman theorem might be a worthy addition to the popular handful of Bregman divergence properties that have been pervasive in machine learning.
The Crossover Process: Learnability and Data Protection from Inference Attacks
It is usual to consider data protection and learnability as conflicting objectives.
Fast $(1+ε)$-approximation of the Löwner extremal matrices of high-dimensional symmetric matrices
Matrix data sets are common nowadays like in biomedical imaging where the Diffusion Tensor Magnetic Resonance Imaging (DT-MRI) modality produces data sets of 3D symmetric positive definite matrices anchored at voxel positions capturing the anisotropic diffusion properties of water molecules in biological tissues.
Fast Learning from Distributed Datasets without Entity Matching
Our goal is to learn a classifier in the cross product space of the two domains, in the hard case in which no shared ID is available -- e. g. due to anonymization.
Loss factorization, weakly supervised learning and label noise robustness
We prove that the empirical risk of most well-known loss functions factors into a linear term aggregating all labels with a term that is label free, and can further be expressed by sums of the loss.
k-variates++: more pluses in the k-means++
For either the specific frameworks considered here, or for the differential privacy setting, there is little to no prior results on the direct application of k-means++ and its approximation bounds --- state of the art contenders appear to be significantly more complex and / or display less favorable (approximation) properties.
Learning Games and Rademacher Observations Losses
We first show that this unexpected equivalence can actually be generalized to other example / rado losses, with necessary and sufficient conditions for the equivalence, exemplified on four losses that bear popular names in various fields: exponential (boosting), mean-variance (finance), Linear Hinge (on-line learning), ReLU (deep learning), and unhinged (statistics).
Rademacher Observations, Private Data, and Boosting
We show that rados comply with various privacy requirements that make them good candidates for machine learning in a privacy framework.
(Almost) No Label No Cry
In Learning with Label Proportions (LLP), the objective is to learn a supervised classifier when, instead of labels, only label proportions for bags of observations are known.
Further heuristics for $k$-means: The merge-and-split heuristic and the $(k,l)$-means
This novel heuristic can improve Hartigan's $k$-means when it has converged to a local minimum.
Optimal interval clustering: Application to Bregman clustering and statistical mixture learning
We present a generic dynamic programming method to compute the optimal clustering of $n$ scalar elements into $k$ pairwise disjoint intervals.
On the Efficient Minimization of Classification Calibrated Surrogates
Bartlett et al (2006) recently proved that a ground condition for convex surrogates, classification calibration, ties up the minimization of the surrogates and classification risks, and left as an important problem the algorithmic questions about the minimization of these surrogates.
