Search Results for author: Rodolphe Jenatton

Found 40 papers, 12 papers with code

Pi-DUAL: Using Privileged Information to Distinguish Clean from Noisy Labels

no code implementations10 Oct 2023 Ke Wang, Guillermo Ortiz-Jimenez, Rodolphe Jenatton, Mark Collier, Efi Kokiopoulou, Pascal Frossard

Label noise is a pervasive problem in deep learning that often compromises the generalization performance of trained models.

When does Privileged Information Explain Away Label Noise?

1 code implementation3 Mar 2023 Guillermo Ortiz-Jimenez, Mark Collier, Anant Nawalgaria, Alexander D'Amour, Jesse Berent, Rodolphe Jenatton, Effrosyni Kokiopoulou

Leveraging privileged information (PI), or features available during training but not at test time, has recently been shown to be an effective method for addressing label noise.

Massively Scaling Heteroscedastic Classifiers

no code implementations30 Jan 2023 Mark Collier, Rodolphe Jenatton, Basil Mustafa, Neil Houlsby, Jesse Berent, Effrosyni Kokiopoulou

Heteroscedastic classifiers, which learn a multivariate Gaussian distribution over prediction logits, have been shown to perform well on image classification problems with hundreds to thousands of classes.

Classification Contrastive Learning +1

On the Adversarial Robustness of Mixture of Experts

no code implementations19 Oct 2022 Joan Puigcerver, Rodolphe Jenatton, Carlos Riquelme, Pranjal Awasthi, Srinadh Bhojanapalli

We next empirically evaluate the robustness of MoEs on ImageNet using adversarial attacks and show they are indeed more robust than dense models with the same computational cost.

Adversarial Robustness Open-Ended Question Answering

Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts

no code implementations6 Jun 2022 Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, Neil Houlsby

MoEs are a natural fit for a multimodal backbone, since expert layers can learn an appropriate partitioning of modalities.

Contrastive Learning

Transfer and Marginalize: Explaining Away Label Noise with Privileged Information

no code implementations18 Feb 2022 Mark Collier, Rodolphe Jenatton, Efi Kokiopoulou, Jesse Berent

Supervised learning datasets often have privileged information, in the form of features which are available at training time but are not available at test time e. g. the ID of the annotator that provided the label.

Predicting the utility of search spaces for black-box optimization:a simple, budget-aware approach

no code implementations15 Dec 2021 Setareh Ariafar, Justin Gilmer, Zack Nado, Jasper Snoek, Rodolphe Jenatton, George E. Dahl

For example, when tuning hyperparameters for machine learning pipelines on a new problem given a limited budget, one must strike a balance between excluding potentially promising regions and keeping the search space small enough to be tractable.

Bayesian Optimization

Sparse MoEs meet Efficient Ensembles

1 code implementation7 Oct 2021 James Urquhart Allingham, Florian Wenzel, Zelda E Mariet, Basil Mustafa, Joan Puigcerver, Neil Houlsby, Ghassen Jerfel, Vincent Fortuin, Balaji Lakshminarayanan, Jasper Snoek, Dustin Tran, Carlos Riquelme Ruiz, Rodolphe Jenatton

Machine learning models based on the aggregated outputs of submodels, either at the activation or prediction levels, often exhibit strong performance compared to individual models.

Few-Shot Learning

Deep Classifiers with Label Noise Modeling and Distance Awareness

no code implementations6 Oct 2021 Vincent Fortuin, Mark Collier, Florian Wenzel, James Allingham, Jeremiah Liu, Dustin Tran, Balaji Lakshminarayanan, Jesse Berent, Rodolphe Jenatton, Effrosyni Kokiopoulou

Uncertainty estimation in deep learning has recently emerged as a crucial area of interest to advance reliability and robustness in safety-critical applications.

Out-of-Distribution Detection

Distilling Ensembles Improves Uncertainty Estimates

no code implementations pproximateinference AABI Symposium 2021 Zelda E Mariet, Rodolphe Jenatton, Florian Wenzel, Dustin Tran

We seek to bridge the performance gap between batch ensembles (ensembles of deep networks with shared parameters) and deep ensembles on tasks which require not only predictions, but also uncertainty estimates for these predictions.

Training independent subnetworks for robust prediction

2 code implementations ICLR 2021 Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew M. Dai, Dustin Tran

Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network.

Hyperparameter Ensembles for Robustness and Uncertainty Quantification

3 code implementations NeurIPS 2020 Florian Wenzel, Jasper Snoek, Dustin Tran, Rodolphe Jenatton

Ensembles over neural network weights trained from different random initialization, known as deep ensembles, achieve state-of-the-art accuracy and calibration.

Image Classification Uncertainty Quantification

On Mixup Regularization

1 code implementation10 Jun 2020 Luigi Carratino, Moustapha Cissé, Rodolphe Jenatton, Jean-Philippe Vert

We show that Mixup can be interpreted as standard empirical risk minimization estimator subject to a combination of data transformation and random perturbation of the transformed data.

Ranked #75 on Image Classification on ObjectNet (using extra training data)

Data Augmentation Image Classification

A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise

no code implementations15 Mar 2020 Mark Collier, Basil Mustafa, Efi Kokiopoulou, Rodolphe Jenatton, Jesse Berent

By tuning the softmax temperature, we improve accuracy, log-likelihood and calibration on both image classification benchmarks with controlled label noise as well as Imagenet-21k which has naturally occurring label noise.

General Classification Image Classification +2

How Good is the Bayes Posterior in Deep Neural Networks Really?

1 code implementation ICML 2020 Florian Wenzel, Kevin Roth, Bastiaan S. Veeling, Jakub Świątkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, Sebastian Nowozin

In this work we cast doubt on the current understanding of Bayes posteriors in popular deep neural networks: we demonstrate through careful MCMC sampling that the posterior predictive induced by the Bayes posterior yields systematically worse predictions compared to simpler methods including point estimates obtained from SGD.

Bayesian Inference Uncertainty Quantification

Constrained Bayesian Optimization with Max-Value Entropy Search

no code implementations15 Oct 2019 Valerio Perrone, Iaroslav Shcherbatyi, Rodolphe Jenatton, Cedric Archambeau, Matthias Seeger

We propose constrained Max-value Entropy Search (cMES), a novel information theoretic-based acquisition function implementing this formulation.

Bayesian Optimization Hyperparameter Optimization

Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning

no code implementations NeurIPS 2019 Valerio Perrone, Huibin Shen, Matthias Seeger, Cedric Archambeau, Rodolphe Jenatton

Despite its simplicity, we show that our approach considerably boosts BO by reducing the size of the search space, thus accelerating the optimization of a variety of black-box optimization problems.

Bayesian Optimization Hyperparameter Optimization +1

Scalable Hyperparameter Transfer Learning

no code implementations NeurIPS 2018 Valerio Perrone, Rodolphe Jenatton, Matthias W. Seeger, Cedric Archambeau

Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization, such as hyperparameter optimization.

Bayesian Optimization Hyperparameter Optimization +2

Bayesian Optimization with Tree-structured Dependencies

no code implementations ICML 2017 Rodolphe Jenatton, Cedric Archambeau, Javier González, Matthias Seeger

The benefit of leveraging this structure is twofold: we explore the search space more efficiently and posterior inference scales more favorably with the number of observations than Gaussian Process-based approaches published in the literature.

Bayesian Optimization Binary Classification +1

Online optimization and regret guarantees for non-additive long-term constraints

no code implementations17 Feb 2016 Rodolphe Jenatton, Jim Huang, Dominik Csiba, Cedric Archambeau

We consider online optimization in the 1-lookahead setting, where the objective does not decompose additively over the rounds of the online game.

Adaptive Algorithms for Online Convex Optimization with Long-term Constraints

no code implementations23 Dec 2015 Rodolphe Jenatton, Jim Huang, Cédric Archambeau

We present an adaptive online gradient descent algorithm to solve online convex optimization problems with long-term constraints , which are constraints that need to be satisfied when accumulated over a finite number of rounds T , but can be violated in intermediate rounds.

Sparse and spurious: dictionary learning with noise and outliers

no code implementations19 Jul 2014 Rémi Gribonval, Rodolphe Jenatton, Francis Bach

A popular approach within the signal processing and machine learning communities consists in modelling signals as sparse linear combinations of atoms selected from a learned dictionary.

Dictionary Learning

On The Sample Complexity of Sparse Dictionary Learning

no code implementations20 Mar 2014 Matthias Seibert, Martin Kleinsteuber, Rémi Gribonval, Rodolphe Jenatton, Francis Bach

The main goal of this paper is to provide a sample complexity estimate that controls to what extent the empirical average deviates from the cost function.

Dictionary Learning

Sample Complexity of Dictionary Learning and other Matrix Factorizations

no code implementations13 Dec 2013 Rémi Gribonval, Rodolphe Jenatton, Francis Bach, Martin Kleinsteuber, Matthias Seibert

Many modern tools in machine learning and signal processing, such as sparse dictionary learning, principal component analysis (PCA), non-negative matrix factorization (NMF), $K$-means clustering, etc., rely on the factorization of a matrix obtained by concatenating high-dimensional vectors from a training collection.

Clustering Dictionary Learning +1

Convex Relaxations for Permutation Problems

no code implementations NeurIPS 2013 Fajwel Fogel, Rodolphe Jenatton, Francis Bach, Alexandre d'Aspremont

Seriation seeks to reconstruct a linear order between variables using unsorted similarity information.

A latent factor model for highly multi-relational data

no code implementations NeurIPS 2012 Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, Guillaume R. Obozinski

While there is a large body of work focused on modeling these data, few considered modeling these multiple types of relationships jointly.

Network Flow Algorithms for Structured Sparsity

no code implementations NeurIPS 2010 Julien Mairal, Rodolphe Jenatton, Francis R. Bach, Guillaume R. Obozinski

Our algorithm scales up to millions of groups and variables, and opens up a whole new range of applications for structured sparse models.

Structured Sparse Principal Component Analysis

no code implementations8 Sep 2009 Rodolphe Jenatton, Guillaume Obozinski, Francis Bach

We present an extension of sparse PCA, or sparse dictionary learning, where the sparsity patterns of all dictionary elements are structured and constrained to belong to a prespecified set of shapes.

Dictionary Learning Face Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.