Search Results for author: Rodolphe Jenatton

Found 30 papers, 5 papers with code

Deep Classifiers with Label Noise Modeling and Distance Awareness

no code implementations6 Oct 2021 Vincent Fortuin, Mark Collier, Florian Wenzel, James Allingham, Jeremiah Liu, Dustin Tran, Balaji Lakshminarayanan, Jesse Berent, Rodolphe Jenatton, Effrosyni Kokiopoulou

Uncertainty estimation in deep learning has recently emerged as a crucial area of interest to advance reliability and robustness in safety-critical applications.

Out-of-Distribution Detection

Scaling Vision with Sparse Mixture of Experts

no code implementations NeurIPS 2021 Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, Neil Houlsby

We present a Vision MoE (V-MoE), a sparse version of the Vision Transformer, that is scalable and competitive with the largest dense networks.

 Ranked #1 on Image Classification on JFT-300M (using extra training data)

Few-Shot Image Classification

Distilling Ensembles Improves Uncertainty Estimates

no code implementations pproximateinference AABI Symposium 2021 Zelda E Mariet, Rodolphe Jenatton, Florian Wenzel, Dustin Tran

We seek to bridge the performance gap between batch ensembles (ensembles of deep networks with shared parameters) and deep ensembles on tasks which require not only predictions, but also uncertainty estimates for these predictions.

Training independent subnetworks for robust prediction

1 code implementation ICLR 2021 Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew M. Dai, Dustin Tran

Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network.

Hyperparameter Ensembles for Robustness and Uncertainty Quantification

2 code implementations NeurIPS 2020 Florian Wenzel, Jasper Snoek, Dustin Tran, Rodolphe Jenatton

Ensembles over neural network weights trained from different random initialization, known as deep ensembles, achieve state-of-the-art accuracy and calibration.

Image Classification

On Mixup Regularization

1 code implementation10 Jun 2020 Luigi Carratino, Moustapha Cissé, Rodolphe Jenatton, Jean-Philippe Vert

Mixup is a data augmentation technique that creates new examples as convex combinationsof training points and labels.

Data Augmentation

A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise

no code implementations15 Mar 2020 Mark Collier, Basil Mustafa, Efi Kokiopoulou, Rodolphe Jenatton, Jesse Berent

By tuning the softmax temperature, we improve accuracy, log-likelihood and calibration on both image classification benchmarks with controlled label noise as well as Imagenet-21k which has naturally occurring label noise.

Classification General Classification +2

How Good is the Bayes Posterior in Deep Neural Networks Really?

1 code implementation ICML 2020 Florian Wenzel, Kevin Roth, Bastiaan S. Veeling, Jakub Świątkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, Sebastian Nowozin

In this work we cast doubt on the current understanding of Bayes posteriors in popular deep neural networks: we demonstrate through careful MCMC sampling that the posterior predictive induced by the Bayes posterior yields systematically worse predictions compared to simpler methods including point estimates obtained from SGD.

Bayesian Inference

Constrained Bayesian Optimization with Max-Value Entropy Search

no code implementations15 Oct 2019 Valerio Perrone, Iaroslav Shcherbatyi, Rodolphe Jenatton, Cedric Archambeau, Matthias Seeger

We propose constrained Max-value Entropy Search (cMES), a novel information theoretic-based acquisition function implementing this formulation.

Hyperparameter Optimization

Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning

no code implementations NeurIPS 2019 Valerio Perrone, Huibin Shen, Matthias Seeger, Cedric Archambeau, Rodolphe Jenatton

Despite its simplicity, we show that our approach considerably boosts BO by reducing the size of the search space, thus accelerating the optimization of a variety of black-box optimization problems.

Hyperparameter Optimization Transfer Learning

Scalable Hyperparameter Transfer Learning

no code implementations NeurIPS 2018 Valerio Perrone, Rodolphe Jenatton, Matthias W. Seeger, Cedric Archambeau

Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization, such as hyperparameter optimization.

Hyperparameter Optimization Transfer Learning

Bayesian Optimization with Tree-structured Dependencies

no code implementations ICML 2017 Rodolphe Jenatton, Cedric Archambeau, Javier González, Matthias Seeger

The benefit of leveraging this structure is twofold: we explore the search space more efficiently and posterior inference scales more favorably with the number of observations than Gaussian Process-based approaches published in the literature.

Gaussian Processes

Online optimization and regret guarantees for non-additive long-term constraints

no code implementations17 Feb 2016 Rodolphe Jenatton, Jim Huang, Dominik Csiba, Cedric Archambeau

We consider online optimization in the 1-lookahead setting, where the objective does not decompose additively over the rounds of the online game.

Adaptive Algorithms for Online Convex Optimization with Long-term Constraints

no code implementations23 Dec 2015 Rodolphe Jenatton, Jim Huang, Cédric Archambeau

We present an adaptive online gradient descent algorithm to solve online convex optimization problems with long-term constraints , which are constraints that need to be satisfied when accumulated over a finite number of rounds T , but can be violated in intermediate rounds.

Sparse and spurious: dictionary learning with noise and outliers

no code implementations19 Jul 2014 Rémi Gribonval, Rodolphe Jenatton, Francis Bach

A popular approach within the signal processing and machine learning communities consists in modelling signals as sparse linear combinations of atoms selected from a learned dictionary.

Dictionary Learning

On The Sample Complexity of Sparse Dictionary Learning

no code implementations20 Mar 2014 Matthias Seibert, Martin Kleinsteuber, Rémi Gribonval, Rodolphe Jenatton, Francis Bach

The main goal of this paper is to provide a sample complexity estimate that controls to what extent the empirical average deviates from the cost function.

Dictionary Learning

Sample Complexity of Dictionary Learning and other Matrix Factorizations

no code implementations13 Dec 2013 Rémi Gribonval, Rodolphe Jenatton, Francis Bach, Martin Kleinsteuber, Matthias Seibert

Many modern tools in machine learning and signal processing, such as sparse dictionary learning, principal component analysis (PCA), non-negative matrix factorization (NMF), $K$-means clustering, etc., rely on the factorization of a matrix obtained by concatenating high-dimensional vectors from a training collection.

Dictionary Learning Generalization Bounds

Convex Relaxations for Permutation Problems

no code implementations NeurIPS 2013 Fajwel Fogel, Rodolphe Jenatton, Francis Bach, Alexandre d'Aspremont

Seriation seeks to reconstruct a linear order between variables using unsorted similarity information.

A latent factor model for highly multi-relational data

no code implementations NeurIPS 2012 Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, Guillaume R. Obozinski

While there is a large body of work focused on modeling these data, few considered modeling these multiple types of relationships jointly.

Network Flow Algorithms for Structured Sparsity

no code implementations NeurIPS 2010 Julien Mairal, Rodolphe Jenatton, Francis R. Bach, Guillaume R. Obozinski

Our algorithm scales up to millions of groups and variables, and opens up a whole new range of applications for structured sparse models.

Hierarchical structure

Structured Sparse Principal Component Analysis

no code implementations8 Sep 2009 Rodolphe Jenatton, Guillaume Obozinski, Francis Bach

We present an extension of sparse PCA, or sparse dictionary learning, where the sparsity patterns of all dictionary elements are structured and constrained to belong to a prespecified set of shapes.

Dictionary Learning Face Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.