Search Results for author: Aurélien Bellet

Found 51 papers, 18 papers with code

The Relative Gaussian Mechanism and its Application to Private Gradient Descent

no code implementations29 Aug 2023 Hadrien Hendrikx, Paul Mangold, Aurélien Bellet

Leveraging this assumption, we introduce the Relative Gaussian Mechanism (RGM), in which the variance of the noise depends on the norm of the output.

Improved Stability and Generalization Analysis of the Decentralized SGD Algorithm

no code implementations5 Jun 2023 Batiste Le Bars, Aurélien Bellet, Marc Tommasi

This paper presents a new generalization error analysis for the Decentralized Stochastic Gradient Descent (D-SGD) algorithm based on algorithmic stability.

Generalization Bounds

Fair Without Leveling Down: A New Intersectional Fairness Definition

no code implementations21 May 2023 Gaurav Maheshwari, Aurélien Bellet, Pascal Denis, Mikaela Keller

In this work, we consider the problem of intersectional group fairness in the classification setting, where the objective is to learn discrimination-free models in the presence of several intersecting sensitive groups.


From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning

1 code implementation24 Feb 2023 Edwige Cyffers, Aurélien Bellet, Debabrota Basu

We study differentially private (DP) machine learning algorithms as instances of noisy fixed-point iterations, in order to derive privacy and utility results from this well-studied framework.

Federated Learning

One-Shot Federated Conformal Prediction

1 code implementation13 Feb 2023 Pierre Humbert, Batiste Le Bars, Aurélien Bellet, Sylvain Arlot

In this paper, we introduce a conformal prediction method to construct prediction sets in a oneshot federated learning setting.

Conformal Prediction Federated Learning

Collaborative Algorithms for Online Personalized Mean Estimation

1 code implementation24 Aug 2022 Mahsa Asadi, Aurélien Bellet, Odalric-Ambrym Maillard, Marc Tommasi

We study the case where some of the distributions have the same mean, and the agents are allowed to actively query information from other agents.

Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averaging

1 code implementation10 Jun 2022 Edwige Cyffers, Mathieu Even, Aurélien Bellet, Laurent Massoulié

In this work, we introduce pairwise network differential privacy, a relaxation of LDP that captures the fact that the privacy leakage from a node $u$ to a node $v$ may depend on their relative position in the graph.

Graph Matching

Fair NLP Models with Differentially Private Text Encoders

1 code implementation12 May 2022 Gaurav Maheshwari, Pascal Denis, Mikaela Keller, Aurélien Bellet

Encoded text representations often capture sensitive attributes about individuals (e. g., race or gender), which raise privacy concerns and can make downstream models unfair to certain groups.


Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data

no code implementations9 Apr 2022 Batiste Le Bars, Aurélien Bellet, Marc Tommasi, Erick Lavoie, Anne-Marie Kermarrec

One of the key challenges in decentralized and federated learning is to design algorithms that efficiently deal with highly heterogeneous data distributions across agents.

Federated Learning

Differentially Private Speaker Anonymization

no code implementations23 Feb 2022 Ali Shahin Shamsabadi, Brij Mohan Lal Srivastava, Aurélien Bellet, Nathalie Vauquier, Emmanuel Vincent, Mohamed Maouche, Marc Tommasi, Nicolas Papernot

We remove speaker information from these attributes by introducing differentially private feature extractors based on an autoencoder and an automatic speech recognizer, respectively, trained using noise layers.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Mitigating Leakage from Data Dependent Communications in Decentralized Computing using Differential Privacy

no code implementations23 Dec 2021 Riad Ladjel, Nicolas Anciaux, Aurélien Bellet, Guillaume Scerri

In this paper, we define a general execution model to control the data-dependence of communications in user-side decentralized computations, in which differential privacy guarantees for communication patterns in global execution plans can be analyzed by combining guarantees obtained on local clusters of nodes.

Differentially Private Federated Learning on Heterogeneous Data

1 code implementation17 Nov 2021 Maxence Noble, Aurélien Bellet, Aymeric Dieuleveut

Federated Learning (FL) is a paradigm for large-scale distributed learning which faces two key challenges: (i) efficient training from highly heterogeneous user data, and (ii) protecting the privacy of participating users.

Federated Learning

Differentially Private Coordinate Descent for Composite Empirical Risk Minimization

no code implementations22 Oct 2021 Paul Mangold, Aurélien Bellet, Joseph Salmon, Marc Tommasi

In this paper, we propose Differentially Private proximal Coordinate Descent (DP-CD), a new method to solve composite DP-ERM problems.

Federated Multi-Task Learning under a Mixture of Distributions

4 code implementations NeurIPS 2021 Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, Richard Vidal

The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models.

Fairness Multi-Task Learning +1

D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning

no code implementations15 Apr 2021 Aurélien Bellet, Anne-Marie Kermarrec, Erick Lavoie

The convergence speed of machine learning models trained with Federated Learning is significantly affected by heterogeneous data partitions, even more so in a fully decentralized setting without a central server.

Federated Learning

Privacy Amplification by Decentralization

1 code implementation9 Dec 2020 Edwige Cyffers, Aurélien Bellet

In this work, we introduce a novel relaxation of local differential privacy (LDP) that naturally arises in fully decentralized algorithms, i. e., when participants exchange information by communicating along the edges of a network graph without central coordinator.

Federated Learning

An Accurate, Scalable and Verifiable Protocol for Federated Differentially Private Averaging

no code implementations12 Jun 2020 César Sabater, Aurélien Bellet, Jan Ramon

Learning from data owned by several parties, as in federated learning, raises challenges regarding the privacy guarantees provided to participants and the correctness of the computation in the presence of malicious parties.

Federated Learning

Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints

no code implementations19 Feb 2020 Robin Vogel, Aurélien Bellet, Stephan Clémençon

We establish generalization bounds for scoring functions learned under such constraints, design practical learning algorithms and show the relevance our approach with numerical experiments on real and synthetic data.

Fairness Generalization Bounds +1

Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?

no code implementations12 Nov 2019 Brij Mohan Lal Srivastava, Aurélien Bellet, Marc Tommasi, Emmanuel Vincent

In this paper, we focus on the protection of speaker identity and study the extent to which users can be recognized based on the encoded representation of their speech as obtained by a deep encoder-decoder architecture trained for ASR.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Private Protocols for U-Statistics in the Local Model and Beyond

no code implementations9 Oct 2019 James Bell, Aurélien Bellet, Adrià Gascón, tejas kulkarni

In this paper, we study the problem of computing $U$-statistics of degree $2$, i. e., quantities that come in the form of averages over pairs of data points, in the local model of differential privacy (LDP).

Clustering Metric Learning

metric-learn: Metric Learning Algorithms in Python

4 code implementations13 Aug 2019 William de Vazelhes, CJ Carey, Yuan Tang, Nathalie Vauquier, Aurélien Bellet

metric-learn is an open source Python package implementing supervised and weakly-supervised distance metric learning algorithms.

BIG-bench Machine Learning Metric Learning +1

Trade-offs in Large-Scale Distributed Tuplewise Estimation and Learning

1 code implementation21 Jun 2019 Robin Vogel, Aurélien Bellet, Stephan Clémençon, Ons Jelassi, Guillaume Papa

The development of cluster computing frameworks has allowed practitioners to scale out various statistical estimation and machine learning algorithms with minimal programming effort.

BIG-bench Machine Learning Clustering +1

Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs

1 code implementation24 Jan 2019 Valentina Zantedeschi, Aurélien Bellet, Marc Tommasi

We consider the fully decentralized machine learning scenario where many users with personal datasets collaborate to learn models through local peer-to-peer exchanges, without a central coordinator.

Escaping the Curse of Dimensionality in Similarity Learning: Efficient Frank-Wolfe Algorithm and Generalization Bounds

1 code implementation20 Jul 2018 Kuan Liu, Aurélien Bellet

Our experiments on datasets with up to one million features demonstrate the ability of our approach to generalize well despite the high dimensionality as well as its superiority compared to several competing methods.

Generalization Bounds Metric Learning

A Probabilistic Theory of Supervised Similarity Learning for Pointwise ROC Curve Optimization

no code implementations ICML 2018 Robin Vogel, Aurélien Bellet, Stéphan Clémençon

In this paper, similarity learning is investigated from the perspective of pairwise bipartite ranking, where the goal is to rank the elements of a database by decreasing order of the probability that they share the same label with some query data point, based on the similarity scores.

Metric Learning

Hiding in the Crowd: A Massively Distributed Algorithm for Private Averaging with Malicious Adversaries

no code implementations27 Mar 2018 Pierre Dellenbach, Aurélien Bellet, Jan Ramon

The amount of personal data collected in our everyday interactions with connected devices offers great opportunities for innovative services fueled by machine learning, as well as raises serious concerns for the privacy of individuals.

A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm

1 code implementation20 Dec 2017 Wenjie Zheng, Aurélien Bellet, Patrick Gallinari

We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint.

Personalized and Private Peer-to-Peer Machine Learning

no code implementations23 May 2017 Aurélien Bellet, Rachid Guerraoui, Mahsa Taziki, Marc Tommasi

The rise of connected personal devices together with privacy concerns call for machine learning algorithms capable of leveraging the data of a large number of agents to learn personalized models under strong privacy requirements.

BIG-bench Machine Learning

Kernel Approximation Methods for Speech Recognition

no code implementations13 Jan 2017 Avner May, Alireza Bagheri Garakani, Zhiyun Lu, Dong Guo, Kuan Liu, Aurélien Bellet, Linxi Fan, Michael Collins, Daniel Hsu, Brian Kingsbury, Michael Picheny, Fei Sha

First, in order to reduce the number of random features required by kernel models, we propose a simple but effective method for feature selection.

feature selection speech-recognition +1

On Graph Reconstruction via Empirical Risk Minimization: Fast Learning Rates and Scalability

no code implementations NeurIPS 2016 Guillaume Papa, Aurélien Bellet, Stephan Clémençon

The problem of predicting connections between a set of data points finds many applications, in systems biology and social network analysis among others.

Clustering Graph Reconstruction

Decentralized Collaborative Learning of Personalized Models over Networks

no code implementations17 Oct 2016 Paul Vanhaesebrouck, Aurélien Bellet, Marc Tommasi

We consider a set of learning agents in a collaborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective.

SGD Algorithms based on Incomplete U-statistics: Large-Scale Minimization of Empirical Risk

no code implementations NeurIPS 2015 Guillaume Papa, Stéphan Clémençon, Aurélien Bellet

In many learning problems, ranging from clustering to ranking through metric learning, empirical estimates of the risk functional consist of an average over tuples (e. g., pairs or triplets) of observations, rather than over individual observations.

Clustering Metric Learning

Extending Gossip Algorithms to Distributed Estimation of U-Statistics

no code implementations NeurIPS 2015 Igor Colin, Aurélien Bellet, Joseph Salmon, Stéphan Clémençon

Efficient and robust algorithms for decentralized estimation in networks are essential to many distributed systems.

Scaling-up Empirical Risk Minimization: Optimization of Incomplete U-statistics

no code implementations12 Jan 2015 Stéphan Clémençon, Aurélien Bellet, Igor Colin

In a wide range of statistical learning problems such as ranking, clustering or metric learning among others, the risk is accurately estimated by $U$-statistics of degree $d\geq 1$, i. e. functionals of the training data with low variance that take the form of averages over $k$-tuples.

Clustering Metric Learning +1

Similarity Learning for High-Dimensional Sparse Data

1 code implementation10 Nov 2014 Kuan Liu, Aurélien Bellet, Fei Sha

A good measure of similarity between data points is crucial to many tasks in machine learning.

Dimensionality Reduction Metric Learning +1

Sparse Compositional Metric Learning

no code implementations15 Apr 2014 Yuan Shi, Aurélien Bellet, Fei Sha

We propose a new approach for metric learning by framing it as learning a sparse combination of locally discriminative metrics that are inexpensive to generate from the training data.

General Classification Metric Learning

A Distributed Frank-Wolfe Algorithm for Communication-Efficient Sparse Learning

no code implementations9 Apr 2014 Aurélien Bellet, YIngyu Liang, Alireza Bagheri Garakani, Maria-Florina Balcan, Fei Sha

We further show that the communication cost of dFW is optimal by deriving a lower-bound on the communication cost required to construct an $\epsilon$-approximate solution.

Sparse Learning

Supervised Metric Learning with Generalization Guarantees

no code implementations17 Jul 2013 Aurélien Bellet

In our third contribution, we extend these ideas to metric learning from feature vectors by proposing a bilinear similarity learning method that efficiently optimizes the (e, g, t)-goodness.

Generalization Bounds Metric Learning

A Survey on Metric Learning for Feature Vectors and Structured Data

no code implementations28 Jun 2013 Aurélien Bellet, Amaury Habrard, Marc Sebban

The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult.

BIG-bench Machine Learning Metric Learning

Robustness and Generalization for Metric Learning

no code implementations5 Sep 2012 Aurélien Bellet, Amaury Habrard

Metric learning has attracted a lot of interest over the last decade, but the generalization ability of such methods has not been thoroughly studied.

Generalization Bounds Metric Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.