Search Results for author:
Found 54 papers, 19 papers with code
Joint Learning of the Graph and the Data Representation for Graph-Based Semi-Supervised Learning
Graph-based semi-supervised learning is appealing when labels are scarce but large amounts of unlabeled data are available.
Privacy Attacks in Decentralized Learning
Decentralized Gradient Descent (D-GD) allows a set of users to perform collaborative learning without sharing their data by iteratively averaging local model updates with their neighbors in a network graph.
Differentially Private Decentralized Learning with Random Walks
The popularity of federated learning comes from the possibility of better scalability and the ability for participants to keep control of their data, improving data security and sovereignty.
Rényi Pufferfish Privacy: General Additive Noise Mechanisms and Privacy Amplification by Iteration
Pufferfish privacy is a flexible generalization of differential privacy that allows to model arbitrary secrets and adversary's prior knowledge about the data.
The Relative Gaussian Mechanism and its Application to Private Gradient Descent
Leveraging this assumption, we introduce the Relative Gaussian Mechanism (RGM), in which the variance of the noise depends on the norm of the output.
Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm
On the contrary, we show, for convex, strongly convex and non-convex functions, that D-SGD can always recover generalization bounds analogous to those of classical SGD, suggesting that the choice of graph does not matter.
Fair Without Leveling Down: A New Intersectional Fairness Definition
In this work, we consider the problem of intersectional group fairness in the classification setting, where the objective is to learn discrimination-free models in the presence of several intersecting sensitive groups.
From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning
We study differentially private (DP) machine learning algorithms as instances of noisy fixed-point iterations, in order to derive privacy and utility results from this well-studied framework.
One-Shot Federated Conformal Prediction
In this paper, we introduce a conformal prediction method to construct prediction sets in a oneshot federated learning setting.
Differential Privacy has Bounded Impact on Fairness in Classification
We theoretically study the impact of differential privacy on fairness in classification.
FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings
In this work, we propose a novel cross-silo dataset suite focused on healthcare, FLamby (Federated Learning AMple Benchmark of Your cross-silo strategies), to bridge the gap between theory and practice of cross-silo FL.
Collaborative Algorithms for Online Personalized Mean Estimation
We study the case where some of the distributions have the same mean, and the agents are allowed to actively query information from other agents.
PEPPER: Empowering User-Centric Recommender Systems over Gossip Learning
To remedy this, we propose PEPPER, a decentralized recommender system based on gossip learning principles.
High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent
In this paper, we study differentially private empirical risk minimization (DP-ERM).
Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averaging
In this work, we introduce pairwise network differential privacy, a relaxation of LDP that captures the fact that the privacy leakage from a node $u$ to a node $v$ may depend on their relative position in the graph.
Fair NLP Models with Differentially Private Text Encoders
Encoded text representations often capture sensitive attributes about individuals (e. g., race or gender), which raise privacy concerns and can make downstream models unfair to certain groups.
Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data
One of the key challenges in decentralized and federated learning is to design algorithms that efficiently deal with highly heterogeneous data distributions across agents.
GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation
In this paper, we study the problem of learning Graph Neural Networks (GNNs) with Differential Privacy (DP).
Differentially Private Speaker Anonymization
We remove speaker information from these attributes by introducing differentially private feature extractors based on an autoencoder and an automatic speech recognizer, respectively, trained using noise layers.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Mitigating Leakage from Data Dependent Communications in Decentralized Computing using Differential Privacy
In this paper, we define a general execution model to control the data-dependence of communications in user-side decentralized computations, in which differential privacy guarantees for communication patterns in global execution plans can be analyzed by combining guarantees obtained on local clusters of nodes.
Differentially Private Federated Learning on Heterogeneous Data
Federated Learning (FL) is a paradigm for large-scale distributed learning which faces two key challenges: (i) efficient training from highly heterogeneous user data, and (ii) protecting the privacy of participating users.
Differentially Private Coordinate Descent for Composite Empirical Risk Minimization
In this paper, we propose Differentially Private proximal Coordinate Descent (DP-CD), a new method to solve composite DP-ERM problems.
Federated Multi-Task Learning under a Mixture of Distributions
The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models.
D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning
The convergence speed of machine learning models trained with Federated Learning is significantly affected by heterogeneous data partitions, even more so in a fully decentralized setting without a central server.
Privacy Amplification by Decentralization
In this work, we introduce a novel relaxation of local differential privacy (LDP) that naturally arises in fully decentralized algorithms, i. e., when participants exchange information by communicating along the edges of a network graph without central coordinator.
An Accurate, Scalable and Verifiable Protocol for Federated Differentially Private Averaging
Learning from data owned by several parties, as in federated learning, raises challenges regarding the privacy guarantees provided to participants and the correctness of the computation in the presence of malicious parties.
Design Choices for X-vector Based Speaker Anonymization
The recently proposed x-vector based anonymization scheme converts any input voice into that of a random pseudo-speaker.
Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints
We establish generalization bounds for scoring functions learned under such constraints, design practical learning algorithms and show the relevance our approach with numerical experiments on real and synthetic data.
Advances and Open Problems in Federated Learning
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?
In this paper, we focus on the protection of speaker identity and study the extent to which users can be recognized based on the encoded representation of their speech as obtained by a deep encoder-decoder architecture trained for ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Evaluating Voice Conversion-based Privacy Protection against Informed Attackers
In this paper, we investigate anonymization methods based on voice conversion.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Private Protocols for U-Statistics in the Local Model and Beyond
In this paper, we study the problem of computing $U$-statistics of degree $2$, i. e., quantities that come in the form of averages over pairs of data points, in the local model of differential privacy (LDP).
metric-learn: Metric Learning Algorithms in Python
metric-learn is an open source Python package implementing supervised and weakly-supervised distance metric learning algorithms.
Trade-offs in Large-Scale Distributed Tuplewise Estimation and Learning
The development of cluster computing frameworks has allowed practitioners to scale out various statistical estimation and machine learning algorithms with minimal programming effort.
Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs
We consider the fully decentralized machine learning scenario where many users with personal datasets collaborate to learn models through local peer-to-peer exchanges, without a central coordinator.
Escaping the Curse of Dimensionality in Similarity Learning: Efficient Frank-Wolfe Algorithm and Generalization Bounds
Our experiments on datasets with up to one million features demonstrate the ability of our approach to generalize well despite the high dimensionality as well as its superiority compared to several competing methods.
A Probabilistic Theory of Supervised Similarity Learning for Pointwise ROC Curve Optimization
In this paper, similarity learning is investigated from the perspective of pairwise bipartite ranking, where the goal is to rank the elements of a database by decreasing order of the probability that they share the same label with some query data point, based on the similarity scores.
Hiding in the Crowd: A Massively Distributed Algorithm for Private Averaging with Malicious Adversaries
The amount of personal data collected in our everyday interactions with connected devices offers great opportunities for innovative services fueled by machine learning, as well as raises serious concerns for the privacy of individuals.
A Distributed Frank-Wolfe Framework for Learning Low-Rank Matrices with the Trace Norm
We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint.
Personalized and Private Peer-to-Peer Machine Learning
The rise of connected personal devices together with privacy concerns call for machine learning algorithms capable of leveraging the data of a large number of agents to learn personalized models under strong privacy requirements.
Kernel Approximation Methods for Speech Recognition
First, in order to reduce the number of random features required by kernel models, we propose a simple but effective method for feature selection.
On Graph Reconstruction via Empirical Risk Minimization: Fast Learning Rates and Scalability
The problem of predicting connections between a set of data points finds many applications, in systems biology and social network analysis among others.
Decentralized Collaborative Learning of Personalized Models over Networks
We consider a set of learning agents in a collaborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective.
Gossip Dual Averaging for Decentralized Optimization of Pairwise Functions
In decentralized networks (of sensors, connected objects, etc.
SGD Algorithms based on Incomplete U-statistics: Large-Scale Minimization of Empirical Risk
In many learning problems, ranging from clustering to ranking through metric learning, empirical estimates of the risk functional consist of an average over tuples (e. g., pairs or triplets) of observations, rather than over individual observations.
Extending Gossip Algorithms to Distributed Estimation of U-Statistics
Efficient and robust algorithms for decentralized estimation in networks are essential to many distributed systems.
Scaling-up Empirical Risk Minimization: Optimization of Incomplete U-statistics
In a wide range of statistical learning problems such as ranking, clustering or metric learning among others, the risk is accurately estimated by $U$-statistics of degree $d\geq 1$, i. e. functionals of the training data with low variance that take the form of averages over $k$-tuples.
How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets
The computational complexity of kernel methods has often been a major barrier for applying them to large-scale learning problems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Similarity Learning for High-Dimensional Sparse Data
A good measure of similarity between data points is crucial to many tasks in machine learning.
Sparse Compositional Metric Learning
We propose a new approach for metric learning by framing it as learning a sparse combination of locally discriminative metrics that are inexpensive to generate from the training data.
A Distributed Frank-Wolfe Algorithm for Communication-Efficient Sparse Learning
We further show that the communication cost of dFW is optimal by deriving a lower-bound on the communication cost required to construct an $\epsilon$-approximate solution.
Supervised Metric Learning with Generalization Guarantees
In our third contribution, we extend these ideas to metric learning from feature vectors by proposing a bilinear similarity learning method that efficiently optimizes the (e, g, t)-goodness.
A Survey on Metric Learning for Feature Vectors and Structured Data
The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult.
Robustness and Generalization for Metric Learning
Metric learning has attracted a lot of interest over the last decade, but the generalization ability of such methods has not been thoroughly studied.
