Search Results for author:
Found 38 papers, 19 papers with code
Federated Learning in Practice: Reflections and Projections
Federated Learning (FL) is a machine learning technique that enables multiple entities to collaboratively learn a shared model without exchanging their local data.
A Hassle-free Algorithm for Private Learning in Practice: Don't Use Tree Aggregation, Use BLTs
Our BLT-DP-FTRL maintains the ease-of-use advantages of tree aggregation, while essentially matching matrix factorization in terms of utility and privacy.
Fine-Tuning Large Language Models with User-Level Differential Privacy
We investigate practical and scalable algorithms for training large language models (LLMs) with user-level differential privacy (DP) in order to provably safeguard all the examples contributed by each user.
Efficient and Near-Optimal Noise Generation for Streaming Differential Privacy
Existing algorithms for differentially private continual counting are either inefficient in terms of their space usage or add an excessive amount of noise, inducing suboptimal utility.
Federated Learning of Gboard Language Models with Differential Privacy
We train language models (LMs) with federated learning (FL) and differential privacy (DP) in the Google Keyboard (Gboard).
Can Public Large Language Models Help Private Cross-device Federated Learning?
We study (differentially) private federated learning (FL) of language models.
An Empirical Evaluation of Federated Contextual Bandit Algorithms
As the adoption of federated learning increases for learning from sensitive data local to user devices, it is natural to ask if the learning can be done using implicit signals generated as users interact with the applications of interest, rather than requiring access to explicit labels which can be difficult to acquire in many tasks.
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
However, while some adoption of DP has happened in industry, attempts to apply DP to real world complex ML models are still few and far between.
One-shot Empirical Privacy Estimation for Federated Learning
Privacy estimation techniques for differentially private (DP) algorithms are useful for comparing against analytical bounds, or to empirically measure privacy loss in settings where known analytical bounds are not tight.
Differentially Private Adaptive Optimization with Delayed Preconditioners
Privacy noise may negate the benefits of using adaptive optimizers in differentially private model training.
Learning to Generate Image Embeddings with User-level Differential Privacy
Small on-device models have been successfully trained with user-level differential privacy (DP) for next word prediction and image classification tasks in the past.
Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learning
We formalize the problem of DP mechanisms for adaptive streams with multiple participations and introduce a non-trivial extension of online matrix factorization DP mechanisms to our setting.
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
Training Production Language Models without Memorizing User Data
This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (DP-FedAvg) technique.
Privacy Amplification via Random Check-Ins
It has privacy/accuracy trade-offs similar to privacy amplification by subsampling/shuffling.
Adaptive Federated Optimization
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own training data.
Is Local SGD Better than Minibatch SGD?
We study local SGD (also known as parallel SGD and federated averaging), a natural and frequently used stochastic distributed optimization method.
Advances and Open Problems in Federated Learning
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
Can You Really Backdoor Federated Learning?
This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task.
Generative Models for Effective ML on Private, Decentralized Datasets
To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact.
Differentially Private Learning with Adaptive Clipping
Existing approaches for training neural networks with user-level differential privacy (e. g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contribution of each user's model update by clipping it to some constant value.
Semi-Cyclic Stochastic Gradient Descent
We consider convex SGD updates with a block-cyclic structure, i. e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Towards Federated Learning at Scale: System Design
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data.
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation.
A General Approach to Adding Differential Privacy to Iterative Training Procedures
In this work we address the practical challenges of training machine learning models on privacy-sensitive datasets by introducing a modular approach that minimizes changes to training algorithms, provides a variety of configuration strategies for the privacy mechanism, and then isolates and simplifies the critical logic that computes the final privacy guarantees.
LEAF: A Benchmark for Federated Settings
Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day.
cpSGD: Communication-efficient and differentially-private distributed SGD
Distributed stochastic gradient descent is an important subroutine in distributed learning.
Learning Differentially Private Recurrent Language Models
We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy.
On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches
The recent, remarkable growth of machine learning has led to intense interest in the privacy of the data on which machine learning relies, and to new techniques for preserving privacy.
Practical Secure Aggregation for Federated Learning on User-Held Data
Secure Aggregation protocols allow a collection of mutually distrust parties, each holding a private value, to collaboratively compute the sum of those values without revealing the values themselves.
Distributed Mean Estimation with Limited Communication
Motivated by the need for distributed learning and optimization algorithms with low communication cost, we study communication efficient algorithms for distributed mean estimation.
Federated Learning: Strategies for Improving Communication Efficiency
We consider learning algorithms for this setting where on each round, each client independently computes an update to the current model based on its local data, and communicates this update to a central server, where the client-side updates are aggregated to compute a new global model.
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
We refer to this setting as Federated Optimization.
Deep Learning with Differential Privacy
Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains.
Communication-Efficient Learning of Deep Networks from Decentralized Data
Modern mobile devices have access to a wealth of data suitable for learning models, which in turn can greatly improve the user experience on the device.
A Survey of Algorithms and Analysis for Adaptive Online Learning
Further, we prove a general and exact equivalence between an arbitrary adaptive Mirror Descent algorithm and a correspond- ing FTRL update, which allows us to analyze any Mirror Descent algorithm in the same framework.
Unconstrained Online Linear Learning in Hilbert Spaces: Minimax Algorithms and Normal Approximations
When $T$ is known, we derive an algorithm with an optimal regret bound (up to constant factors).
