Search Results for author:
Found 11 papers, 5 papers with code
Understanding Unintended Memorization in Language Models Under Federated Learning
Thus, we initiate a formal study to understand the effect of different components of FL on unintended memorization in trained NWP models.
UserLibri: A Dataset for ASR Personalization Using Only Text
We experiment on a user-clustered LibriSpeech corpus, supplemented with personalized text-only data for each user from Project Gutenberg.
Public Data-Assisted Mirror Descent for Private Model Training
In this paper, we revisit the problem of using in-distribution public data to improve the privacy/utility trade-offs for differentially private (DP) model training.
Revealing and Protecting Labels in Distributed Training
Prior works have demonstrated that labels can be revealed analytically from the last layer of certain models (e. g., ResNet), or they can be reconstructed jointly with model inputs by using Gradients Matching [Zhu et al'19] with additional knowledge about the current state of the model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter It
We show that a dropout rate of 0. 2 can reduce the speaker identity accuracy to 0% top-1 (0. 5% top-5).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Training Production Language Models without Memorizing User Data
This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (DP-FedAvg) technique.
Understanding Unintended Memorization in Federated Learning
In this paper, we initiate a formal study to understand the effect of different components of canonical FL on unintended memorization in trained models, comparing with the central learning setting.
Generative Models for Effective ML on Private, Decentralized Datasets
To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact.
Federated Learning for Emoji Prediction in a Mobile Keyboard
We show that a word-level recurrent neural network can predict emoji from text typed on a mobile keyboard.
Differentially Private Learning with Adaptive Clipping
Existing approaches for training neural networks with user-level differential privacy (e. g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contribution of each user's model update by clipping it to some constant value.
Federated Learning for Mobile Keyboard Prediction
We train a recurrent neural network language model using a distributed, on-device learning framework called federated learning for the purpose of next-word prediction in a virtual keyboard for smartphones.
