Search Results for author:
Found 35 papers, 19 papers with code
Scaff-PD: Communication Efficient Fair and Robust Federated Learning
We present Scaff-PD, a fast and communication-efficient algorithm for distributionally robust federated learning.
Provably Personalized and Robust Federated Learning
Clustering clients with similar objectives and learning a model per cluster is an intuitive and interpretable approach to personalization in federated learning.
Evaluating and Incentivizing Diverse Data Contributions in Collaborative Learning
This creates a tension between the principal (the FL platform designer) who cares about global performance and the agents (the data collectors) who care about local performance.
Federated Conformal Predictors for Distributed Uncertainty Quantification
Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning since it can be easily applied as a post-processing step to already trained models.
Online Learning in a Creator Economy
In this paper, we initiate the study of online learning in the creator economy by modeling the creator economy as a three-party game between the users, platform, and content creators, with the platform interacting with the content creator under a principal-agent model through contracts to encourage better content.
FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings
In this work, we propose a novel cross-silo dataset suite focused on healthcare, FLamby (Federated Learning AMple Benchmark of Your cross-silo strategies), to bridge the gap between theory and practice of cross-silo FL.
TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels
Leveraging this observation, we propose a Train-Convexify-Train (TCT) procedure to sidestep this issue: first, learn features using off-the-shelf methods (e. g., FedAvg); then, optimize a convexified problem obtained from the network's empirical neural tangent kernel approximation.
Mechanisms that Incentivize Data Sharing in Federated Learning
Federated learning is typically considered a beneficial technology which allows multiple agents to collaborate with each other, improve the accuracy of their models, and solve problems which are otherwise too data-intensive / expensive to be solved individually.
Optimization with Access to Auxiliary Information
We investigate the fundamental optimization question of minimizing a target function $f$, whose gradients are expensive to compute or have limited availability, given access to some auxiliary side function $h$ whose gradients are cheap or more available.
Agree to Disagree: Diversity through Disagreement for Better Transferability
This behavior can hinder the transferability of trained models by (i) favoring the learning of simpler but spurious features -- present in the training data but absent from the test data -- and (ii) by only leveraging a small subset of predictive features.
Byzantine-Robust Decentralized Learning via ClippedGossip
In this paper, we study the challenging task of Byzantine-robust decentralized training on arbitrary communication graphs.
Breaking the centralized barrier for cross-device federated learning
Federated learning (FL) is a challenging setting for optimization due to the heterogeneity of the data across different clients which gives rise to the client drift phenomenon.
Linear Speedup in Personalized Collaborative Learning
Collaborative training can improve the accuracy of a model for a user by trading off the model's bias (introduced by using data from other users who are potentially different) against its variance (due to the limited amount of data on any single user).
Towards Model Agnostic Federated Learning Using Knowledge Distillation
Is it possible to design an universal API for federated learning using which an ad-hoc group of data-holders (agents) collaborate with each other and perform federated learning?
RelaySum for Decentralized Deep Learning on Heterogeneous Data
A key challenge, primarily in decentralized deep learning, remains the handling of differences between the workers' local data distributions.
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
In this paper, we investigate and identify the limitation of several decentralized optimization algorithms for different degrees of data heterogeneity.
Learning from History for Byzantine Robust Optimization
Secondly, we prove that even if the aggregation rules may succeed in limiting the influence of the attackers in a single round, the attackers can couple their attacks across time eventually leading to divergence.
Practical Low-Rank Communication Compression in Decentralized Deep Learning
Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models.
Byzantine-Robust Learning on Heterogeneous Datasets via Resampling
In Byzantine-robust distributed optimization, a central server wants to train a machine learning model over data distributed across multiple workers.
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
Federated learning (FL) is a challenging setting for optimization due to the heterogeneity of the data across different clients which gives rise to the client drift phenomenon.
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning
Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models.
Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing
In Byzantine robust distributed or federated learning, a central server wants to train a machine learning model over data distributed across multiple workers.
Secure Byzantine-Robust Machine Learning
Increasingly machine learning systems are being deployed to edge servers and devices (e. g. mobile phones) and trained in a collaborative manner.
Why are Adaptive Methods Good for Attention Models?
While stochastic gradient descent (SGD) is still the \emph{de facto} algorithm in deep learning, adaptive methods like Clipped SGD/Adam have been observed to outperform SGD across important tasks, such as attention models.
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
We obtain tight convergence rates for FedAvg and prove that it suffers from `client-drift' when the data is heterogeneous (non-iid), resulting in unstable and slow convergence.
Why ADAM Beats SGD for Attention Models
While stochastic gradient descent (SGD) is still the de facto algorithm in deep learning, adaptive methods like Adam have been observed to outperform SGD across important tasks, such as attention models.
The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication
We analyze (stochastic) gradient descent (SGD) with delayed updates on smooth quasi-convex and non-convex functions and derive concise, non-asymptotic, convergence rates.
Amplifying Rényi Differential Privacy via Shuffling
Differential privacy is a useful tool to build machine learning models which do not release too much information about the training data.
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization.
Accelerating Gradient Boosting Machine
This is the first GBM type of algorithm with theoretically-justified accelerated convergence rate.
Error Feedback Fixes SignSGD and other Gradient Compression Schemes
These issues arise because of the biased nature of the sign compression operator.
Efficient Greedy Coordinate Descent for Composite Problems
For these problems we provide (i) the first linear rates of convergence independent of $n$, and show that our greedy update rule provides speedups similar to those obtained in the smooth case.
Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients
We show that Newton's method converges globally at a linear rate for objective functions whose Hessians are stable.
On Matching Pursuit and Coordinate Descent
Exploiting the connection between the two algorithms, we present a unified analysis of both, providing affine invariant sublinear $\mathcal{O}(1/t)$ rates on smooth objectives and linear convergence on strongly convex objectives.
