Leveraging Function Space Aggregation for Federated Learning at Scale

no code implementations • 17 Nov 2023 • Nikita Dhawan, Nicole Mitchell, Zachary Charles, Zachary Garrett, Gintare Karolina Dziugaite

Many federated learning algorithms, including the canonical Federated Averaging (FedAvg), take a direct (possibly weighted) average of the client parameter updates, motivated by results in distributed optimization.

Distributed Optimization Federated Learning

Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning

1 code implementation • NeurIPS 2023 • Zachary Charles, Nicole Mitchell, Krishna Pillutla, Michael Reneer, Zachary Garrett

Finally, it is framework-agnostic.

Federated Learning Language Modelling +1

JaxPruner: A concise library for sparsity research

1 code implementation • 27 Apr 2023 • Joo Hyung Lee, Wonpyo Park, Nicole Mitchell, Jonathan Pilault, Johan Obando-Ceron, Han-Byul Kim, Namhoon Lee, Elias Frantar, Yun Long, Amir Yazdanbakhsh, Shivani Agrawal, Suvinay Subramanian, Xin Wang, Sheng-Chun Kao, Xingyao Zhang, Trevor Gale, Aart Bik, Woohyun Han, Milen Ferev, Zhonglin Han, Hong-Seok Kim, Yann Dauphin, Gintare Karolina Dziugaite, Pablo Samuel Castro, Utku Evci

This paper introduces JaxPruner, an open-source JAX-based pruning and sparse training library for machine learning research.

Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory

1 code implementation • 7 Jan 2022 • Nicole Mitchell, Johannes Ballé, Zachary Charles, Jakub Konečný

A significant bottleneck in federated learning (FL) is the network communication cost of sending model updates from client devices to the central server.

Federated Learning Quantization