no code implementations • 25 Apr 2024 • Jiachen Liu, Zhiyu Wu, Jae-Won Chung, Fan Lai, Myungjin Lee, Mosharaf Chowdhury
The advent of large language models (LLMs) has transformed text-based services, enabling capabilities ranging from real-time translation to AI-driven chatbots.
no code implementations • 21 Apr 2024 • Yuxuan Zhu, Jiachen Liu, Mosharaf Chowdhury, Fan Lai
Federated learning (FL) aims to train machine learning (ML) models across potentially millions of edge client devices.
no code implementations • 9 Feb 2024 • Haizhong Zheng, Xiaoyan Bai, Beidi Chen, Fan Lai, Atul Prakash
The emergence of activation sparsity in LLMs provides a natural approach to reduce this cost by involving only parts of the parameters for inference.
no code implementations • 9 Jan 2024 • Qinyi Luo, Penghan Wang, Wei zhang, Fan Lai, Jiachen Mao, Xiaohan Wei, Jun Song, Wei-Yu Tsai, Shuai Yang, Yuxi Hu, Xuehai Qian
Huge embedding tables in modern Deep Learning Recommender Models (DLRM) require prohibitively large memory during training and inference.
no code implementations • 13 Dec 2023 • Jiachen Liu, Fan Lai, Ding Ding, Yiwen Zhang, Mosharaf Chowdhury
Scheduling edge resources among multiple FL jobs is different from GPU scheduling for cloud ML because of the ephemeral nature and planetary scale of participating devices as well as the overlapping resource requirements of diverse FL jobs.
no code implementations • 29 Oct 2022 • Jiachen Liu, Fan Lai, Yinwei Dai, Aditya Akella, Harsha Madhyastha, Mosharaf Chowdhury
In this paper, we explore an additional layer of complexity to mitigate such heterogeneity by grouping clients with statistically similar data distributions (cohorts).
1 code implementation • 28 Oct 2022 • Haizhong Zheng, Rui Liu, Fan Lai, Atul Prakash
We then propose a novel one-shot coreset selection method, Coverage-centric Coreset Selection (CCS), that jointly considers overall data coverage upon a distribution as well as the importance of each example.
no code implementations • 9 Jun 2022 • Sanjay Sri Vallabh Singapuram, Fan Lai, Chuheng Hu, Mosharaf Chowdhury
The need to train DNN models on end-user devices (e. g., smartphones) is increasing with the need to improve data privacy and reduce communication overheads.
no code implementations • 17 Jan 2022 • Yiding Wang, Decang Sun, Kai Chen, Fan Lai, Mosharaf Chowdhury
To explore this, we first introduce the notion of training plasticity to quantify the training progress of internal DNN layers.
1 code implementation • 21 Jul 2021 • Naichen Shi, Fan Lai, Raed Al Kontar, Mosharaf Chowdhury
In this paper we propose Fed-ensemble: a simple approach that bringsmodel ensembling to federated learning (FL).
3 code implementations • 24 May 2021 • Fan Lai, Yinwei Dai, Sanjay S. Singapuram, Jiachen Liu, Xiangfeng Zhu, Harsha V. Madhyastha, Mosharaf Chowdhury
We present FedScale, a federated learning (FL) benchmarking suite with realistic datasets and a scalable runtime to enable reproducible FL research.
1 code implementation • 12 Oct 2020 • Fan Lai, Xiangfeng Zhu, Harsha V. Madhyastha, Mosharaf Chowdhury
In this paper, we propose Oort to improve the performance of federated training and testing with guided participant selection.