AgentScope: A Flexible yet Robust Multi-Agent Platform

1 code implementation • 21 Feb 2024 • Dawei Gao, Zitao Li, Weirui Kuang, Xuchen Pan, Daoyuan Chen, Zhijian Ma, Bingchen Qian, Liuyi Yao, Lin Zhu, Chen Cheng, Hongzhu Shi, Yaliang Li, Bolin Ding, Jingren Zhou

With the rapid advancement of Large Language Models (LLMs), significant progress has been made in multi-agent applications.

EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models

1 code implementation • 1 Feb 2024 • Xuchen Pan, Yanxi Chen, Yaliang Li, Bolin Ding, Jingren Zhou

This work introduces EE-Tuning, a lightweight and economical solution to training/tuning early-exit large language models (LLMs).

EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism

1 code implementation • 8 Dec 2023 • Yanxi Chen, Xuchen Pan, Yaliang Li, Bolin Ding, Jingren Zhou

We present EE-LLM, a framework for large-scale training and inference of early-exit large language models (LLMs).

Data-Juicer: A One-Stop Data Processing System for Large Language Models

1 code implementation • 5 Sep 2023 • Daoyuan Chen, Yilun Huang, Zhijian Ma, Hesen Chen, Xuchen Pan, Ce Ge, Dawei Gao, Yuexiang Xie, Zhaoyang Liu, Jinyang Gao, Yaliang Li, Bolin Ding, Jingren Zhou

A data recipe is a mixture of data from different sources for training LLMs, which plays a vital role in LLMs' performance.

Distributed Computing

FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning

1 code implementation • 1 Sep 2023 • Weirui Kuang, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao, Xuchen Pan, Yuexiang Xie, Yaliang Li, Bolin Ding, Jingren Zhou

When several entities have similar interested tasks, but their data cannot be shared because of privacy concerns regulations, federated learning (FL) is a mainstream solution to leverage the data of different entities.

Benchmarking Federated Learning +1

FS-Real: Towards Real-World Cross-Device Federated Learning

no code implementations • 23 Mar 2023 • Daoyuan Chen, Dawei Gao, Yuexiang Xie, Xuchen Pan, Zitao Li, Yaliang Li, Bolin Ding, Jingren Zhou

Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data, which attracts increasing attention in both academia and industry.

Federated Learning