Data Mixing Made Efficient: A Bivariate Scaling Law for Language Model Pretraining

no code implementations • 23 May 2024 • Ce Ge, Zhijian Ma, Daoyuan Chen, Yaliang Li, Bolin Ding

Large language models exhibit exceptional generalization capabilities, primarily attributed to the utilization of diversely sourced data.

UniDM: A Unified Framework for Data Manipulation with Large Language Models

no code implementations • 10 May 2024 • Yichen Qian, Yongyi He, Rong Zhu, Jintao Huang, Zhijian Ma, Haibin Wang, Yaohua Wang, Xiuyu Sun, Defu Lian, Bolin Ding, Jingren Zhou

In this paper, inspired by the cross-task generality of LLMs on NLP tasks, we pave the first step to design an automatic and general solution to tackle with data manipulation tasks.

AgentScope: A Flexible yet Robust Multi-Agent Platform

1 code implementation • 21 Feb 2024 • Dawei Gao, Zitao Li, Xuchen Pan, Weirui Kuang, Zhijian Ma, Bingchen Qian, Fei Wei, WenHao Zhang, Yuexiang Xie, Daoyuan Chen, Liuyi Yao, Hongyi Peng, Zeyu Zhang, Lin Zhu, Chen Cheng, Hongzhu Shi, Yaliang Li, Bolin Ding, Jingren Zhou

With the rapid advancement of Large Language Models (LLMs), significant progress has been made in multi-agent applications.

Multi-agent Integration

Data-Juicer: A One-Stop Data Processing System for Large Language Models

1 code implementation • 5 Sep 2023 • Daoyuan Chen, Yilun Huang, Zhijian Ma, Hesen Chen, Xuchen Pan, Ce Ge, Dawei Gao, Yuexiang Xie, Zhaoyang Liu, Jinyang Gao, Yaliang Li, Bolin Ding, Jingren Zhou

A data recipe is a mixture of data from different sources for training LLMs, which plays a vital role in LLMs' performance.

Distributed Computing