Characterization of Large Language Model Development in the Datacenter

1 code implementation • 12 Mar 2024 • Qinghao Hu, Zhisheng Ye, Zerui Wang, Guoteng Wang, Meng Zhang, Qiaoling Chen, Peng Sun, Dahua Lin, Xiaolin Wang, Yingwei Luo, Yonggang Wen, Tianwei Zhang

Large Language Models (LLMs) have presented impressive performance across several transformative tasks.

Language Modelling Large Language Model +1

Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision

no code implementations • 24 May 2022 • Wei Gao, Qinghao Hu, Zhisheng Ye, Peng Sun, Xiaolin Wang, Yingwei Luo, Tianwei Zhang, Yonggang Wen

Deep learning (DL) shows its prosperity in a wide variety of fields.

Scheduling

The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning

no code implementations • 9 Nov 2021 • Raed Kontar, Naichen Shi, Xubo Yue, Seokhyun Chung, Eunshin Byon, Mosharaf Chowdhury, Judy Jin, Wissam Kontar, Neda Masoud, Maher Noueihed, Chinedum E. Okwudire, Garvesh Raskutti, Romesh Saigal, Karandeep Singh, Zhisheng Ye

The Internet of Things (IoT) is on the verge of a major paradigm shift.

Federated Learning Meta-Learning +1

A Unifying Framework for Variance Reduction Algorithms for Finding Zeroes of Monotone Operators

no code implementations • 22 Jun 2019 • Xun Zhang, William B. Haskell, Zhisheng Ye

This framework includes a number of existing deterministic and variance-reduced algorithms for function minimization as special cases, and it is also applicable to more general problems such as saddle-point problems and variational inequalities.