Search Results for author: Yun Dai

Found 5 papers, 1 papers with code

Liger Kernel: Efficient Triton Kernels for LLM Training

1 code implementation14 Oct 2024 Pin-Lun Hsu, Yun Dai, Vignesh Kothapalli, Qingquan Song, Shao Tang, Siyu Zhu, Steven Shimizu, Shivam Sahni, Haowen Ning, Yanning Chen

Training Large Language Models (LLMs) efficiently at scale presents a formidable challenge, driven by their ever-increasing computational demands and the need for enhanced performance.

Chunking

Enhancing Stability for Large Language Models Training in Constrained Bandwidth Networks

no code implementations28 Jun 2024 Yun Dai, Tejas Dharamsi, Byron Hsu, Tao Song, Hamed Firooz

Training extremely large language models (LLMs) with billions of parameters is a computationally intensive task that pushes the limits of current data parallel training systems.

Cannot find the paper you are looking for? You can Submit a new open access paper.