Search Results for author:
Found 1 papers, 0 papers with code
Rethinking Memory and Communication Cost for Efficient Large Language Model Training
In this paper, we rethink the impact of memory consumption and communication costs on the training speed of large language models, and propose a memory-communication balanced strategy set Partial Redundancy Optimizer (PaRO).
Cannot find the paper you are looking for? You can
Submit a new open access paper.
