Search Results for author:
Found 6 papers, 4 papers with code
How Far Are We from Optimal Reasoning Efficiency?
To reduce the efficiency gap, we propose REO-RL, a class of Reinforcement Learning algorithms that minimizes REG by targeting a sparse set of token budgets.
AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
Most existing large-scale RL systems for LLMs are synchronous, alternating generation and training in a batch setting where rollouts in each training batch are generated by the same model.
On Designing Effective RL Reward at Training Time for LLM Reasoning
In this work, we evaluate popular reward models for RL training, including the Outcome-supervised Reward Model (ORM) and the Process-supervised Reward Model (PRM), and train a collection of LLMs for math problems using RL by combining these learned rewards with success rewards.
ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocation
To overcome this limitation, we propose a novel technique named parameter ReaLlocation, which dynamically adapts the parallelization strategies for different workloads during training by redistributing LLM parameters across the training cluster.
Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study
However, in academic benchmarks, state-of-the-art results are often achieved via reward-free methods, such as Direct Preference Optimization (DPO).
SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores
Following this abstraction, we develop a scalable, efficient, and extensible distributed RL system called ReaLlyScalableRL, which allows efficient and massively parallelized training and easy development of customized algorithms.
