Search Results for author:
Found 4 papers, 3 papers with code
On Designing Effective RL Reward at Training Time for LLM Reasoning
In this work, we evaluate popular reward models for RL training, including the Outcome-supervised Reward Model (ORM) and the Process-supervised Reward Model (PRM), and train a collection of LLMs for math problems using RL by combining these learned rewards with success rewards.
ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocation
To overcome this limitation, we propose a novel technique named parameter ReaLlocation, which dynamically adapts the parallelization strategies for different workloads during training by redistributing LLM parameters across the training cluster.
Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study
However, in academic benchmarks, state-of-the-art results are often achieved via reward-free methods, such as Direct Preference Optimization (DPO).
SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores
Following this abstraction, we develop a scalable, efficient, and extensible distributed RL system called ReaLlyScalableRL, which allows efficient and massively parallelized training and easy development of customized algorithms.
