Search Results for author:
Found 11 papers, 4 papers with code
Rethinking Information Structures in RLHF: Reward Generalization from a Graph Theory Perspective
Then, based on this framework, we introduce the IBN to analyze generalization in the reward modeling stage of RLHF.
Aligner: Achieving Efficient Alignment through Weak-to-Strong Correction
Here we introduce Aligner, a new efficient alignment paradigm that bypasses the whole RLHF process by learning the correctional residuals between the aligned and the unaligned answers.
AI Alignment: A Comprehensive Survey
The former aims to make AI systems aligned via alignment training, while the latter aims to gain evidence about the systems' alignment and govern them appropriately to avoid exacerbating misalignment risks.
Safe RLHF: Safe Reinforcement Learning from Human Feedback
However, the inherent tension between the objectives of helpfulness and harmlessness presents a significant challenge during LLM training.
Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark
By introducing this benchmark, we aim to facilitate the evaluation and comparison of safety performance, thus fostering the development of reinforcement learning for safer, more reliable, and responsible real-world applications.
Red Teaming Game: A Game-Theoretic Framework for Red Teaming Language Models
In this paper, we present Red-teaming Game (RTG), a general game-theoretic framework without manual annotation.
Baichuan 2: Open Large-scale Language Models
Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing the need for extensive feature engineering.
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs).
OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research
AI systems empowered by reinforcement learning (RL) algorithms harbor the immense potential to catalyze societal advancement, yet their deployment is often impeded by significant safety concerns.
Proactive Multi-Camera Collaboration For 3D Human Pose Estimation
This paper presents a multi-agent reinforcement learning (MARL) scheme for proactive Multi-Camera Collaboration in 3D Human Pose Estimation in dynamic human crowds.
TorchOpt: An Efficient Library for Differentiable Optimization
TorchOpt further provides a high-performance distributed execution runtime.
