Search Results for author:
Found 22 papers, 6 papers with code
Technical Report: Enhancing LLM Reasoning with Reward-guided Tree Search
It is primarily constructed around a tree search algorithm, where the policy model navigates a dynamically expanding tree guided by a specially trained reward model.
Boosting Deductive Reasoning with Step Signals In RLHF
Logical reasoning is a crucial task for Large Language Models (LLMs), enabling them to tackle complex problems.
Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
Conventionally, preference data is learned and encoded into a scalar reward model that connects a value head with an LLM to produce a scalar score as preference or reward.
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown
Reward models (RM) play a critical role in aligning generations of large language models (LLM) to human expectations.
Reward-Robust RLHF in LLMs
As Large Language Models (LLMs) continue to progress toward more advanced forms of intelligence, Reinforcement Learning from Human Feedback (RLHF) is increasingly seen as a key pathway toward achieving Artificial General Intelligence (AGI).
3D-Properties: Identifying Challenges in DPO and Charting a Path Forward
Aligning large language models (LLMs) with human preference has recently gained tremendous attention, with the canonical yet costly RLHF-PPO and the simple and straightforward Direct Preference Optimization (DPO) as two examples.
Exploring the LLM Journey from Cognition to Expression with Linear Representations
This paper presents an in-depth examination of the evolution and interplay of cognitive and expressive capabilities in large language models (LLMs), with a specific focus on Baichuan-7B and Baichuan-33B, an advanced bilingual (Chinese and English) LLM series.
SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling
Human preference alignment is critical in building powerful and reliable large language models (LLMs).
Reward Generalization in RLHF: A Topological Perspective
As a solution, we introduce a theoretical framework for investigating reward generalization in reinforcement learning from human feedback (RLHF), focusing on the topology of information flow at both macro and micro levels.
Baichuan 2: Open Large-scale Language Models
Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing the need for extensive feature engineering.
Task Aware Dreamer for Task Generalization in Reinforcement Learning
Extensive experiments in both image-based and state-based tasks show that TAD can significantly improve the performance of handling different tasks simultaneously, especially for those with high TDR, and display a strong generalization ability to unseen tasks.
Model-based Reinforcement Learning with a Hamiltonian Canonical ODE Network
Model-based reinforcement learning usually suffers from a high sample complexity in training the world model, especially for the environments with complex dynamics.
Model-based Reinforcement Learning
reinforcement-learning
+2
On the Reuse Bias in Off-Policy Reinforcement Learning
In this paper, we reveal that the instability is also related to a new notion of Reuse Bias of IS -- the bias in off-policy evaluation caused by the reuse of the replay buffer for evaluation and optimization.
Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Risk
Though deep reinforcement learning (DRL) has obtained substantial success, it may encounter catastrophic failures due to the intrinsic uncertainty of both transition and observation.
Policy Learning for Robust Markov Decision Process with a Mismatched Generative Model
Our goal is to identify a near-optimal robust policy for the perturbed testing environment, which introduces additional technical difficulties as we need to simultaneously estimate the training environment uncertainty from samples and find the worst-case perturbation for testing.
Tianshou: a Highly Modularized Deep Reinforcement Learning Library
In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend.
Towards Safe Reinforcement Learning via Constraining Conditional Value at Risk
Though deep reinforcement learning (DRL) has obtained substantial success, it may encounter catastrophic failures due to the intrinsic uncertainty caused by stochastic policies and environment variability.
Adaptive N-step Bootstrapping with Off-policy Data
In this work, we point out that the optimal value of n actually differs on each data point, while the fixed value n is a rough average of them.
Satellite-Terrestrial Channel Characterization in High-Speed Railway Environment at 22.6 GHz
In this paper, the satellite-terrestrial channel at 22. 6 GHz is characterized for a typical high-speed railway (HSR) environment.
Lazy-CFR: fast and near-optimal regret minimization for extensive games with imperfect information
In this paper, we present Lazy-CFR, a CFR algorithm that adopts a lazy update strategy to avoid traversing the whole game tree in each round.
Reward Shaping via Meta-Learning
Reward shaping is one of the most effective methods to tackle the crucial yet challenging problem of credit assignment in Reinforcement Learning (RL).
Lazy-CFR: fast and near optimal regret minimization for extensive games with imperfect information
In this paper, we present a novel technique, lazy update, which can avoid traversing the whole game tree in CFR, as well as a novel analysis on the regret of CFR with lazy update.
