Search Results for author:
Found 13 papers, 6 papers with code
Why Transformers Need Adam: A Hessian Perspective
SGD performs worse than Adam by a significant margin on Transformers, but the reason remains unclear.
Policy Optimization in RLHF: The Impact of Out-of-preference Data
These methods, either explicitly or implicitly, learn a reward model from preference data and differ in the data used for policy optimization to unlock the generalization ability of the reward model.
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
Based on these properties, we develop ReMax, a new algorithm tailored for RLHF.
Provably Efficient Adversarial Imitation Learning with Unknown Transitions
Adversarial imitation learning (AIL), a subset of IL methods, is particularly promising, but its theoretical foundation in the presence of unknown transitions has yet to be fully developed.
Deploying Offline Reinforcement Learning with Human Feedback
To address this issue, we propose an alternative framework that involves a human supervising the RL models and providing additional feedback in the online deployment phase.
Theoretical Analysis of Offline Imitation With Supplementary Dataset
This paper considers a situation where, besides the small amount of expert data, a supplementary dataset is available, which can be collected cheaply from sub-optimal policies.
Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis
Imitation learning learns a policy from expert trajectories.
A Note on Target Q-learning For Solving Finite MDPs with A Generative Oracle
In particular, we demonstrate that the sample complexity of the target Q-learning algorithm in [Lee and He, 2020] is $\widetilde{\mathcal O}(|\mathcal S|^2|\mathcal A|^2 (1-\gamma)^{-5}\varepsilon^{-2})$.
Rethinking ValueDice: Does It Really Improve Performance?
First, we show that ValueDice could reduce to BC under the offline setting.
HyperDQN: A Randomized Exploration Method for Deep Reinforcement Learning
However, it is limited to the case where 1) a good feature is known in advance and 2) this feature is fixed during the training: if otherwise, RLSVI suffers an unbearable computational burden to obtain the posterior samples of the parameter in the $Q$-value function.
On Generalization of Adversarial Imitation Learning and Beyond
For some MDPs, we show that vanilla AIL has a worse sample complexity than BC.
Error Bounds of Imitating Policies and Environments
In this paper, we firstly analyze the value gap between the expert policy and imitated policies by two imitation methods, behavioral cloning and generative adversarial imitation.
On Value Discrepancy of Imitation Learning
We also show that the framework leads to the value discrepancy of GAIL in an order of O((1-\gamma)^{-1}).
