Search Results for author:
Found 6 papers, 4 papers with code
From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function
Standard RLHF deploys reinforcement learning in a specific token-level MDP, while DPO is derived as a bandit problem in which the whole response of the model is treated as a single arm.
Contrastive Preference Learning: Learning from Human Feedback without RL
Thus, learning a reward function from feedback is not only based on a flawed assumption of human preference, but also leads to unwieldy optimization challenges that stem from policy gradients or bootstrapping in the RL phase.
Improving Long-Horizon Imitation Through Instruction Prediction
Complex, long-horizon planning and its combinatorial nature pose steep challenges for learning-based agents.
Distance Weighted Supervised Learning for Offline Interaction Data
To bridge the gap between IL and RL, we introduce Distance Weighted Supervised Learning or DWSL, a supervised method for learning goal-conditioned policies from offline data.
Extreme Q-Learning: MaxEnt RL without Entropy
Using EVT, we derive our \emph{Extreme Q-Learning} framework and consequently online and, for the first time, offline MaxEnt Q-learning algorithms, that do not explicitly require access to a policy or its entropy.
Few-Shot Preference Learning for Human-in-the-Loop RL
Contrary to most works that focus on query selection to \emph{minimize} the amount of data required for learning reward functions, we take an opposite approach: \emph{expanding} the pool of available data by viewing human-in-the-loop RL through the more flexible lens of multi-task learning.
