Search Results for author:
Found 9 papers, 1 papers with code
Dataset Reset Policy Optimization for RLHF
Motivated by the fact that offline preference dataset provides informative states (i. e., data that is preferred by the labelers), our new algorithm, Dataset Reset Policy Optimization (DR-PO), integrates the existing offline preference dataset into the online policy training procedure via dataset reset: it directly resets the policy optimizer to the states in the offline dataset, instead of always starting from the initial state distribution.
Optimal Multi-Distribution Learning
Focusing on a hypothesis class of Vapnik-Chervonenkis (VC) dimension $d$, we propose a novel algorithm that yields an $varepsilon$-optimal randomized hypothesis with a sample complexity on the order of $(d+k)/\varepsilon^2$ (modulo some logarithmic factor), matching the best-known lower bound.
Provably Efficient CVaR RL in Low-rank MDPs
We study CVaR RL in low-rank MDPs with nonlinear function approximation.
Provable Reward-Agnostic Preference-Based Reinforcement Learning
Preference-based Reinforcement Learning (PbRL) is a paradigm in which an RL agent learns to optimize a task using pair-wise preference-based feedback over trajectories, rather than explicit reward signals.
Provable Offline Preference-Based Reinforcement Learning
Our proposed algorithm consists of two main steps: (1) estimate the implicit reward using Maximum Likelihood Estimation (MLE) with general function approximation from offline data and (2) solve a distributionally robust planning problem over a confidence set around the MLE.
PAC Reinforcement Learning for Predictive State Representations
We show that given a realizable model class, the sample complexity of learning the near optimal policy only scales polynomially with respect to the statistical complexity of the model class, without any explicit polynomial dependence on the size of the state and observation spaces.
Decentralized Optimistic Hyperpolicy Mirror Descent: Provably No-Regret Learning in Markov Games
We study decentralized policy learning in Markov games where we control a single agent to play with nonstationary and possibly adversarial opponents.
Offline Reinforcement Learning with Realizability and Single-policy Concentrability
Sample-efficiency guarantees for offline reinforcement learning (RL) often rely on strong assumptions on both the function classes (e. g., Bellman-completeness) and the data coverage (e. g., all-policy concentrability).
Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence
These can often be accounted for via regularized RL, which augments the target value function with a structure-promoting regularizer.
