Search Results for author:
Found 17 papers, 2 papers with code
ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
In this paper, we develop a framework for building multi-turn RL algorithms for fine-tuning LLMs, that preserves the flexibility of existing single-turn RL methods for LLMs (e. g., proximal policy optimization), while accommodating multiple turns, long horizons, and delayed rewards effectively.
Is Offline Decision Making Possible with Only Few Samples? Reliable Decisions in Data-Starved Bandits via Trust Region Enhancement
Surprisingly, in this work, we demonstrate that even in such a data-starved setting it may still be possible to find a policy competitive with the optimal one.
When is Realizability Sufficient for Off-Policy Reinforcement Learning?
Model-free algorithms for reinforcement learning typically require a condition called Bellman completeness in order to successfully operate off-policy with function approximation, unless additional conditions are met.
Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning
Such instability can be observed even with linear function approximation.
Bellman Residual Orthogonalization for Offline Reinforcement Learning
We propose and analyze a reinforcement learning principle that approximates the Bellman equations by enforcing their validity only along an user-defined space of test functions.
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning
Actor-critic methods are widely used in offline reinforcement learning practice, but are not so well-understood theoretically.
Design of Experiments for Stochastic Contextual Linear Bandits
In the stochastic linear contextual bandit setting there exist several minimax procedures for exploration with policies that are reactive to the data being acquired.
Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation
Policy optimization methods are popular reinforcement learning algorithms, because their incremental and on-policy nature makes them more stable than the value-based counterparts.
Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL
Several practical applications of reinforcement learning involve an agent learning from past data without the possibility of further exploration.
Provably Efficient Reward-Agnostic Navigation with Linear Value Iteration
There has been growing progress on theoretical analyses for provably efficient learning in MDPs with linear function approximation, but much of the existing work has made strong assumptions to enable exploration by conventional exploration frameworks.
Learning Near Optimal Policies with Low Inherent Bellman Error
This has two important consequences: 1) it shows that exploration is possible using only \emph{batch assumptions} with an algorithm that achieves the optimal statistical rate for the setting we consider, which is more general than prior work on low-rank MDPs 2) the lack of closedness (measured by the inherent Bellman error) is only amplified by $\sqrt{d_t}$ despite working in the online setting.
Limiting Extrapolation in Linear Approximate Value Iteration
We prove that if the features at any state can be represented as a convex combination of features at the anchor points, then errors are propagated linearly over iterations (instead of exponentially) and our method achieves a polynomial sample complexity bound in the horizon and the number of anchor points.
Almost Horizon-Free Structure-Aware Best Policy Identification with a Generative Model
This paper focuses on the problem of computing an $\epsilon$-optimal policy in a discounted Markov Decision Process (MDP) provided that we can access the reward and transition function through a generative model.
Problem Dependent Reinforcement Learning Bounds Which Can Identify Bandit Structure in MDPs
In order to make good decision under uncertainty an agent must learn from observations.
Frequentist Regret Bounds for Randomized Least-Squares Value Iteration
We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning (RL).
Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds
Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict.
Robust Super-Level Set Estimation using Gaussian Processes
This paper focuses on the problem of determining as large a region as possible where a function exceeds a given threshold with high probability.
