Search Results for author:
Found 19 papers, 1 papers with code
LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models
Developing such algorithms requires tasks that can gauge progress on algorithm design, provide accessible and reproducible evaluations for multi-turn interactions, and cover a range of task properties and challenges in improving reinforcement learning algorithms.
Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations
LLMs trained with supervised fine-tuning or "single-step" RL, as with standard RLHF, might struggle which tasks that require such goal-directed behavior, since they are not trained to optimize for overall conversational outcomes after multiple turns of interaction.
Offline RL with Observation Histories: Analyzing and Improving Sample Complexity
Theoretically, we show that standard offline RL algorithms conditioned on observation histories suffer from poor sample complexity, in accordance with the above intuition.
ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis
When writing programs, people have the ability to tackle a new complex task by decomposing it into smaller and more familiar subtasks.
Multi-Task Off-Policy Learning from Bandit Feedback
We prove per-task bounds on the suboptimality of the learned policies, which show a clear improvement over not using the hierarchical model.
On the Sensitivity of Reward Inference to Misspecified Human Models
This begs the question: how accurate do these models need to be in order for the reward inference to be accurate?
Confidence-Conditioned Value Functions for Offline Reinforcement Learning
This approach can be implemented in practice by conditioning the Q-function from existing conservative algorithms on the confidence. We theoretically show that our learned value functions produce conservative estimates of the true value at any desired confidence.
When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?
To answer this question, we characterize the properties of environments that allow offline RL methods to perform better than BC methods, even when only provided with expert data.
Compositional Generalization and Decomposition in Neural Program Synthesis
We first characterize several different axes along which program synthesis methods would be desired to generalize, e. g., length generalization, or the ability to combine known subroutines in new ways that do not occur in the training data.
Deep Hierarchy in Bandits
We use this exact posterior to analyze the Bayes regret of HierTS in Gaussian bandits.
Hierarchical Bayesian Bandits
We provide a unified view of all these problems, as learning to act in a hierarchical Bayesian bandit.
Should I Run Offline Reinforcement Learning or Behavioral Cloning?
In this paper, our goal is to characterize environments and dataset compositions where offline RL leads to better performance than BC.
Thompson Sampling with a Mixture Prior
We study Thompson sampling (TS) in online decision making, where the uncertain environment is sampled from a mixture distribution.
Non-Stationary Latent Bandits
The key idea is to frame this problem as a latent bandit, where the prototypical models of user behavior are learned offline and the latent state of the user is inferred online from its interactions with the models.
Latent Programmer: Discrete Latent Codes for Program Synthesis
The latent codes are learned using a self-supervised learning principle, in which first a discrete autoencoder is trained on the output sequences, and then the resulting latent codes are used as intermediate targets for the end-to-end sequence prediction task.
Latent Bandits Revisited
A latent bandit problem is one in which the learning agent knows the arm reward distributions conditioned on an unknown discrete latent state.
Non-Stationary Off-Policy Optimization
This approach is practical and analyzable, and we provide guarantees on both the quality of off-policy optimization and the regret during online deployment.
Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions
We focus on the problem of predicting future states of entities in complex, real-world driving scenarios.
Ensemble Maximum Entropy Classification and Linear Regression for Author Age Prediction
The evolution of the internet has created an abundance of unstructured data on the web, a significant part of which is textual.
