Search Results for author:
Found 19 papers, 9 papers with code
What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?
We find that a model's generalization behavior can be effectively characterized by a training metric we call pre-memorization train accuracy: the accuracy of model samples on training queries before they begin to copy the exact reasoning steps from the training set.
Octo: An Open-Source Generalist Robot Policy
In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces.
Accelerating Exploration with Unlabeled Prior Data
Learning to solve tasks from a sparse reward signal is a major challenge for standard reinforcement learning (RL) algorithms.
Robotic Offline RL from Internet Videos via Value-Function Pre-Training
Our system, called V-PTR, combines the benefits of pre-training on video data with robotic offline RL approaches that train on diverse robot data, resulting in value functions and policies for manipulation tasks that perform better, act robustly, and generalize broadly.
HIQL: Offline Goal-Conditioned RL with Latent States as Actions
This structure can be very useful, as assessing the quality of actions for nearby goals is typically easier than for more distant goals.
Reinforcement Learning from Passive Data via Latent Intentions
Passive observational data, such as human videos, is abundant and rich in information, yet remains largely untapped by current RL methods.
Distributionally Adaptive Meta Reinforcement Learning
In this work, we develop a framework for meta-RL algorithms that are able to behave appropriately under test-time distribution shifts in the space of tasks.
Offline RL Policies Should be Trained to be Adaptive
Offline RL algorithms must account for the fact that the dataset they are provided may leave many facets of the environment unknown.
Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability
Generalization is a central challenge for the deployment of reinforcement learning (RL) systems in the real world.
Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning
We identify an implicit under-parameterization phenomenon in value-based deep RL methods that use bootstrapping: when value functions, approximated using deep neural networks, are trained with gradient descent using iterated regression onto target values generated by previous instances of the value network, more gradient updates decrease the expressivity of the current value network.
Representations for Stable Off-Policy Reinforcement Learning
Reinforcement learning with function approximation can be unstable and even divergent, especially when combined with off-policy learning and Bellman updates.
An operator view of policy gradient methods
We cast policy gradient methods as the repeated application of two operators: a policy improvement operator $\mathcal{I}$, which maps any policy $\pi$ to a better one $\mathcal{I}\pi$, and a projection operator $\mathcal{P}$, which finds the best approximation of $\mathcal{I}\pi$ in the set of realizable policies.
On Catastrophic Interference in Atari 2600 Games
Our study provides a clear empirical link between catastrophic interference and sample efficiency in reinforcement learning.
Learning to Reach Goals via Iterated Supervised Learning
Current reinforcement learning (RL) algorithms can be brittle and difficult to use, especially when learning goal-reaching behaviors from sparse rewards.
Learning to Reach Goals Without Reinforcement Learning
By maximizing the likelihood of good actions provided by an expert demonstrator, supervised imitation learning can produce effective policies without the algorithmic complexities and optimization challenges of reinforcement learning, at the cost of requiring an expert demonstrator -- typically a person -- to provide the demonstrations.
Learning Actionable Representations with Goal Conditioned Policies
Most prior work on representation learning has focused on generative approaches, learning representations that capture all the underlying factors of variation in the observation space in a more disentangled or well-ordered manner.
Learning Actionable Representations with Goal-Conditioned Policies
Most prior work on representation learning has focused on generative approaches, learning representations that capture all underlying factors of variation in the observation space in a more disentangled or well-ordered manner.
Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definition
We propose variational inverse control with events (VICE), which generalizes inverse reinforcement learning methods to cases where full demonstrations are not needed, such as when only samples of desired goal states are available.
Divide-and-Conquer Reinforcement Learning
In this paper, we develop a novel algorithm that instead partitions the initial state space into "slices", and optimizes an ensemble of policies, each on a different slice.
