Search Results for author:
Found 74 papers, 21 papers with code
Accelerating Residual Reinforcement Learning with Uncertainty Estimation
While Residual RL is more sample-efficient than finetuning the entire base policy, existing methods struggle with sparse rewards and are designed for deterministic base policies.
Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning
Developing agents capable of exploring, planning and learning in complex open-ended environments is a grand challenge in artificial intelligence (AI).
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Optimal Interactive Learning on the Job via Facility Location Planning
Collaborative robots must continually adapt to novel tasks and user preferences without overburdening the user.
Knowledge Retention for Continual Model-Based Reinforcement Learning
We propose DRAGO, a novel approach for continual model-based reinforcement learning aimed at improving the incremental development of world models across a sequence of tasks that differ in their reward functions but not the state space or dynamics.
V-HOP: Visuo-Haptic 6D Object Pose Tracking
Building on this representation, we introduce a new visuo-haptic transformer-based object pose tracker that seamlessly integrates visual and haptic input.
Automating Curriculum Learning for Reinforcement Learning using a Skill-Based Bayesian Network
A major challenge for reinforcement learning is automatically generating curricula to reduce training time or improve performance in some target task.
Visual Theory of Mind Enables the Invention of Proto-Writing
Symbolic writing systems are graphical semiotic codes that are ubiquitous in modern society but are otherwise absent in the animal kingdom.
Skill Generalization with Verbs
Our method allows a robot to generate a trajectory for a novel object based on a verb, which can then be used as input to a motion planner.
Latent-Predictive Empowerment: Measuring Empowerment without a Simulator
Recent empowerment methods learn diverse skillsets by maximizing the mutual information between skills and states; however, these approaches require a model of the transition dynamics, which can be challenging to learn in realistic settings with high-dimensional and stochastic observations.
EPO: Hierarchical LLM Agents with Environment Preference Optimization
Long-horizon decision-making tasks present significant challenges for LLM-based agents due to the need for extensive planning over multiple steps.
Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy
Our metric, the $\lambda$-discrepancy, is the difference between two distinct temporal difference (TD) value estimates, each computed using TD($\lambda$) with a different value of $\lambda$.
Learning Abstract World Model for Value-preserving Planning with Options
General-purpose agents require fine-grained controls and rich sensory inputs to perform a wide range of tasks.
Model-based Reinforcement Learning for Parameterized Action Spaces
We propose a novel model-based reinforcement learning algorithm -- Dynamics Learning and predictive control with Parameterized Actions (DLPA) -- for Parameterized Action Markov Decision Processes (PAMDPs).
Language-guided Skill Learning with Temporal Variational Inference
We present an algorithm for skill discovery from expert demonstrations.
Verifiably Following Complex Robot Instructions with Foundation Models
When instructing robots, users want to flexibly express constraints, refer to arbitrary landmarks, and verify robot behavior, while robots must disambiguate instructions into specifications and ground instruction referents in the real world.
MinePlanner: A Benchmark for Long-Horizon Planning in Large Minecraft Worlds
We propose a new benchmark for planning tasks based on the Minecraft game.
Improved Inference of Human Intent by Combining Plan Recognition and Language Feedback
We propose a novel framework for plan and goal recognition in partially observable domains -- Dialogue for Goal Recognition (D4GR) enabling a robot to rectify its belief in human progress by asking clarification questions about noisy sensor data and sub-optimal human actions.
Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learning
Our framework makes two specific contributions.
Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learning
We propose a new method for count-based exploration in high-dimensional state spaces.
Exploiting Contextual Structure to Generate Useful Auxiliary Tasks
We propose an approach that maximizes experience reuse while learning to solve a given task by generating and simultaneously learning useful auxiliary tasks.
A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems
Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed.
Effects of Data Geometry in Early Deep Learning
Deep neural networks can approximate functions on different types of data, from images to graphs, with varied underlying structure.
On the Geometry of Reinforcement Learning in Continuous State and Action Spaces
Central to our work is the idea that the transition dynamics induce a low dimensional manifold of reachable states embedded in the high-dimensional nominal state space.
Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex
AlphaZero, an approach to reinforcement learning that couples neural networks and Monte Carlo tree search (MCTS), has produced state-of-the-art strategies for traditional board games like chess, Go, shogi, and Hex.
Model-based Lifelong Reinforcement Learning with Bayesian Exploration
We propose a model-based lifelong reinforcement-learning approach that estimates a hierarchical Bayesian posterior distilling the common structure shared across different tasks.
Constrained Dynamic Movement Primitives for Safe Learning of Motor Skills
In this paper, we present constrained dynamic movement primitives (CDMP) which can allow for constraint satisfaction in the robot workspace.
RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents
We define precise syntax and grounding semantics for RLang, and provide a parser that grounds RLang programs to an algorithm-agnostic \textit{partial} world model and policy that can be exploited by an RL agent.
Meta-Learning Parameterized Skills
We propose a novel parameterized skill-learning algorithm that aims to learn transferable parameterized skills and synthesize them into a new action space that supports efficient learning in long-horizon tasks.
Characterizing the Action-Generalization Gap in Deep Q-Learning
We study the action generalization ability of deep Q-learning in discrete action spaces.
Learning Abstract and Transferable Representations for Planning
We propose a framework for autonomously learning state abstractions of an agent's environment, given a set of skills.
Adaptive Online Value Function Approximation with Wavelets
We further demonstrate that a fixed wavelet basis set performs comparably against the high-performing Fourier basis on Mountain Car and Acrobot, and that the adaptive methods provide a convenient approach to addressing an oversized initial basis set, while demonstrating performance comparable to, or greater than, the fixed wavelet basis.
Hierarchical Reinforcement Learning of Locomotion Policies in Response to Approaching Objects: A Preliminary Study
Animals such as rabbits and birds can instantly generate locomotion behavior in reaction to a dynamic, approaching object, such as a person or a rock, despite having possibly never seen the object before and having limited perception of the object's properties.
Deep Reinforcement Learning
Hierarchical Reinforcement Learning
+4
Coarse-Grained Smoothness for RL in Metric Spaces
We propose a new coarse-grained smoothness definition that generalizes the notion of Lipschitz continuity, is more widely applicable, and allows us to compute significantly tighter bounds on Q-functions, leading to improved learning.
Guided Policy Search for Parameterized Skills using Adverbs
We present a method for using adverb phrases to adjust skill parameters via learned adverb-skill groundings.
Towards Optimal Correlational Object Search
In realistic applications of object search, robots will need to locate target objects in complex environments while coping with unreliable sensors, especially for small or hard-to-detect objects.
Learning to Infer Kinematic Hierarchies for Novel Object Instances
Manipulating an articulated object requires perceiving itskinematic hierarchy: its parts, how each can move, and howthose motions are coupled.
Generalizing to New Domains by Mapping Natural Language to Lifted LTL
We compare our method of mapping natural language task specifications to intermediate contextual queries against state-of-the-art CopyNet models capable of translating natural language to LTL, by evaluating whether correct LTL for manipulation and navigation task specifications can be output, and show that our method outperforms the CopyNet model on unseen object references.
Bayesian Exploration for Lifelong Reinforcement Learning
A central question in reinforcement learning (RL) is how to leverage prior knowledge to accelerate learning in new tasks.
HAC Explore: Accelerating Exploration with Hierarchical Reinforcement Learning
Sparse rewards and long time horizons remain challenging for reinforcement learning algorithms.
Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings
Learning continuous control in high-dimensional sparse reward settings, such as robotic manipulation, is a challenging problem due to the number of samples often required to obtain accurate optimal value and policy estimates.
Learning Markov State Abstractions for Deep Reinforcement Learning
A fundamental assumption of reinforcement learning in Markov decision processes (MDPs) is that the relevant decision process is, in fact, Markov.
Bootstrapping Motor Skill Learning with Motion Planning
We propose using kinematic motion planning as a completely autonomous, sample efficient way to bootstrap motor skill learning for object manipulation.
Task Scoping: Generating Task-Specific Abstractions for Planning in Open-Scope Models
Unfortunately, planning to solve any specific task using an open-scope model is computationally intractable - even for state-of-the-art methods - due to the many states and actions that are necessarily present in the model but irrelevant to that problem.
Skill Discovery for Exploration and Planning using Deep Skill Graphs
Temporal abstraction provides an opportunity to drastically lower the decision making burden facing reinforcement learning agents in rich sensorimotor spaces.
On the Relationship Between Structure in Natural Language and Models of Sequential Decision Processes
Human language is distinguished by powerful semantics, rich structure, and incredible flexibility.
Visual Transfer for Reinforcement Learning via Wasserstein Domain Confusion
We introduce Wasserstein Adversarial Proximal Policy Optimization (WAPPO), a novel algorithm for visual transfer in Reinforcement Learning that explicitly learns to align the distributions of extracted features between a source and target task.
Multi-Resolution POMDP Planning for Multi-Object Search in 3D
Robots operating in households must find objects on shelves, under tables, and in cupboards.
Option Discovery using Deep Skill Chaining
Autonomously discovering temporally extended actions, or skills, is a longstanding goal of hierarchical reinforcement learning.
Exploration in Reinforcement Learning with Deep Covering Options
While many option discovery methods have been proposed to accelerate exploration in reinforcement learning, they are often heuristic.
Efficient Black-Box Planning Using Macro-Actions with Focused Effects
Focused macros dramatically improve black-box planning efficiency across a wide range of planning domains, sometimes beating even state-of-the-art planners with access to a full domain model.
Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control
We demonstrate that our model trained on 33% of the possible goals is able to generalize to more than 90% of the targets in the scene for both simulation and robot experiments.
Zero-Shot Policy Transfer with Disentangled Attention
In such settings, agents are trained in similar environments, such as simulators, and are then transferred to the original environment.
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
A key challenge in intelligent robotics is creating robots that are capable of directly interacting with the world around them to achieve their goals.
Robotics
Grounding Language Attributes to Objects using Bayesian Eigenobjects
Our system is able to disambiguate between novel objects, observed via depth images, based on natural language descriptions.
Learning Portable Representations for High-Level Planning
We present a framework for autonomously learning a portable representation that describes a collection of low-level continuous environments.
Probabilistic Category-Level Pose Estimation via Segmentation and Predicted-Shape Priors
We introduce a new method for category-level pose estimation which produces a distribution over predicted poses by integrating 3D shape estimates from a generative object model with segmentation information.
Discovering Options for Exploration by Minimizing Cover Time
One of the main challenges in reinforcement learning is solving tasks with sparse reward.
Finding Options that Minimize Planning Time
We formalize the problem of selecting the optimal set of options for planning as that of computing the smallest set of options so that planning converges in less than a given maximum of value-iteration passes.
Policy and Value Transfer in Lifelong Reinforcement Learning
We consider the problem of how best to use prior experience to bootstrap lifelong learning, where an agent faces a series of task instances drawn from some task distribution.
Hybrid Bayesian Eigenobjects: Combining Linear Subspace and Deep Network Methods for 3D Robot Vision
We introduce Hybrid Bayesian Eigenobjects (HBEOs), a novel representation for 3D objects designed to allow a robot to jointly estimate the pose, class, and full 3D geometry of a novel object observed from a single viewpoint in a single practical framework.
Learning Multi-Level Hierarchies with Hindsight
Hierarchical agents have the potential to solve sequential decision making tasks with greater sample efficiency than their non-hierarchical counterparts because hierarchical agents can break down tasks into sets of subtasks that only require short sequences of decisions.
Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes
We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings.
Active Exploration for Learning Symbolic Representations
We introduce an online active exploration algorithm for data-efficiently learning an abstract symbolic model of an environment.
Mean Actor Critic
We propose a new algorithm, Mean Actor-Critic (MAC), for discrete-action continuous-state reinforcement learning.
Ranked #1 on
Continuous Control
on Cart Pole (OpenAI Gym)
Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processes
We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings.
Transfer Learning Across Patient Variations with Hidden Parameter Markov Decision Processes
Due to physiological variation, patients diagnosed with the same condition may exhibit divergent, but related, responses to the same treatments.
Policy Evaluation Using the Ω-Return
The benefit of the Ω-return is that it accounts for the correlation of different length returns.
Constructing Abstraction Hierarchies Using a Skill-Symbol Loop
We describe a framework for building abstraction hierarchies whereby an agent alternates skill- and representation-acquisition phases to construct a sequence of increasingly abstract Markov decision processes.
Reinforcement Learning with Parameterized Actions
We introduce a model-free algorithm for learning in Markov decision processes with parameterized actions-discrete actions with continuous parameters.
Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations
Control applications often feature tasks with similar, but not identical, dynamics.
TD_gamma: Re-evaluating Complex Backups in Temporal Difference Learning
We show that the lambda-return target used in the TD(lambda) family of algorithms is the maximum likelihood estimator for a specific model of how the variance of an n-step return estimate increases with n. We introduce the gamma-return estimator, an alternative target based on a more accurate model of variance, which defines the TD_gamma family of complex-backup temporal difference learning algorithms.
Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories
We demonstrate that CST constructs an appropriate skill tree that can be further refined through learning in a challenging continuous domain, and that it can be used to segment demonstration trajectories on a mobile manipulator into chains of skills where each skill is assigned an appropriate abstraction.
Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining
We introduce skill chaining, a skill discovery method for reinforcement learning agents in continuous domains, that builds chains of skills leading to an end-of-task reward.
