Search Results for author:
Found 31 papers, 12 papers with code
Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community
The ability to interact with machines using natural human language is becoming not just commonplace, but expected.
Verifiably Following Complex Robot Instructions with Foundation Models
We propose Language Instruction grounding for Motion Planning (LIMP), a system that leverages foundation models and temporal logics to generate instruction-conditioned semantic maps that enable robots to verifiably follow expressive and long-horizon instructions with open vocabulary referents and complex spatiotemporal constraints.
Improved Inference of Human Intent by Combining Plan Recognition and Language Feedback
We propose a novel framework for plan and goal recognition in partially observable domains -- Dialogue for Goal Recognition (D4GR) enabling a robot to rectify its belief in human progress by asking clarification questions about noisy sensor data and sub-optimal human actions.
Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents
Recent advancements in large language models (LLMs) have enabled a new research domain, LLM agents, for solving robotics and planning tasks by leveraging the world knowledge and general reasoning abilities of LLMs obtained during pretraining.
Language-Conditioned Observation Models for Visual Object Search
In this work, we bridge the gap in realistic object search by posing the search problem as a partially observable Markov decision process (POMDP) where the object detector and visual sensor noise in the observation model is determined by a single Deep Neural Network conditioned on complex language descriptions.
A System for Generalized 3D Multi-Object Search
GenMOS takes as input point cloud observations of the local region, object detection results, and localization of the robot's view pose, and outputs a 6D viewpoint to move to through online planning.
Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments
We propose Lang2LTL, a modular system and a software package that leverages large language models (LLMs) to ground temporal navigational commands to LTL specifications in environments without prior language data.
CAPE: Corrective Actions from Precondition Errors using Large Language Models
Our improvements transfer to a Boston Dynamics Spot robot initialized with a set of skills (specified in language) and associated preconditions, where CAPE improves the correctness metric of the executed task plans by 76. 49% compared to SayCan.
RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agents
We define precise syntax and grounding semantics for RLang, and provide a parser that grounds RLang programs to an algorithm-agnostic \textit{partial} world model and policy that can be exploited by an RL agent.
Where were my keys? -- Aggregating Spatial-Temporal Instances of Objects for Efficient Retrieval over Long Periods of Time
Robots equipped with situational awareness can help humans efficiently find their lost objects by leveraging spatial and temporal structure.
Towards Optimal Correlational Object Search
In realistic applications of object search, robots will need to locate target objects in complex environments while coping with unreliable sensors, especially for small or hard-to-detect objects.
Generalizing to New Domains by Mapping Natural Language to Lifted LTL
We compare our method of mapping natural language task specifications to intermediate contextual queries against state-of-the-art CopyNet models capable of translating natural language to LTL, by evaluating whether correct LTL for manipulation and navigation task specifications can be output, and show that our method outperforms the CopyNet model on unseen object references.
Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings
Learning continuous control in high-dimensional sparse reward settings, such as robotic manipulation, is a challenging problem due to the number of samples often required to obtain accurate optimal value and policy estimates.
Dialogue Object Search
To this end, we introduce a new task, dialogue object search: A robot is tasked to search for a target object (e. g. fork) in a human environment (e. g., kitchen), while engaging in a "video call" with a remote human who has additional but inexact knowledge about the target's location.
Bootstrapping Motor Skill Learning with Motion Planning
We propose using kinematic motion planning as a completely autonomous, sample efficient way to bootstrap motor skill learning for object manipulation.
Spatial Language Understanding for Object Search in Partially Observed City-scale Environments
We propose SLOOP (Spatial Language Object-Oriented POMDP), a new framework for partially observable decision making with a probabilistic observation model for spatial language.
Task Scoping: Generating Task-Specific Abstractions for Planning in Open-Scope Models
Unfortunately, planning to solve any specific task using an open-scope model is computationally intractable - even for state-of-the-art methods - due to the many states and actions that are necessarily present in the model but irrelevant to that problem.
Generating Handwriting via Decoupled Style Descriptors
Instead, we introduce the Decoupled Style Descriptor (DSD) model for handwriting, which factors both character- and writer-level styles and allows our model to represent an overall greater space of styles.
Robot Object Retrieval with Contextual Natural Language Queries
The model takes in a language command containing a verb, for example "Hand me something to cut," and RGB images of candidate objects and selects the object that best satisfies the task specified by the verb.
Multi-Resolution POMDP Planning for Multi-Object Search in 3D
Robots operating in households must find objects on shelves, under tables, and in cupboards.
pomdp_py: A Framework to Build and Solve POMDP Problems
In this paper, we present pomdp_py, a general purpose Partially Observable Markov Decision Process (POMDP) library written in Python and Cython.
Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control
We demonstrate that our model trained on 33% of the possible goals is able to generalize to more than 90% of the targets in the scene for both simulation and robot experiments.
Grounding Language Attributes to Objects using Bayesian Eigenobjects
Our system is able to disambiguate between novel objects, observed via depth images, based on natural language descriptions.
Planning with State Abstractions for Non-Markovian Task Specifications
Often times, we specify tasks for a robot using temporal language that can also span different levels of abstraction.
Generalized Grounding Graphs: A Probabilistic Framework for Understanding Grounded Commands
Symbolic models capture linguistic structure but have not scaled successfully to handle the diverse language produced by untrained users.
Implementing the Deep Q-Network
The Deep Q-Network proposed by Mnih et al. [2015] has become a benchmark and building point for much deep reinforcement learning research.
Deep Abstract Q-Networks
We examine the problem of learning and planning on high-dimensional domains with long horizons and sparse rewards.
Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning
One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks.
A Tale of Two DRAGGNs: A Hybrid Approach for Interpreting Action-Oriented and Goal-Oriented Instructions
Robots operating alongside humans in diverse, stochastic environments must be able to accurately interpret natural language commands.
Accurately and Efficiently Interpreting Human-Robot Instructions of Varying Granularities
In this work, by grounding commands to all the tasks or subtasks available in a hierarchical planning framework, we arrive at a model capable of interpreting language at multiple levels of specificity ranging from coarse to more granular.
