Search Results for author:
Found 49 papers, 18 papers with code
Latent Weight Diffusion: Generating Policies from Trajectories
We employ a diffusion denoising model within this latent space to learn its distribution.
Zero-Shot Generalization of Vision-Based RL Without Data Augmentation
Specifically, we revisit the role of latent disentanglement in RL and show how combining it with a model of associative memory achieves zero-shot generalization on difficult task variations without relying on data augmentation.
SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systems
By ensuring that training more closely replicates expert behavior in safety-critical states, our approach results in safer policies despite policy errors during the test time.
Can Large Language Models Solve Robot Routing?
We systematically investigate the performance of LLMs in robot routing by constructing a dataset with 80 unique robot routing problems across 8 variants in both single and multi-robot settings.
Guaranteed Trust Region Optimization via Two-Phase KL Penalization
Then, we show that introducing a "fixup" phase is sufficient to guarantee a trust region is enforced on every policy update while adding fewer than 5% additional gradient steps in practice.
Conditionally Combining Robot Skills using Large Language Models
This paper combines two contributions.
HyperPPO: A scalable method for finding small policies for robotic control
We demonstrate that the neural policies estimated by HyperPPO are capable of decentralized control of a Crazyflie2. 1 quadrotor.
Collision Avoidance and Navigation for a Quadrotor Swarm Using End-to-end Deep Reinforcement Learning
In this work, we propose an end-to-end DRL approach to control quadrotor swarms in environments with obstacles.
LEMMA: Learning Language-Conditioned Multi-Robot Manipulation
We introduce a benchmark for LanguagE-Conditioned Multi-robot MAnipulation (LEMMA) focused on task allocation and long-horizon object manipulation based on human language instructions in a tabletop setting.
QuadSwarm: A Modular Multi-Quadrotor Simulator for Deep Reinforcement Learning with Direct Thrust Control
In addition to speed, such simulators need to model the physics of the robots and their interaction with the environment to a level acceptable for transferring policies learned in simulation to reality.
RREx-BoT: Remote Referring Expressions with a Bag of Tricks
Armed with this intuition, using only a generic vision-language scoring model with minor modifications for 3d encoding and operating in an embodied environment, we demonstrate an absolute performance gain of 9. 84% on remote object grounding above state of the art models for REVERIE and of 5. 04% on FAO.
OpenD: A Benchmark for Language-Driven Door and Drawer Opening
We introduce OPEND, a benchmark for learning how to use a hand to open cabinet doors or drawers in a photo-realistic and physics-reliable simulation environment driven by language instruction.
CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigation
Our results on the coarse-grained instruction following task of REVERIE demonstrate the navigational capability of CLIP, surpassing the supervised baseline in terms of both success rate (SR) and success weighted by path length (SPL).
Efficiently Learning Small Policies for Locomotion and Manipulation
Neural control of memory-constrained, agile robots requires small, yet highly performant models.
CH-MARL: A Multimodal Benchmark for Cooperative, Heterogeneous Multi-Agent Reinforcement Learning
We propose a multimodal (vision-and-language) benchmark for cooperative and heterogeneous multi-agent learning.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Learning Deformable Object Manipulation from Expert Demonstrations
We test DMfD on a set of representative manipulation tasks for a 1-dimensional rope and a 2-dimensional cloth from the SoftGym suite of tasks, each with state and image observations.
Deformable Object Manipulation
General Reinforcement Learning
+1
A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search
Physically rearranging objects is an important capability for embodied agents.
Inferring Articulated Rigid Body Dynamics from RGBD Video
Being able to reproduce physical phenomena ranging from light interaction to contact mechanics, simulators are becoming increasingly useful in more and more application domains where real-world interaction or labeled data are difficult to obtain.
DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following
Language-guided Embodied AI benchmarks requiring an agent to navigate an environment and manipulate objects typically allow one-way communication: the human user gives a natural language command to the agent, and the agent can only follow the command passively.
Privacy Preserving Visual Question Answering
We introduce a novel privacy-preserving methodology for performing Visual Question Answering on the edge.
Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation
Learning complex manipulation tasks in realistic, obstructed environments is a challenging problem due to hard exploration in the presence of obstacles and high-dimensional visual observations.
LUMINOUS: Indoor Scene Generation for Embodied AI Challenges
However, current simulators for Embodied AI (EAI) challenges only provide simulated indoor scenes with a limited number of layouts.
A Simple Approach to Continual Learning by Transferring Skill Parameters
We take a fresh look at this problem, by considering a setting in which the robot is limited to storing that knowledge and experience only in the form of learned skill policies.
Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems
In this survey article, we analyze how resilience is achieved in networks of agents and multi-robot systems that are able to overcome adversity by leveraging system-wide complementarity, diversity, and redundancy - often involving a reconfiguration of robotic capabilities to provide some key ability that was not present in the system a priori.
Adaptive Sampling using POMDPs with Domain-Specific Considerations
We find that by using knowledge of the number of rollouts allocated, the agent can more effectively choose actions to explore.
Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulation
We address the latter problem of estimating parameters through a Bayesian inference approach that approximates a posterior distribution over simulation parameters given real sensor measurements.
NeuralSim: Augmenting Differentiable Simulators with Neural Networks
Differentiable simulators provide an avenue for closing the sim-to-real gap by enabling the use of efficient, gradient-based optimization algorithms to find the simulation parameters that best fit the observed sensor readings.
Robotics
Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments
In contrast, motion planners use explicit models of the agent and environment to plan collision-free paths to faraway goals, but suffer from inaccurate models in tasks that require contacts with the environment.
Learning Equality Constraints for Motion Planning on Manifolds
Constrained robot motion planning is a widely used technique to solve complex robot tasks.
Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap
We present a differentiable simulation architecture for articulated rigid-body dynamics that enables the augmentation of analytical models with neural networks at any point of the computation.
Confidence-rich grid mapping
Representing the environment is a fundamental task in enabling robots to act autonomously in unknown environments.
Learning Manifolds for Sequential Motion Planning
Motion planning with constraints is an important part of many real-world robotic systems.
Robotics Computational Geometry
Efficient Adaptation for End-to-End Vision-Based Robotic Manipulation
One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments, but most robot learning systems today are deployed as fixed policies which do not adapt after deployment.
Sampling-Based Motion Planning on Sequenced Manifolds
We address the problem of planning robot motions in constrained configuration spaces where the constraints change throughout the motion.
Robotics Computational Geometry
Plan-Space State Embeddings for Improved Reinforcement Learning
We show how these embedding spaces can then be used as an augmentation to the robot state in reinforcement learning problems.
Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
One of the great promises of robot learning systems is that they will be able to learn from their mistakes and continuously adapt to ever-changing environments.
On Localizing a Camera from a Single Image
Public cameras often have limited metadata describing their attributes.
Experimental Comparison of Global Motion Planning Algorithms for Wheeled Mobile Robots
Planning smooth and energy-efficient motions for wheeled mobile robots is a central task for applications ranging from autonomous driving to service and intralogistic robotics.
Encoding Physical Constraints in Differentiable Newton-Euler Algorithm
The recursive Newton-Euler Algorithm (RNEA) is a popular technique for computing the dynamics of robots.
Robotics
Automatic Differentiation and Continuous Sensitivity Analysis of Rigid Body Dynamics
A key ingredient to achieving intelligent behavior is physical understanding that equips robots with the ability to reason about the effects of their actions in a dynamic environment.
Physics-based Simulation of Continuous-Wave LIDAR for Localization, Calibration and Tracking
Light Detection and Ranging (LIDAR) sensors play an important role in the perception stack of autonomous robots, supplying mapping and localization pipelines with depth measurements of the environment.
Resilient Coverage: Exploring the Local-to-Global Trade-off
We propose a centralized control framework to select suitable robots from a heterogeneous pool and place them at appropriate locations to monitor a region for events of interest.
Interactive Differentiable Simulation
While learning-based models of the environment dynamics have contributed to significant improvements in sample efficiency compared to model-free reinforcement learning algorithms, they typically fail to generalize to system states beyond the training data, while often grounding their predictions on non-interpretable latent variables.
Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotors
Quadrotor stabilizing controllers often require careful, model-specific tuning for safe operation.
Robotics
Auto-conditioned Recurrent Mixture Density Networks for Learning Generalizable Robot Skills
Personal robots assisting humans must perform complex manipulation tasks that are typically difficult to specify in traditional motion planning pipelines, where multiple objectives must be met and the high-level context be taken into consideration.
Soft Value Iteration Networks for Planetary Rover Path Planning
In order to deal with complex terrain observations and policy learning, we propose a value iteration recurrence, referred to as the soft value iteration network (SVIN).
A Spatio-Temporal Representation for the Orienteering Problem with Time-Varying Profits
Different from most existing works where the profits are assumed to be static, in this work we investigate a variant that has arbitrary time-dependent profits.
Informative Planning and Online Learning with Sparse Gaussian Processes
A big challenge in environmental monitoring is the spatiotemporal variation of the phenomena to be observed.
A Solution to Time-Varying Markov Decision Processes
We consider a decision-making problem where the environment varies both in space and time.
