Search Results for author:
Found 30 papers, 9 papers with code
OpenEQA: Embodied Question Answering in the Era of Foundation Models
We present a modern formulation of Embodied Question Answering (EQA) as the task of understanding an environment well enough to answer questions about it in natural language.
What do we learn from a large-scale study of pre-trained visual representations in sim and real environments?
We present a large empirical investigation on the use of pre-trained visual representations (PVRs) for training downstream policies that execute real-world tasks.
EgoAdapt: A multi-stream evaluation study of adaptation to real-world egocentric user video
We introduce an evaluation framework that directly exploits the user's data stream with new metrics to measure the adaptation gain over the population model, online generalization, and hindsight performance.
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?
Contrary to inferences from prior work, we find that scaling dataset size and diversity does not improve performance universally (but does so on average).
BC-IRL: Learning Generalizable Reward Functions from Demonstrations
How well do reward functions learned with inverse reinforcement learning (IRL) generalize?
Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
Object rearrangement is a challenge for embodied agents because solving these tasks requires generalizing across a combinatorially large set of configurations of entities and their locations.
Cross-Domain Transfer via Semantic Skill Imitation
We propose an approach for semantic imitation, which uses demonstrations from a source domain, e. g. human videos, to accelerate reinforcement learning (RL) in a different target domain, e. g. a robotic manipulator in a simulated kitchen.
Model Based Meta Learning of Critics for Policy Gradients
In this paper we present a framework to meta-learn the critic for gradient-based policy learning.
Differentiable and Learnable Robot Models
Building differentiable simulations of physical processes has recently received an increasing amount of attention.
Block Contextual MDPs for Continual Learning
In this work, we propose to examine this continual reinforcement learning setting through the block contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity.
Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learning
Inverse reinforcement learning is a paradigm motivated by the goal of learning general reward functions from demonstrated behaviours.
Habitat 2.0: Training Home Assistants to Rearrange their Habitat
We introduce Habitat 2. 0 (H2. 0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios.
Quasi-Equivalence Discovery for Zero-Shot Emergent Communication
In contrast, in this work, we present a novel problem setting and the Quasi-Equivalence Discovery (QED) algorithm that allows for zero-shot coordination (ZSC), i. e., discovering protocols that can generalize to independently trained agents.
Multi-Modal Learning of Keypoint Predictive Models for Visual Object Manipulation
Our evaluation shows that our approach learns to consistently predict visual keypoints on objects in the manipulator's hand, and thus can easily facilitate learning an extended kinematic chain to include the object grasped in various configurations, from a few seconds of visual data.
Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations
Indeed, emergent communication is now a vibrant field of research, with common settings involving discrete cheap-talk channels.
Model-Based Inverse Reinforcement Learning from Visual Demonstrations
Scaling model-based inverse reinforcement learning (IRL) to real robotic manipulation tasks with unknown dynamics remains an open problem.
Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation
Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC) but can struggle in the context of insertion.
Adversarial Continual Learning
We show that shared features are significantly less prone to forgetting and propose a novel hybrid continual learning framework that learns a disjoint representation for task-invariant and task-specific features required to solve a sequence of tasks.
Learning State-Dependent Losses for Inverse Dynamics Learning
In both settings, the structured and state-dependent learned losses improve online adaptation speed, when compared to standard, state-independent loss functions.
Encoding Physical Constraints in Differentiable Newton-Euler Algorithm
The recursive Newton-Euler Algorithm (RNEA) is a popular technique for computing the dynamics of robots.
Robotics
Generalized Inner Loop Meta-Learning
Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem.
Meta Learning via Learned Loss
We present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures.
Meta-Learning via Learned Loss
This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time.
Curious iLQR: Resolving Uncertainty in Model-based RL
In this work, we propose a model-based reinforcement learning (MBRL) framework that combines Bayesian modeling of the system dynamics with curious iLQR, an iterative LQR approach that considers model uncertainty.
Model-based Reinforcement Learning
reinforcement-learning
+2
A Hierarchical Bayesian Linear Regression Model with Local Features for Stochastic Dynamics Approximation
One of the challenges in model-based control of stochastic dynamical systems is that the state transition dynamics are involved, and it is not easy or efficient to make good-quality predictions of the states.
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control
In this work, we present an approach to deep visuomotor control using structured deep dynamics models.
Online Learning of a Memory for Learning Rates
The promise of learning to learn for robotics rests on the hope that by extracting some information about the learning process itself we can speed up subsequent similar learning tasks.
Robust Gaussian Filtering using a Pseudo Measurement
The contribution of this paper is to show that any Gaussian filter can be made compatible with fat-tailed sensor models by applying one simple change: Instead of filtering with the physical measurement, we propose to filter with a pseudo measurement obtained by applying a feature function to the physical measurement.
Incremental Local Gaussian Regression
Locally weighted regression (LWR) was created as a nonparametric method that can approximate a wide range of functions, is computationally efficient, and can learn continually from very large amounts of incrementally collected data.
Local Gaussian Regression
Locally weighted regression was created as a nonparametric learning method that is computationally efficient, can learn from very large amounts of data and add data incrementally.
