Search Results for author:
Found 53 papers, 18 papers with code
TactoFind: A Tactile Only System for Object Retrieval
We study the problem of object retrieval in scenarios where visual sensing is absent, object shapes are unknown beforehand and objects can move freely, like grabbing objects out of a drawer.
Statistical Learning under Heterogenous Distribution Shift
We show that, when the class $\mathcal{F}$ is "simpler" than $\mathcal{G}$ (measured, e. g., in terms of its metric entropy), our predictor is more resilient to \emph{heterogenous covariate shifts} in which the shift in $\mathbf{x}$ is much greater than that in $\mathbf{y}$.
Aligning Robot and Human Representations
To act in the world, robots rely on a representation of salient task aspects: for example, to carry a cup of coffee, a robot must consider movement efficiency and cup orientation in its behaviour.
Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior
Learned locomotion policies can rapidly adapt to diverse environments similar to those experienced during training but lack a mechanism for fast tuning when they fail in an out-of-distribution test environment.
Is Conditional Generative Modeling all you need for Decision-Making?
We further demonstrate the advantages of modeling policies as conditional diffusion models by considering two other conditioning variables: constraints and skills.
Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learning
To address this, we propose a reward learning approach, Graph-based Equivalence Mappings (GEM), that can discover spatial goal representations that are aligned with the intended goal specification, enabling successful generalization in unseen environments.
Visual Dexterity: In-hand Dexterous Manipulation from Depth
In-hand object reorientation is necessary for performing many dexterous manipulation tasks, such as tool use in unstructured environments that remain beyond the reach of current robots.
SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields
This formalism is implemented in three steps: assigning a consistent local coordinate frame to the task-relevant object parts, determining the location and orientation of this coordinate frame on unseen object instances, and executing an action that brings these frames into the desired alignment.
Redeeming Intrinsic Rewards via Constrained Optimization
However, on easy exploration tasks, the agent gets distracted by intrinsic rewards and performs unnecessary exploration even when sufficient task (also called extrinsic) reward is available.
Distributionally Adaptive Meta Reinforcement Learning
In this work, we develop a framework for meta-RL algorithms that are able to behave appropriately under test-time distribution shifts in the space of tasks.
Stable Object Reorientation using Contact Plane Registration
We present a system for accurately predicting stable orientations for diverse rigid objects.
Offline RL Policies Should be Trained to be Adaptive
Offline RL algorithms must account for the fact that the dataset they are provided may leave many facets of the environment unknown.
Visual Pre-training for Navigation: What Can We Learn from Noise?
We hypothesize a sufficient representation of the current view and the goal view for a navigation policy can be learned by predicting the location and size of a crop of the current view that corresponds to the goal.
Overcoming the Spectral Bias of Neural Value Approximation
Value approximation using deep neural networks is at the heart of off-policy deep reinforcement learning, and is often the primary module that provides learning signals to the rest of the algorithm.
Rapid Locomotion via Reinforcement Learning
Agile maneuvers such as sprinting and high-speed turning in the wild are challenging for legged robots.
Bilinear value networks
The dominant framework for off-policy multi-goal reinforcement learning involves estimating goal conditioned Q-value function.
Topological Experience Replay
State-of-the-art deep Q-learning methods update Q-values using state transition tuples sampled from the experience replay buffer.
Stubborn: A Strong Baseline for Indoor Object Navigation
We present a strong baseline that surpasses the performance of previously published methods on the Habitat Challenge task of navigating to a target object in indoor environments.
Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Our performance generalizes across both object instances and 6-DoF object poses, and significantly outperforms a recent baseline that relies on 2D descriptors.
A System for General In-Hand Object Re-Orientation
The videos of the learned policies are available at: https://taochenshh. github. io/projects/in-hand-reorientation.
Equivariant Contrastive Learning
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
Understanding the Generalization Gap in Visual Reinforcement Learning
Deep Reinforcement Learning (RL) agents have achieved superhuman performance on several video game suites.
Bi-linear Value Networks for Multi-goal Reinforcement Learning
We simultaneously learn both components.
Equivariant Self-Supervised Learning: Encouraging Equivariance in Representations
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
An End-to-End Differentiable Framework for Contact-Aware Robot Design
Existing methods for co-optimization are limited and fail to explore a rich space of designs.
3D Neural Scene Representations for Visuomotor Control
Humans have a strong intuitive understanding of the 3D environment around us.
Learning Task Informed Abstractions
Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features.
Model-based Reinforcement Learning
reinforcement-learning
+1
Residual Model Learning for Microrobot Control
RML provides a general framework for learning from extremely small amounts of interaction data, and our experiments with HAMR clearly demonstrate that RML substantially outperforms existing techniques.
The Low-Rank Simplicity Bias in Deep Networks
We show empirically that our claim holds true on finite width linear and non-linear models on practical learning paradigms and show that on natural data, these are often the solutions that generalize well.
Learning to Recover from Failures using Memory
Learning from past mistakes is a quintessential aspect of intelligence.
A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects
We present a framework for solving long-horizon planning problems involving manipulation of rigid objects that operates directly from a point-cloud observation, i. e. without prior object models.
OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning
Reinforcement learning (RL) has achieved impressive performance in a variety of online settings in which an agent's ability to query the environment for transitions and rewards is effectively unlimited.
AdaScale SGD: A User-Friendly Algorithm for Distributed Training
When using large-batch training to speed up stochastic gradient descent, learning rates must adapt to new batch sizes in order to maximize speed-ups and preserve model quality.
Exploring Exploration: Comparing Children with RL Agents in Unified Environments
Research in developmental psychology consistently shows that children explore the world thoroughly and efficiently and that this exploration allows them to learn.
Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning
Learning robotic manipulation tasks using reinforcement learning with sparse rewards is currently impractical due to the outrageous data requirements.
AdaScale SGD: A Scale-Invariant Algorithm for Distributed Training
When using distributed training to speed up stochastic gradient descent, learning rates must adapt to new scales in order to maintain training effectiveness.
Classification in the dark using tactile exploration
Combining information from different sensory modalities to execute goal directed actions is a key aspect of human intelligence.
Superposition of many models into one
We present a method for storing multiple models within a single set of parameters.
Learning Instance Segmentation by Interaction
The agent uses its current segmentation model to infer pixels that constitute objects and refines the segmentation model by interacting with these pixels.
Zero-Shot Visual Imitation
In our framework, the role of the expert is only to communicate the goals (i. e., what to imitate) during inference.
Investigating Human Priors for Playing Video Games
What makes humans so good at solving seemingly complex video games?
What Will Happen Next? Forecasting Player Moves in Sports Videos
A large number of very popular team sports involve the act of one team trying to score a goal against the other.
A Computer Vision Pipeline for Automated Determination of Cardiac Structure and Function and Detection of Disease by Two-Dimensional Echocardiography
Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways including enabling low-cost serial assessment of cardiac function in the primary care and rural setting.
Curiosity-driven Exploration by Self-supervised Prediction
In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether.
Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation
Manipulation of deformable objects, such as ropes and cloth, is an important but challenging problem in robotics.
Learning to Perform Physics Experiments via Deep Reinforcement Learning
When encountering novel objects, humans are able to infer a wide range of physical properties such as mass, friction and deformability by interacting with them in a goal driven way.
What makes ImageNet good for transfer learning?
Which is better: more classes or more examples per class?
Learning to Poke by Poking: Experiential Learning of Intuitive Physics
We investigate an experiential learning paradigm for acquiring an internal model of intuitive physics.
Learning Visual Predictive Models of Physics for Playing Billiards
The ability to plan and execute goal specific actions in varied, unexpected settings is a central requirement of intelligent agents.
Human Pose Estimation with Iterative Error Feedback
Hierarchical feature extractors such as Convolutional Networks (ConvNets) have achieved impressive performance on a variety of classification tasks using purely feedforward processing.
Ranked #42 on
Pose Estimation
on MPII Human Pose
Learning to See by Moving
We show that given the same number of training images, features learnt using egomotion as supervision compare favourably to features learnt using class-label as supervision on visual tasks of scene recognition, object recognition, visual odometry and keypoint matching.
Pixels to Voxels: Modeling Visual Representation in the Human Brain
We find that both classes of models accurately predict brain activity in high-level visual areas, directly from pixels and without the need for any semantic tags or hand annotation of images.
Analyzing the Performance of Multilayer Neural Networks for Object Recognition
In the last two years, convolutional neural networks (CNNs) have achieved an impressive suite of results on standard recognition datasets and tasks.
