Search Results for author:
Found 27 papers, 10 papers with code
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment.
Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans.
Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning
Hierarchical reinforcement learning aims to enable this by providing a bank of low-level skills as action abstractions.
ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation
To validate our method, we apply ReLMoGen to two types of tasks: 1) Interactive Navigation tasks, navigation problems where interactions with the environment are required to reach the destination, and 2) Mobile Manipulation tasks, manipulation tasks that require moving the robot base.
Adversarial Generative Grammars for Human Activity Prediction
In this paper we propose an adversarial generative grammar model for future prediction.
ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
In particular, the agent is initialized at a random location and pose in an environment and asked to find an instance of an object category, e. g., find a chair, by navigating to it.
Modeling Long-horizon Tasks as Sequential Interaction Landscapes
We show that complex plans can be carried out when executing the robotic task and the robot can interactively adapt to changes in the environment and recover from failure cases.
Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environments
We present Interactive Gibson Benchmark, the first comprehensive benchmark for training and evaluating Interactive Navigation: robot navigation strategies where physical interaction with objects is allowed and even encouraged to accomplish a task.
Long Range Neural Navigation Policies for the Real World
Learned Neural Network based policies have shown promising results for robot navigation.
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments.
Evolving Space-Time Neural Architectures for Videos
We present a new method for finding video CNN architectures that capture rich spatio-temporal information in videos.
Ranked #18 on
Action Classification
on Moments in Time
Self-supervisory Signals for Object Discovery and Detection
In robotic applications, we often face the challenge of discovering new objects while having very little or no labelled training data.
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control
In robotics, this ability is referred to as visual servoing: moving a tool or end-point to a desired location using primarily visual feedback.
Visual Representations for Semantic Target Driven Navigation
We propose to using high level semantic and contextual features including segmentation and detection masks obtained by off-the-shelf state-of-the-art vision as observations and use deep network to learn the navigation policy.
Sim2Real View Invariant Visual Servoing by Recurrent Control
To this end, we train a deep recurrent controller that can automatically determine which actions move the end-point of a robotic arm to a desired object.
No Fuss Distance Metric Learning using Proxies
Traditionally, for this problem supervision is expressed in the form of sets of points that follow an ordinal relationship -- an anchor point $x$ is similar to a set of positive points $Y$, and dissimilar to a set of negative points $Z$, and a loss defined over these distances is minimized.
Towards Accurate Multi-person Pose Estimation in the Wild
Trained on COCO data alone, our final system achieves average precision of 0. 649 on the COCO test-dev set and the 0. 643 test-standard sets, outperforming the winner of the 2016 COCO keypoints challenge and other recent state-of-art.
Ranked #6 on
Keypoint Detection
on COCO test-challenge
Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing.
Chained Predictions Using Convolutional Neural Networks
In this model the output variables for a given input are predicted sequentially using neural networks.
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
Current approaches for fine-grained recognition do the following: First, recruit experts to annotate a dataset of images, optionally also collecting more structured data in the form of part annotations and bounding boxes.
Generation and Comprehension of Unambiguous Object Descriptions
We propose a method that can generate an unambiguous description (known as a referring expression) of a specific object or region in an image, and which can also comprehend or interpret such an expression to infer which object is being described.
Pose Embeddings: A Deep Architecture for Learning to Match Human Poses
We present a method for learning an embedding that places images of humans in similar poses nearby.
Show and Tell: A Neural Image Caption Generator
Experiments on several datasets show the accuracy of the model and the fluency of the language it learns solely from image descriptions.
Ranked #3 on
Image Retrieval with Multi-Modal Query
on MIT-States
DeepPose: Human Pose Estimation via Deep Neural Networks
We propose a method for human pose estimation based on Deep Neural Networks (DNNs).
Deep Convolutional Ranking for Multilabel Image Annotation
Multilabel image annotation is one of the most important challenges in computer vision with many real-world applications.
Scalable Object Detection using Deep Neural Networks
Deep convolutional neural networks have recently achieved state-of-the-art performance on a number of image recognition benchmarks, including the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC-2012).
Deep Neural Networks for Object Detection
Deep Neural Networks (DNNs) have recently shown outstanding performance on the task of whole image classification.
