Search Results for author:
Found 36 papers, 13 papers with code
Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts
We empirically show that our sparse Mobile Vision MoEs (V-MoEs) can achieve a better trade-off between performance and efficiency than the corresponding dense ViTs.
Principles and Guidelines for Evaluating Social Robot Navigation Algorithms
A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation.
Value function estimation using conditional diffusion models for control
A fairly reliable trend in deep reinforcement learning is that the performance scales with the number of parameters, provided a complimentary scaling in amount of training data.
On Robustness in Multimodal Learning
Multimodal learning is defined as learning over multiple heterogeneous input modalities such as video, audio, and text.
STAIR: Learning Sparse Text and Image Representation in Grounded Tokens
We extend the CLIP model and build a sparse text and image representation (STAIR), where the image and text are mapped to a sparse token space.
Perceptual Grouping in Contrastive Vision-Language Models
In this work we examine how well vision-language models are able to understand where objects reside within an image and group together visually related parts of the imagery.
Ranked #1 on
Unsupervised Semantic Segmentation
on PASCAL VOC 2012 val
(using extra training data)
Retrospectives on the Embodied AI Workshop
We present a retrospective on the state of Embodied AI research.
Gesture2Path: Imitation Learning for Gesture-aware Navigation
Our experiments indicate that our method is able to successfully interpret complex human gestures and to use them as a signal to generate socially compliant trajectories for navigation tasks.
GAUDI: A Neural Architect for Immersive 3D Scene Generation
We introduce GAUDI, a generative model capable of capturing the distribution of complex and realistic 3D scenes that can be rendered immersively from a moving camera.
Ranked #1 on
Image Generation
on ARKitScenes
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment.
Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans.
Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning
Hierarchical reinforcement learning aims to enable this by providing a bank of low-level skills as action abstractions.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation
To validate our method, we apply ReLMoGen to two types of tasks: 1) Interactive Navigation tasks, navigation problems where interactions with the environment are required to reach the destination, and 2) Mobile Manipulation tasks, manipulation tasks that require moving the robot base.
Adversarial Generative Grammars for Human Activity Prediction
In this paper we propose an adversarial generative grammar model for future prediction.
ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects
In particular, the agent is initialized at a random location and pose in an environment and asked to find an instance of an object category, e. g., find a chair, by navigating to it.
Modeling Long-horizon Tasks as Sequential Interaction Landscapes
We show that complex plans can be carried out when executing the robotic task and the robot can interactively adapt to changes in the environment and recover from failure cases.
Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environments
We present Interactive Gibson Benchmark, the first comprehensive benchmark for training and evaluating Interactive Navigation: robot navigation strategies where physical interaction with objects is allowed and even encouraged to accomplish a task.
Long Range Neural Navigation Policies for the Real World
Learned Neural Network based policies have shown promising results for robot navigation.
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments.
Evolving Space-Time Neural Architectures for Videos
We present a new method for finding video CNN architectures that capture rich spatio-temporal information in videos.
Ranked #20 on
Action Classification
on Moments in Time
Self-supervisory Signals for Object Discovery and Detection
In robotic applications, we often face the challenge of discovering new objects while having very little or no labelled training data.
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control
In robotics, this ability is referred to as visual servoing: moving a tool or end-point to a desired location using primarily visual feedback.
Visual Representations for Semantic Target Driven Navigation
We propose to using high level semantic and contextual features including segmentation and detection masks obtained by off-the-shelf state-of-the-art vision as observations and use deep network to learn the navigation policy.
Sim2Real View Invariant Visual Servoing by Recurrent Control
To this end, we train a deep recurrent controller that can automatically determine which actions move the end-point of a robotic arm to a desired object.
No Fuss Distance Metric Learning using Proxies
Traditionally, for this problem supervision is expressed in the form of sets of points that follow an ordinal relationship -- an anchor point $x$ is similar to a set of positive points $Y$, and dissimilar to a set of negative points $Z$, and a loss defined over these distances is minimized.
Towards Accurate Multi-person Pose Estimation in the Wild
Trained on COCO data alone, our final system achieves average precision of 0. 649 on the COCO test-dev set and the 0. 643 test-standard sets, outperforming the winner of the 2016 COCO keypoints challenge and other recent state-of-art.
Ranked #6 on
Keypoint Detection
on COCO test-challenge
Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing.
Chained Predictions Using Convolutional Neural Networks
In this model the output variables for a given input are predicted sequentially using neural networks.
The Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
Current approaches for fine-grained recognition do the following: First, recruit experts to annotate a dataset of images, optionally also collecting more structured data in the form of part annotations and bounding boxes.
Ranked #4 on
Fine-Grained Image Classification
on CUB-200-2011
(using extra training data)
Generation and Comprehension of Unambiguous Object Descriptions
We propose a method that can generate an unambiguous description (known as a referring expression) of a specific object or region in an image, and which can also comprehend or interpret such an expression to infer which object is being described.
Pose Embeddings: A Deep Architecture for Learning to Match Human Poses
We present a method for learning an embedding that places images of humans in similar poses nearby.
Show and Tell: A Neural Image Caption Generator
Experiments on several datasets show the accuracy of the model and the fluency of the language it learns solely from image descriptions.
Ranked #3 on
Image Retrieval with Multi-Modal Query
on MIT-States
Deep Convolutional Ranking for Multilabel Image Annotation
Multilabel image annotation is one of the most important challenges in computer vision with many real-world applications.
DeepPose: Human Pose Estimation via Deep Neural Networks
We propose a method for human pose estimation based on Deep Neural Networks (DNNs).
Scalable Object Detection using Deep Neural Networks
Deep convolutional neural networks have recently achieved state-of-the-art performance on a number of image recognition benchmarks, including the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC-2012).
Deep Neural Networks for Object Detection
Deep Neural Networks (DNNs) have recently shown outstanding performance on the task of whole image classification.
