Search Results for author:
Found 16 papers, 7 papers with code
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
Given a video demonstration of a manipulation task and current visual observations, Vid2Robot directly produces robot actions.
FlexCap: Generating Rich, Localized, and Flexible Captions in Images
The model, FlexCap, is trained to produce length-conditioned captions for input bounding boxes, and this allows control over the information density of its output, with descriptions ranging from concise object labels to detailed captions.
RT-H: Action Hierarchies Using Language
Predicting these language motions as an intermediate step between tasks and actions forces the policy to learn the shared structure of low-level motions across seemingly disparate tasks.
AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
Q-Match: Self-Supervised Learning by Matching Distributions Induced by a Queue
In semi-supervised learning, student-teacher distribution matching has been successful in improving performance of models using unlabeled data in conjunction with few labeled samples.
Visuomotor Control in Multi-Object Scenes Using Object-Aware Representations
Our self-supervised representations are learned by observing the agent freely interacting with different parts of the environment and is queried in two different settings: (i) policy learning and (ii) object location prediction.
XIRL: Cross-embodiment Inverse Reinforcement Learning
We investigate the visual cross-embodiment imitation setting, in which agents learn policies from videos of other agents (such as humans) demonstrating the same task, but with stark differences in their embodiments -- shape, actions, end-effector dynamics, etc.
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations
On semi-supervised learning benchmarks we improve performance significantly when only 1% ImageNet labels are available, from 53. 8% to 56. 5%.
Ranked #1 on
Image Classification
on PASCAL VOC 2007
Counting Out Time: Class Agnostic Video Repetition Counting in the Wild
We present an approach for estimating the period with which an action is repeated in a video.
An Analysis of Object Representations in Deep Visual Trackers
Despite these strong priors, we show that deep trackers often default to tracking by saliency detection - without relying on the object instance representation.
Inducing Stronger Object Representations in Deep Visual Trackers
Fully convolutional deep correlation networks are integral components of state-of- the-art approaches to single object visual tracking.
Temporal Cycle-Consistency Learning
We introduce a self-supervised representation learning method based on the task of temporal alignment between videos.
Ranked #1 on
Video Alignment
on UPenn Action
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
We identify two issues with the family of algorithms based on the Adversarial Imitation Learning framework.
Learning Actionable Representations from Visual Observations
In this work we explore a new approach for robots to teach themselves about the world simply by observing it.
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
In this paper, we propose a simple approach to generate large annotated instance datasets with minimal effort.
Deep Cuboid Detection: Beyond 2D Bounding Boxes
We present a Deep Cuboid Detector which takes a consumer-quality RGB image of a cluttered scene and localizes all 3D cuboids (box-like objects).
