Search Results for author:
Found 18 papers, 8 papers with code
OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videos
The model is trained and evaluated on the OVR dataset, and its performance assessed with and without using text to specify the target class to count.
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
Vid2Robot uses cross-attention transformer layers between video features and the current robot state to produce the actions and perform the same task as shown in the video.
FlexCap: Generating Rich, Localized, and Flexible Captions in Images
The model, FlexCap, is trained to produce length-conditioned captions for input bounding boxes, and this allows control over the information density of its output, with descriptions ranging from concise object labels to detailed captions.
RT-H: Action Hierarchies Using Language
Predicting these language motions as an intermediate step between tasks and actions forces the policy to learn the shared structure of low-level motions across seemingly disparate tasks.
ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation
Diverse demonstration datasets have powered significant advances in robot learning, but the dexterity and scale of such data can be limited by the hardware cost, the hardware robustness, and the ease of teleoperation.
AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
Q-Match: Self-Supervised Learning by Matching Distributions Induced by a Queue
In semi-supervised learning, student-teacher distribution matching has been successful in improving performance of models using unlabeled data in conjunction with few labeled samples.
Visuomotor Control in Multi-Object Scenes Using Object-Aware Representations
Our self-supervised representations are learned by observing the agent freely interacting with different parts of the environment and is queried in two different settings: (i) policy learning and (ii) object location prediction.
XIRL: Cross-embodiment Inverse Reinforcement Learning
We investigate the visual cross-embodiment imitation setting, in which agents learn policies from videos of other agents (such as humans) demonstrating the same task, but with stark differences in their embodiments -- shape, actions, end-effector dynamics, etc.
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations
On semi-supervised learning benchmarks we improve performance significantly when only 1% ImageNet labels are available, from 53. 8% to 56. 5%.
Ranked #1 on
Image Classification
on PASCAL VOC 2007
Counting Out Time: Class Agnostic Video Repetition Counting in the Wild
We present an approach for estimating the period with which an action is repeated in a video.
Ranked #1 on
Repetitive Action Counting
on Countix
An Analysis of Object Representations in Deep Visual Trackers
Despite these strong priors, we show that deep trackers often default to tracking by saliency detection - without relying on the object instance representation.
Inducing Stronger Object Representations in Deep Visual Trackers
Fully convolutional deep correlation networks are integral components of state-of- the-art approaches to single object visual tracking.
Temporal Cycle-Consistency Learning
We introduce a self-supervised representation learning method based on the task of temporal alignment between videos.
Ranked #1 on
Video Alignment
on UPenn Action
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
We identify two issues with the family of algorithms based on the Adversarial Imitation Learning framework.
Learning Actionable Representations from Visual Observations
In this work we explore a new approach for robots to teach themselves about the world simply by observing it.
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
In this paper, we propose a simple approach to generate large annotated instance datasets with minimal effort.
Deep Cuboid Detection: Beyond 2D Bounding Boxes
We present a Deep Cuboid Detector which takes a consumer-quality RGB image of a cluttered scene and localizes all 3D cuboids (box-like objects).
