Search Results for author:
Found 32 papers, 19 papers with code
ImageBind: One Embedding Space To Bind Them All
We show that all combinations of paired data are not necessary to train such a joint embedding, and only image-paired data is sufficient to bind the modalities together.
Ranked #6 on
Zero-Shot Video Retrieval
on MSR-VTT
The effectiveness of MAE pre-pretraining for billion-scale pretraining
While MAE has only been shown to scale with the size of models, we find that it scales with the size of the training dataset as well.
Ranked #1 on
Zero-Shot Transfer Image Classification
on Food-101
(using extra training data)
Learning to Substitute Ingredients in Recipes
Recipe personalization through ingredient substitution has the potential to help people meet their dietary needs and preferences, avoid potential allergens, and ease culinary exploration in everyone's kitchen.
Cut and Learn for Unsupervised Object Detection and Instance Segmentation
We propose Cut-and-LEaRn (CutLER), a simple approach for training unsupervised object detection and segmentation models.
Ranked #1 on
Unsupervised Instance Segmentation
on UVO
What You Say Is What You Show: Visual Narration Detection in Instructional Videos
Narrated "how-to" videos have emerged as a promising data source for a wide range of learning problems, from learning visual representations to training robot policies.
HierVL: Learning Hierarchical Video-Language Embeddings
Video-language embeddings are a promising avenue for injecting semantics into visual representations, but existing methods capture only short-term associations between seconds-long video clips and their accompanying text.
Ranked #1 on
Action Recognition
on Charades-Ego
Learning Video Representations from Large Language Models
We introduce LaViLa, a new approach to learning video-language representations by leveraging Large Language Models (LLMs).
OmniMAE: Single Model Masked Pretraining on Images and Videos
Furthermore, this model can be learned by dropping 90% of the image and 95% of the video patches, enabling extremely fast training of huge model architectures.
Omnivore: A Single Model for Many Visual Modalities
Prior work has studied different visual modalities in isolation and developed separate architectures for recognition of images, videos, and 3D data.
Ranked #1 on
Scene Recognition
on SUN-RGBD
(using extra training data)
Detecting Twenty-thousand Classes using Image-level Supervision
For the first time, we train a detector with all the twenty-one-thousand classes of the ImageNet dataset and show that it generalizes to new datasets without finetuning.
Ranked #2 on
Open Vocabulary Object Detection
on OpenImages-v4
Mask2Former for Video Instance Segmentation
We find Mask2Former also achieves state-of-the-art performance on video instance segmentation without modifying the architecture, the loss or even the training pipeline.
Masked-attention Mask Transformer for Universal Image Segmentation
While only the semantics of each task differ, current research focuses on designing specialized architectures for each task.
Ranked #2 on
Semantic Segmentation
on Mapillary val
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
An End-to-End Transformer Model for 3D Object Detection
We propose 3DETR, an end-to-end Transformer based object detection model for 3D point clouds.
Ranked #13 on
3D Object Detection
on ScanNetV2
Anticipative Video Transformer
We propose Anticipative Video Transformer (AVT), an end-to-end attention-based video modeling architecture that attends to the previously observed video in order to anticipate future actions.
Ranked #1 on
Action Anticipation
on EPIC-KITCHENS-100 (test)
(using extra training data)
3D Spatial Recognition without Spatially Labeled 3D
We introduce WyPR, a Weakly-supervised framework for Point cloud Recognition, requiring only scene-level class tags as supervision.
Physical Reasoning Using Dynamics-Aware Models
A common approach to solving physical reasoning tasks is to train a value learner on example tasks.
Ranked #1 on
Visual Reasoning
on PHYRE-1B-Within
Self-Supervised Pretraining of 3D Features on any Point-Cloud
Pretraining on large labeled datasets is a prerequisite to achieve good performance in many computer vision tasks like 2D object recognition, video classification etc.
Forward Prediction for Physical Reasoning
Physical reasoning requires forward prediction: the ability to forecast what will happen next given some initial world state.
Ranked #2 on
Visual Reasoning
on PHYRE-1B-Within
Video Understanding as Machine Translation
With the advent of large-scale multimodal video datasets, especially sequences with audio or transcribed speech, there has been a growing interest in self-supervised learning of video representations.
CATER: A diagnostic dataset for Compositional Actions & TEmporal Reasoning
In this work, we build a video dataset with fully observable and controllable object and scene bias, and which truly requires spatiotemporal understanding in order to be solved.
Are we asking the right questions in MovieQA?
Joint vision and language tasks like visual question answering are fascinating because they explore high-level understanding, but at the same time, can be more prone to language biases.
MetaPix: Few-Shot Video Retargeting
We address the task of unsupervised retargeting of human actions from one video to another.
CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning
In this work, we build a video dataset with fully observable and controllable object and scene bias, and which truly requires spatiotemporal understanding in order to be solved.
DistInit: Learning Video Representations Without a Single Labeled Video
In this work, we propose an alternative approach to learning video representations that require no semantically labeled videos and instead leverages the years of effort in collecting and labeling large and clean still-image datasets.
Ranked #70 on
Action Recognition
on HMDB-51
(using extra training data)
Video Action Transformer Network
We introduce the Action Transformer model for recognizing and localizing human actions in video clips.
Ranked #5 on
Action Recognition
on AVA v2.1
A Better Baseline for AVA
We introduce a simple baseline for action localization on the AVA dataset.
Ranked #11 on
Action Recognition
on AVA v2.1
Binge Watching: Scaling Affordance Learning from Sitcoms
In this paper, we tackle the challenge of creating one of the biggest dataset for learning affordances.
Detect-and-Track: Efficient Pose Estimation in Videos
This paper addresses the problem of estimating and tracking human body keypoints in complex, multi-person video.
Ranked #7 on
Keypoint Detection
on COCO test-challenge
Attentional Pooling for Action Recognition
We introduce a simple yet surprisingly powerful model to incorporate attention in action recognition and human object interaction tasks.
Ranked #7 on
Human-Object Interaction Detection
on HICO
ActionVLAD: Learning spatio-temporal aggregation for action classification
In this work, we introduce a new video representation for action classification that aggregates local convolutional features across the entire spatio-temporal extent of the video.
Learning a Predictable and Generative Vector Representation for Objects
The network consists of two components: (a) an autoencoder that ensures the representation is generative; and (b) a convolutional network that ensures the representation is predictable.
