Search Results for author:
Found 55 papers, 34 papers with code
VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation
Existing approaches to unsupervised video instance segmentation typically rely on motion estimates and experience difficulties tracking small or divergent motions.
GeneCIS: A Benchmark for General Conditional Image Similarity
In this paper, we propose the GeneCIS ('genesis') benchmark, which measures models' ability to adapt to a range of similarity conditions.
ImageBind: One Embedding Space To Bind Them All
We show that all combinations of paired data are not necessary to train such a joint embedding, and only image-paired data is sufficient to bind the modalities together.
Ranked #7 on
Zero-Shot Video Retrieval
on MSR-VTT
DINOv2: Learning Robust Visual Features without Supervision
The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision.
Ranked #1 on
Image Retrieval
on AmsterTime
(using extra training data)
MOST: Multiple Object localization with Self-supervised Transformers for object discovery
In this work, we present Multiple Object localization with Self-supervised Transformers (MOST) that uses features of transformers trained using self-supervised learning to localize multiple objects in real world images.
The effectiveness of MAE pre-pretraining for billion-scale pretraining
While MAE has only been shown to scale with the size of models, we find that it scales with the size of the training dataset as well.
Ranked #1 on
Zero-Shot Transfer Image Classification
on Food-101
(using extra training data)
RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data
Semi-supervised learning aims to train a model using limited labels.
Cut and Learn for Unsupervised Object Detection and Instance Segmentation
We propose Cut-and-LEaRn (CutLER), a simple approach for training unsupervised object detection and segmentation models.
Ranked #1 on
Unsupervised Instance Segmentation
on UVO
Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities
With these capabilities in mind, we ask: Do vision-language models exhibit gender bias when performing zero-shot image classification, object detection and semantic segmentation?
A Simple Recipe for Competitive Low-compute Self supervised Vision Models
Our main insight is that existing joint-embedding based SSL methods can be repurposed for knowledge distillation from a large self-supervised teacher to a small student model.
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
This paper demonstrates an approach for learning highly semantic image representations without relying on hand-crafted data-augmentations.
Learning Video Representations from Large Language Models
We introduce LaViLa, a new approach to learning video-language representations by leveraging Large Language Models (LLMs).
Ranked #1 on
Action Recognition
on Charades-Ego
MonoNeRF: Learning Generalizable NeRFs from Monocular Videos without Camera Pose
We propose a generalizable neural radiance fields - MonoNeRF, that can be trained on large-scale monocular videos of moving in static scenes without any ground-truth annotations of depth and camera poses.
The Hidden Uniform Cluster Prior in Self-Supervised Learning
A successful paradigm in representation learning is to perform self-supervised pretraining using tasks based on mini-batch statistics (e. g., SimCLR, VICReg, SwAV, MSN).
OmniMAE: Single Model Masked Pretraining on Images and Videos
Furthermore, this model can be learned by dropping 90% of the image and 95% of the video patches, enabling extremely fast training of huge model architectures.
Masked Siamese Networks for Label-Efficient Learning
We propose Masked Siamese Networks (MSN), a self-supervised learning framework for learning image representations.
Self-Supervised Image Classification
Self-Supervised Learning
+1
A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments
We show that for a training loss to be stable under DA sampling, the model's saliency map (gradient of the loss with respect to the model's input) must align with the smallest eigenvector of the sample variance under the considered DA augmentation, hinting at a possible explanation on why models tend to shift their focus from edges to textures.
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision
Discriminative self-supervised learning allows training models on any random group of internet images, and possibly recover salient information that helps differentiate between the images.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
(using extra training data)
Omnivore: A Single Model for Many Visual Modalities
Prior work has studied different visual modalities in isolation and developed separate architectures for recognition of images, videos, and 3D data.
Ranked #1 on
Scene Recognition
on SUN-RGBD
(using extra training data)
Detecting Twenty-thousand Classes using Image-level Supervision
For the first time, we train a detector with all the twenty-one-thousand classes of the ImageNet dataset and show that it generalizes to new datasets without finetuning.
Ranked #2 on
Open Vocabulary Object Detection
on OpenImages-v4
Mask2Former for Video Instance Segmentation
We find Mask2Former also achieves state-of-the-art performance on video instance segmentation without modifying the architecture, the loss or even the training pipeline.
Ranked #10 on
Video Instance Segmentation
on YouTube-VIS validation
Masked-attention Mask Transformer for Universal Image Segmentation
While only the semantics of each task differ, current research focuses on designing specialized architectures for each task.
Ranked #2 on
Semantic Segmentation
on Mapillary val
Frame Averaging for Invariant and Equivariant Network Design
For example, Euclidean motion invariant/equivariant graph or point cloud neural networks.
An End-to-End Transformer Model for 3D Object Detection
We propose 3DETR, an end-to-end Transformer based object detection model for 3D point clouds.
Ranked #14 on
3D Object Detection
on ScanNetV2
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers
In video transformers, the time dimension is often treated in the same way as the two spatial dimensions.
Ranked #14 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
3D Spatial Recognition without Spatially Labeled 3D
We introduce WyPR, a Weakly-supervised framework for Point cloud Recognition, requiring only scene-level class tags as supervision.
Emerging Properties in Self-Supervised Vision Transformers
In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets).
Ranked #2 on
Visual Place Recognition
on Laurel Caverns
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
This paper proposes a novel method of learning by predicting view assignments with support samples (PAWS).
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
Ranked #1 on
Visual Question Answering (VQA)
on CLEVR-Humans
Generalized Referring Expression Comprehension
Phrase Grounding
+9
Robust Audio-Visual Instance Discrimination
Second, since self-supervised contrastive learning relies on random sampling of negative instances, instances that are semantically similar to the base instance can be used as faulty negatives.
Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning
First, for space, we show that spatial augmentations such as cropping do work well for videos too, but that previous implementations, due to the high processing and memory cost, could not do this at a scale sufficient for it to work well.
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
This causes the embedding vectors of distorted versions of a sample to be similar, while minimizing the redundancy between the components of these vectors.
Ranked #11 on
Image Classification
on Places205
Self-supervised Pretraining of Visual Features in the Wild
Recently, self-supervised learning methods like MoCo, SimCLR, BYOL and SwAV have reduced the gap with supervised methods.
Ranked #6 on
Image Classification
on Places205
Self-Supervised Image Classification
Self-Supervised Learning
+1
Self-Supervised Pretraining of 3D Features on any Point-Cloud
Pretraining on large labeled datasets is a prerequisite to achieve good performance in many computer vision tasks like 2D object recognition, video classification etc.
MDETR - Modulated Detection for End-to-End Multi-Modal Understanding
We also investigate the utility of our model as an object detector on a given label set when fine-tuned in a few-shot setting.
Ranked #2 on
Referring Expression Comprehension
on Talk2Car
(using extra training data)
Can Temporal Information Help with Contrastive Self-Supervised Learning?
To this end, we present Temporal-aware Contrastive self-supervised learningTaCo, as a general paradigm to enhance video CSL.
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
In addition, we also propose a new data augmentation strategy, multi-crop, that uses a mix of views with different resolutions in place of two full-resolution views, without increasing the memory or compute requirements much.
Ranked #1 on
Contrastive Learning
on imagenet-1k
Audio-Visual Instance Discrimination with Cross-Modal Agreement
Our method uses contrastive learning for cross-modal discrimination of video from audio and vice-versa.
Ranked #3 on
Self-Supervised Audio Classification
on ESC-50
In Defense of Grid Features for Visual Question Answering
Popularized as 'bottom-up' attention, bounding box (or region) based visual features have recently surpassed vanilla grid-based convolutional features as the de facto standard for vision and language tasks like visual question answering (VQA).
Ranked #18 on
Visual Question Answering (VQA)
on VQA v2 test-std
ClusterFit: Improving Generalization of Visual Representations
Pre-training convolutional neural networks with weakly-supervised and self-supervised strategies is becoming increasingly popular for several computer vision tasks.
Ranked #52 on
Image Classification
on iNaturalist 2018
Self-Supervised Learning of Pretext-Invariant Representations
The goal of self-supervised learning from images is to construct image representations that are semantically meaningful via pretext tasks that do not require semantic annotations for a large training set of images.
Ranked #7 on
Contrastive Learning
on imagenet-1k
Does Object Recognition Work for Everyone?
The paper analyzes the accuracy of publicly available object-recognition systems on a geographically diverse dataset.
3D-RelNet: Joint Object and Relational Network for 3D Prediction
We propose an approach to predict the 3D shape and pose for the objects present in a scene.
Scaling and Benchmarking Self-Supervised Visual Representation Learning
Self-supervised learning aims to learn representations from the data itself without explicit manual supervision.
Evaluating Text-to-Image Matching using Binary Image Selection (BISON)
Providing systems the ability to relate linguistic and visual content is one of the hallmarks of computer vision.
Learning by Asking Questions
We also show that our model asks questions that generalize to state-of-the-art VQA models and to novel test time distributions.
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
In this paper, we propose a simple approach to generate large annotated instance datasets with minimal effort.
From Red Wine to Red Tomato: Composition With Context
In this paper, we present a simple method that respects contextuality in order to compose classifiers of known visual concepts.
Visual Storytelling
We introduce the first dataset for sequential vision-to-language, and explore how this data may be used for the task of visual storytelling.
Cross-stitch Networks for Multi-task Learning
In this paper, we propose a principled approach to learn shared representations in ConvNets using multi-task learning.
Ranked #98 on
Semantic Segmentation
on NYU Depth v2
Shuffle and Learn: Unsupervised Learning using Temporal Order Verification
With this simple task and no semantic labels, we learn a powerful visual representation using a Convolutional Neural Network (CNN).
Ranked #48 on
Self-Supervised Action Recognition
on HMDB51
Generating Natural Questions About an Image
There has been an explosion of work in the vision & language community during the past few years from image captioning to video transcription, and answering questions about images.
Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels
When human annotators are given a choice about what to label in an image, they apply their own subjective judgments on what to ignore and what to mention.
Watch and Learn: Semi-Supervised Learning for Object Detectors From Video
We present a semi-supervised approach that localizes multiple unknown object instances in long videos.
Watch and Learn: Semi-Supervised Learning of Object Detectors from Videos
We present a semi-supervised approach that localizes multiple unknown object instances in long videos.
