Search Results for author:
Found 31 papers, 22 papers with code
Causal Diffusion Transformers for Generative Modeling
We introduce Causal Diffusion as the autoregressive (AR) counterpart of Diffusion models.
Classification Done Right for Vision-Language Pre-Training
Due to the absence of the text encoding as contrastive target, SuperClass does not require a text encoder and does not need to maintain a large batch size as CLIP does.
LLaVA-Critic: Learning to Evaluate Multimodal Models
We introduce LLaVA-Critic, the first open-source large multimodal model (LMM) designed as a generalist evaluator to assess performance across a wide range of multimodal tasks.
Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles
Modern hierarchical vision transformers have added several vision-specific components in the pursuit of supervised classification performance.
Ranked #1 on
Image Classification
on iNaturalist 2019
(using extra training data)
Diffusion Models as Masked Autoencoders
There has been a longstanding belief that generation can facilitate a true understanding of visual data.
The effectiveness of MAE pre-pretraining for billion-scale pretraining
While MAE has only been shown to scale with the size of models, we find that it scales with the size of the training dataset as well.
Ranked #1 on
Few-Shot Image Classification
on ImageNet - 10-shot
(using extra training data)
Reversible Vision Transformers
Reversible Vision Transformers achieve a reduced memory footprint of up to 15. 5x at roughly identical model complexity, parameters and accuracy, demonstrating the promise of reversible vision transformers as an efficient backbone for hardware resource limited training regimes.
MAViL: Masked Audio-Video Learners
We present Masked Audio-Video Learners (MAViL) to train audio-visual representations.
Scaling Language-Image Pre-training via Masking
We present Fast Language-Image Pre-training (FLIP), a simple and more efficient method for training CLIP.
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference
Vision Transformers (ViTs) have shown impressive performance but still require a high computation cost as compared to convolutional neural networks (CNNs), one reason is that ViTs' attention measures global similarities and thus has a quadratic complexity with the number of input tokens.
Masked Autoencoders As Spatiotemporal Learners
We randomly mask out spacetime patches in videos and learn an autoencoder to reconstruct them in pixels.
On the Importance of Asymmetry for Siamese Representation Learning
Many recent self-supervised frameworks for visual representation learning are based on certain forms of Siamese networks.
Unified Transformer Tracker for Object Tracking
Although UniTrack \cite{wang2021different} demonstrates that a shared appearance model with multiple heads can be used to tackle individual tracking tasks, it fails to exploit the large-scale tracking datasets for training and performs poorly on single object tracking.
MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition
Instead of trying to process more frames at once like most existing methods, we propose to process videos in an online fashion and cache "memory" at each iteration.
Ranked #6 on
Action Anticipation
on EPIC-KITCHENS-100
(using extra training data)
Masked Feature Prediction for Self-Supervised Visual Pre-Training
We present Masked Feature Prediction (MaskFeat) for self-supervised pre-training of video models.
Ranked #8 on
Action Recognition
on AVA v2.2
(using extra training data)
MViTv2: Improved Multiscale Vision Transformers for Classification and Detection
In this paper, we study Multiscale Vision Transformers (MViTv2) as a unified architecture for image and video classification, as well as object detection.
Ranked #1 on
Action Classification
on Kinetics-600
(GFLOPs metric)
PyTorchVideo: A Deep Learning Library for Video Understanding
We introduce PyTorchVideo, an open-source deep-learning library that provides a rich set of modular, efficient, and reproducible components for a variety of video understanding tasks, including classification, detection, self-supervised learning, and low-level processing.
A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning
We present a large-scale study on unsupervised spatiotemporal representation learning from videos.
Ranked #3 on
Self-Supervised Action Recognition
on HMDB51
Representation Learning
Self-Supervised Action Recognition
+1
Multiscale Vision Transformers
We evaluate this fundamental architectural prior for modeling the dense nature of visual signals for a variety of video recognition tasks where it outperforms concurrent vision transformers that rely on large scale external pre-training and are 5-10x more costly in computation and parameters.
Ranked #14 on
Action Classification
on Charades
Beyond Short Clips: End-to-End Video-Level Learning with Collaborative Memories
The standard way of training video models entails sampling at each iteration a single clip from a video and optimizing the clip prediction with respect to the video-level label.
Multiview Pseudo-Labeling for Semi-supervised Learning from Video
We present a multiview pseudo-labeling approach to video learning, a novel framework that uses complementary views in the form of appearance and motion information for semi-supervised learning in video.
HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval
Video-Text Retrieval has been a hot research topic with the growth of multimedia data on the internet.
Can Temporal Information Help with Contrastive Self-Supervised Learning?
To this end, we present Temporal-aware Contrastive self-supervised learningTaCo, as a general paradigm to enhance video CSL.
Improved Baselines with Momentum Contrastive Learning
Contrastive unsupervised learning has recently shown encouraging progress, e. g., in Momentum Contrast (MoCo) and SimCLR.
Ranked #3 on
Contrastive Learning
on imagenet-1k
Momentum Contrast for Unsupervised Visual Representation Learning
This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning.
Ranked #11 on
Contrastive Learning
on imagenet-1k
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
Similarly, the output feature maps of a convolution layer can also be seen as a mixture of information at different frequencies.
Ranked #151 on
Action Classification
on Kinetics-400
Long-Term Feature Banks for Detailed Video Understanding
To understand the world, we humans constantly need to relate the present to the past, and put events in context.
Ranked #4 on
Egocentric Activity Recognition
on EPIC-KITCHENS-55
SlowFast Networks for Video Recognition
We present SlowFast networks for video recognition.
Ranked #4 on
Action Recognition
on AVA v2.1
Stacked Latent Attention for Multimodal Reasoning
Attention has shown to be a pivotal development in deep learning and has been used for a multitude of multimodal learning tasks such as visual question answering and image captioning.
Efficient K-Shot Learning with Regularized Deep Networks
To efficiently search for optimal groupings conditioned on the input data, we propose a reinforcement learning search strategy using recurrent networks to learn the optimal group assignments for each network layer.
Going Deeper into First-Person Activity Recognition
Our appearance stream encodes prior knowledge of the egocentric paradigm by explicitly training the network to segment hands and localize objects.
