Search Results for author:
Found 36 papers, 26 papers with code
Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering
Distributed learning has become the standard approach for training large-scale machine learning models across private data silos.
Apple Intelligence Foundation Language Models
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute.
Idempotence and Perceptual Image Compression
However, we find that theoretically: 1) Conditional generative model-based perceptual codec satisfies idempotence; 2) Unconditional generative model with idempotence constraint is equivalent to conditional generative codec.
Bandwidth-efficient Inference for Neural Image Compression
With neural networks growing deeper and feature maps growing larger, limited communication bandwidth with external memory (or DRAM) and power constraints become a bottleneck in implementing network inference on mobile and edge devices.
Conditional Perceptual Quality Preserving Image Compression
We propose conditional perceptual quality, an extension of the perceptual quality defined in \citet{blau2018perception}, by conditioning it on user defined information.
R-MAE: Regions Meet Masked Autoencoders
In this work, we explore regions as a potential visual analogue of words for self-supervised image representation learning.
Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles
Modern hierarchical vision transformers have added several vision-specific components in the pursuit of supervised classification performance.
Ranked #1 on
Image Classification
on iNaturalist 2019
(using extra training data)
Diffusion Models as Masked Autoencoders
There has been a longstanding belief that generation can facilitate a true understanding of visual data.
Efficient Semantic Segmentation by Altering Resolutions for Compressed Videos
In this paper, we propose an altering resolution framework called AR-Seg for compressed videos to achieve efficient VSS.
Reversible Vision Transformers
Reversible Vision Transformers achieve a reduced memory footprint of up to 15. 5x at roughly identical model complexity, parameters and accuracy, demonstrating the promise of reversible vision transformers as an efficient backbone for hardware resource limited training regimes.
MAViL: Masked Audio-Video Learners
We present Masked Audio-Video Learners (MAViL) to train audio-visual representations.
Scaling Language-Image Pre-training via Masking
We present Fast Language-Image Pre-training (FLIP), a simple and more efficient method for training CLIP.
Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization
Our experiments show the proposed adaptations improve egocentric query detection, leading to a better visual query localization system in both 2D and 3D configurations.
Negative Frames Matter in Egocentric Visual Query 2D Localization
The repeated gradient computation of the same object lead to an inefficient training; (2) The false positive rate is high on background frames.
Masked Autoencoders As Spatiotemporal Learners
We randomly mask out spacetime patches in videos and learn an autoencoder to reconstruct them in pixels.
Exploring Plain Vision Transformer Backbones for Object Detection
This design enables the original ViT architecture to be fine-tuned for object detection without needing to redesign a hierarchical backbone for pre-training.
Ranked #1 on
Cross-Domain Few-Shot Object Detection
on Clipark1k
(mAP metric)
Cross-Domain Few-Shot Object Detection
Instance Segmentation
+3
MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition
Instead of trying to process more frames at once like most existing methods, we propose to process videos in an online fashion and cache "memory" at each iteration.
Ranked #5 on
Action Anticipation
on EPIC-KITCHENS-100
(using extra training data)
MViTv2: Improved Multiscale Vision Transformers for Classification and Detection
In this paper, we study Multiscale Vision Transformers (MViTv2) as a unified architecture for image and video classification, as well as object detection.
Ranked #1 on
Action Classification
on Kinetics-600
(GFLOPs metric)
Benchmarking Detection Transfer Learning with Vision Transformers
The complexity of object detection methods can make this benchmarking non-trivial when new architectures, such as Vision Transformer (ViT) models, arrive.
PyTorchVideo: A Deep Learning Library for Video Understanding
We introduce PyTorchVideo, an open-source deep-learning library that provides a rich set of modular, efficient, and reproducible components for a variety of video understanding tasks, including classification, detection, self-supervised learning, and low-level processing.
Masked Autoencoders Are Scalable Vision Learners
Our MAE approach is simple: we mask random patches of the input image and reconstruct the missing pixels.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Multiscale Vision Transformers
We evaluate this fundamental architectural prior for modeling the dense nature of visual signals for a variety of video recognition tasks where it outperforms concurrent vision transformers that rely on large scale external pre-training and are 5-10x more costly in computation and parameters.
Ranked #14 on
Action Classification
on Charades
Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos
We introduce an approach for pre-training egocentric video models using large-scale third-person video datasets.
Modality Compensation Network: Cross-Modal Adaptation for Action Recognition
In this work, we propose a Modality Compensation Network (MCN) to explore the relationships of different modalities, and boost the representations for human action recognition.
EGO-TOPO: Environment Affordances from Egocentric Video
We introduce a model for environment affordances that is learned directly from egocentric video.
SimpleDet: A Simple and Versatile Distributed Framework for Object Detection and Instance Recognition
A Simple and Versatile Framework for Object Detection and Instance Recognition
Scale-Aware Trident Networks for Object Detection
In this work, we first present a controlled experiment to investigate the effect of receptive fields for scale variation in object detection.
Ranked #102 on
Object Detection
on COCO test-dev
Temporal Bilinear Networks for Video Action Recognition
Temporal modeling in videos is a fundamental yet challenging problem in computer vision.
PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding
Despite the fact that many 3D human activity benchmarks being proposed, most existing action datasets focus on the action recognition tasks for the segmented videos.
Demystifying Neural Style Transfer
Neural Style Transfer has recently demonstrated very exciting results which catches eyes in both academia and industry.
Factorized Bilinear Models for Image Recognition
Although Deep Convolutional Neural Networks (CNNs) have liberated their power in various computer vision tasks, the most important components of CNN, convolutional layers and fully connected layers, are still limited to linear transformations.
Online Human Action Detection using Joint Classification-Regression Recurrent Neural Networks
In this paper, we study the problem of online action detection from streaming skeleton data.
Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
Skeleton based action recognition distinguishes human actions using the trajectories of skeleton joints, which provide a very good representation for describing actions.
Revisiting Batch Normalization For Practical Domain Adaptation
However, it is still a common annoyance during the training phase, that one has to prepare at least thousands of labeled images to fine-tune a network to a specific domain.
