Search Results for author:
Found 74 papers, 31 papers with code
DiffusionAD: Denoising Diffusion for Anomaly Detection
We introduce a new pipeline, DiffusionAD, to anomaly detection.
Ranked #1 on
Unsupervised Anomaly Detection
on DAGM2007
PromptFusion: Decoupling Stability and Plasticity for Continual Learning
Such a trade-off is referred to as the stabilityplasticity dilemma and is a more general and challenging problem for continual learning.
Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimization
Contrastive Language-Image Pretraining (CLIP) has demonstrated impressive zero-shot learning abilities for image understanding, yet limited effort has been made to investigate CLIP for zero-shot video recognition.
Vision Transformers Are Good Mask Auto-Labelers
We propose Mask Auto-Labeler (MAL), a high-quality Transformer-based mask auto-labeling framework for instance segmentation using only box annotations.
Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning
The dynamic expansion architecture is becoming popular in class incremental learning, mainly due to its advantages in alleviating catastrophic forgetting.
Look Before You Match: Instance Understanding Matters in Video Object Segmentation
Towards this goal, we present a two-branch network for VOS, where the query-based instance segmentation (IS) branch delves into the instance details of the current frame and the VOS branch performs spatial-temporal matching with the memory bank.
Ranked #1 on
Semi-Supervised Video Object Segmentation
on Long Video Dataset
(using extra training data)
Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection
Online media data, in the forms of images and videos, are becoming mainstream communication channels.
Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning
For the choice of teacher models, we observe that students taught by video teachers perform better on temporally-heavy video tasks, while image teachers transfer stronger spatial representations for spatially-heavy video tasks.
Ranked #1 on
Action Recognition
on AVA v2.2
(using extra training data)
Prototypical Residual Networks for Anomaly Detection and Localization
Anomaly detection and localization are widely used in industrial manufacturing for its efficiency and effectiveness.
Ranked #1 on
supervised anomaly detection
on MVTec AD
(using extra training data)
ResFormer: Scaling ViTs with Multi-Resolution Training
Vision Transformers (ViTs) have achieved overwhelming success, yet they suffer from vulnerable resolution scalability, i. e., the performance drops drastically when presented with input resolutions that are unseen during training.
SVFormer: Semi-supervised Video Transformer for Action Recognition
In this paper, we investigate the use of transformer models under the SSL setting for action recognition.
Semi-Supervised Single-View 3D Reconstruction via Prototype Shape Priors
In particular, we introduce an attention-guided prototype shape prior module for guiding realistic object reconstruction.
Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation
Most existing methods for multi-source unsupervised domain adaptation (UDA) rely on a common encoder to extract domain-invariant features.
Multi-Source Unsupervised Domain Adaptation
Unsupervised Domain Adaptation
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
This paper presents OmniVL, a new foundation model to support both image-language and video-language tasks using one universal architecture.
Ranked #1 on
Zero-Shot Video Retrieval
on MSR-VTT
Incorporating Locality of Images to Generate Targeted Transferable Adversarial Examples
In this paper, we revisit adversarial examples with targeted transferability from the perspective of universality and find that highly universal adversarial perturbations tend to be more transferable.
Video Mobile-Former: Video Recognition with Efficient Global Spatial-temporal Modeling
To avoid significant computational cost incurred by computing self-attention between the large number of local patches in videos, we propose to use very few global tokens (e. g., 6) for a whole video in Transformers to exchange information with 3D-CNNs with a cross-attention mechanism.
Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding
We design a detection hub to dynamically adapt queries on category embedding based on the different distributions of datasets.
Deeper Insights into the Robustness of ViTs towards Common Corruptions
With Vision Transformers (ViTs) making great advances in a variety of computer vision tasks, recent literature have proposed various variants of vanilla ViTs to achieve better efficiency and efficacy.
ObjectFormer for Image Manipulation Detection and Localization
Recent advances in image editing techniques have posed serious challenges to the trustworthiness of multimedia data, which drives the research of image tampering detection.
Rethinking Nearest Neighbors for Visual Classification
In this paper, we investigate $k$-Nearest-Neighbor (k-NN) classifiers, a classical model-free learning method from the pre-deep learning era, as an augmentation to modern neural network based approaches.
Cross-Modal Transferable Adversarial Attacks from Images to Videos
This paper investigates the transferability of adversarial perturbation across different modalities, i. e., leveraging adversarial perturbation generated on white-box image models to attack black-box video models.
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation
Our experiments show that there is a trade-off between understanding tasks and generation tasks while using the same model, and a feasible way to improve both tasks is to use more data.
BEVT: BERT Pretraining of Video Transformers
This design is motivated by two observations: 1) transformers learned on image datasets provide decent spatial priors that can ease the learning of video transformers, which are often times computationally-intensive if trained from scratch; 2) discriminative clues, i. e., spatial and temporal information, needed to make correct predictions vary among different videos due to large intra-class and inter-class variations.
Ranked #4 on
Action Recognition
on Diving-48
AdaViT: Adaptive Vision Transformers for Efficient Image Recognition
To this end, we introduce AdaViT, an adaptive computation framework that learns to derive usage policies on which patches, self-attention heads and transformer blocks to use throughout the backbone on a per-input basis, aiming to improve inference efficiency of vision transformers with a minimal drop of accuracy for image recognition.
Efficient Video Transformers with Spatial-Temporal Token Selection
Video transformers have achieved impressive results on major video recognition benchmarks, which however suffer from high computational cost.
Semi-Supervised Vision Transformers
Surprisingly, we show Vision Transformers perform significantly worse than Convolutional Neural Networks when only a small set of labeled data is available.
Attacking Video Recognition Models with Bullet-Screen Comments
On both UCF-101 and HMDB-51 datasets, our BSC attack method can achieve about 90\% fooling rate when attacking three mainstream video recognition models, while only occluding \textless 8\% areas in the video.
Boosting the Transferability of Video Adversarial Examples via Temporal Translation
To this end, we propose to boost the transferability of video adversarial examples for black-box attacks on video recognition models.
Self-supervised Learning for Semi-supervised Temporal Language Grounding
Given a text description, Temporal Language Grounding (TLG) aims to localize temporal boundaries of the segments that contain the specified semantics in an untrimmed video.
Towards Transferable Adversarial Attacks on Vision Transformers
We evaluate the transferability of attacks on state-of-the-art ViTs, CNNs and robustly trained CNNs.
A Multimodal Framework for Video Ads Understanding
There is a growing trend in placing video advertisements on social platforms for online marketing, which demands automatic approaches to understand the contents of advertisements effectively.
FT-TDR: Frequency-guided Transformer and Top-Down Refinement Network for Blind Face Inpainting
Blind face inpainting refers to the task of reconstructing visual contents without explicitly indicating the corrupted regions in a face image.
Cross-domain Contrastive Learning for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain.
Rethinking Pseudo Labels for Semi-Supervised Object Detection
In this paper, we introduce certainty-aware pseudo labels tailored for object detection, which can effectively estimate the classification and localization quality of derived pseudo labels.
Ranked #5 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
VideoLT: Large-scale Long-tailed Video Recognition
In this paper, we introduce VideoLT, a large-scale long-tailed video recognition dataset, as a step toward real-world video recognition.
M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers
We present a novel architecture for 3D object detection, M3DeTR, which combines different point cloud representations (raw, voxels, bird-eye view) with different feature scales based on multi-scale feature pyramids.
Ranked #1 on
3D Object Detection
on KITTI Cars Hard val
HCMS: Hierarchical and Conditional Modality Selection for Efficient Video Recognition
Conventional video recognition pipelines typically fuse multimodal features for improved performance.
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
The widespread dissemination of Deepfakes demands effective approaches that can detect perceptually convincing forged images.
Exploring Visual Engagement Signals for Representation Learning
Visual engagement in social media platforms comprises interactions with photo posts including comments, shares, and likes.
THAT: Two Head Adversarial Training for Improving Robustness at Scale
Many variants of adversarial training have been proposed, with most research focusing on problems with relatively few classes.
Deep Video Inpainting Detection
This paper studies video inpainting detection, which localizes an inpainted region in a video both spatially and temporally.
2D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognition
Then, only frames and convolutions that are selected by the selection network are used in the 3D model to generate predictions.
Ranked #9 on
Action Recognition
on ActivityNet
GTA: Global Temporal Attention for Video Action Understanding
To this end, we introduce Global Temporal Attention (GTA), which performs global temporal attention on top of spatial attention in a decoupled manner.
Intentonomy: a Dataset and Study towards Human Intent Understanding
Based on our findings, we conduct further study to quantify the effect of attending to object and context classes as well as textual information in the form of hashtags when training an intent classifier.
Robust Optimization as Data Augmentation for Large-scale Graphs
Data augmentation helps neural networks generalize better by enlarging the training set, but it remains an open question how to effectively augment graph data to enhance the performance of GNNs (Graph Neural Networks).
Ranked #1 on
Graph Property Prediction
on ogbg-ppa
Prepare for the Worst: Generalizing across Domain Shifts with Adversarial Batch Normalization
Adversarial training is the industry standard for producing models that are robust to small adversarial perturbations.
Encoding Robustness to Image Style via Adversarial Feature Perturbations
We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce models that are robust to various unseen distributional shifts.
Recognizing Instagram Filtered Images with Feature De-stylization
Deep neural networks have been shown to suffer from poor generalization when small perturbations are added (like Gaussian noise), yet little work has been done to evaluate their robustness to more natural image transformations like photo filters.
Learning from Noisy Anchors for One-stage Object Detection
State-of-the-art object detectors rely on regressing and classifying an extensive list of possible anchors, which are divided into positive and negative samples based on their intersection-over-union (IoU) with corresponding groundtruth objects.
LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognition
This paper presents LiteEval, a simple yet effective coarse-to-fine framework for resource efficient video recognition, suitable for both online and offline scenarios.
Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors
We present a systematic study of adversarial attacks on state-of-the-art object detection frameworks.
Efficient Object Embedding for Spliced Image Retrieval
Detecting spliced images is one of the emerging challenges in computer vision.
ACE: Adapting to Changing Environments for Semantic Segmentation
However, neural classifiers are often extremely brittle when confronted with domain shift---changes in the input distribution that occur over time.
An Analysis of Pre-Training on Object Detection
We analyze how well their features generalize to tasks like image classification, semantic segmentation and object detection on small datasets like PASCAL-VOC, Caltech-256, SUN-397, Flowers-102 etc.
M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning
Instead of sequentially distilling knowledge only from the last model, we directly leverage all previous model snapshots.
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
The Regretful Navigation Agent for Vision-and-Language Navigation
As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.
Compatible and Diverse Fashion Image Inpainting
The latent representations are jointly optimized with the corresponding generation network to condition the synthesis process, encouraging a diverse set of generated results that are visually compatible with existing fashion garments.
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
Natural Language Visual Grounding
Vision and Language Navigation
+2
AdaFrame: Adaptive Frame Selection for Fast Video Recognition
We present AdaFrame, a framework that adaptively selects relevant frames on a per-input basis for fast video recognition.
DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
In particular, given an image from the source domain and unlabeled samples from the target domain, the generator synthesizes new images on-the-fly to resemble samples from the target domain in appearance and the segmentation network further refines high-level features before predicting semantic maps, both of which leverage feature statistics of sampled images from the target domain.
VITON: An Image-based Virtual Try-on Network
We present an image-based VIirtual Try-On Network (VITON) without using 3D information in any form, which seamlessly transfers a desired clothing item onto the corresponding region of a person using a coarse-to-fine strategy.
BlockDrop: Dynamic Inference Paths in Residual Networks
Very deep convolutional neural networks offer excellent recognition results, yet their computational expense limits their impact for many real-world applications.
Automatic Spatially-aware Fashion Concept Discovery
This paper proposes an automatic spatially-aware concept discovery approach using weakly labeled image-text data from shopping websites.
Learning Fashion Compatibility with Bidirectional LSTMs
To this end, we propose to jointly learn a visual-semantic embedding and the compatibility relationships among fashion items in an end-to-end fashion.
Aggregating Frame-level Features for Large-Scale Video Classification
This paper introduces the system we developed for the Google Cloud & YouTube-8M Video Understanding Challenge, which can be considered as a multi-label classification problem defined on top of the large scale YouTube-8M Dataset.
Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classification
More specifically, we utilize three Convolutional Neural Networks (CNNs) operating on appearance, motion and audio signals to extract their corresponding features.
Weakly-Supervised Spatial Context Networks
In particular, we propose spatial context networks that learn to predict a representation of one image patch from another image patch, within the same image, conditioned on their real-valued relative spatial offset.
Deep Learning for Video Classification and Captioning
Accelerated by the tremendous increase in Internet bandwidth and storage space, video data has been generated, published and spread explosively, becoming an indispensable part of today's big data.
Harnessing Object and Scene Semantics for Large-Scale Video Understanding
Large-scale action recognition and video categorization are important problems in computer vision.
Fusing Multi-Stream Deep Networks for Video Classification
A multi-stream framework is proposed to fully utilize the rich multimodal information in videos.
Evaluating Two-Stream CNN for Video Classification
In this paper, we conduct an in-depth study to investigate important implementation options that may affect the performance of deep nets on video classification.
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
In this paper, we propose a hybrid deep learning framework for video classification, which is able to model static spatial information, short-term motion, as well as long-term temporal clues in the videos.
Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
In this paper, we study the challenging problem of categorizing videos according to high-level semantics such as the existence of a particular human action or a complex event.
