Search Results for author:
Found 61 papers, 21 papers with code
Hierarchical Visual-Textual Graph for Temporal Activity Localization via Language
Temporal Activity Localization via Language (TALL) in video is a recently proposed challenging vision task, and tackling it requires fine-grained understanding of the video content, however, this is overlooked by most of the existing works.
Boosting the Transferability of Video Adversarial Examples via Temporal Translation
To this end, we propose to boost the transferability of video adversarial examples for black-box attacks on video recognition models.
Two-stage Visual Cues Enhancement Network for Referring Image Segmentation
Referring Image Segmentation (RIS) aims at segmenting the target object from an image referred by one given natural language expression.
Self-supervised Learning for Semi-supervised Temporal Language Grounding
To achieve good performance with limited annotations, we tackle this task in a semi-supervised way and propose a unified Semi-supervised Temporal Language Grounding (STLG) framework.
Towards Transferable Adversarial Attacks on Vision Transformers
The results of these experiments demonstrate that the proposed dual attack can greatly boost transferability between ViTs and from ViTs to CNNs.
A Multimodal Framework for Video Ads Understanding
There is a growing trend in placing video advertisements on social platforms for online marketing, which demands automatic approaches to understand the contents of advertisements effectively.
Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
We empirically demonstrate the effectiveness of our RSLAD approach over existing adversarial training and distillation methods in improving the robustness of small models against state-of-the-art attacks including the AutoAttack.
FT-TDR: Frequency-guided Transformer and Top-Down Refinement Network for Blind Face Inpainting
Blind face inpainting refers to the task of reconstructing visual contents without explicitly indicating the corrupted regions in a face image.
Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data
Secondly, a novel disentangle module together with a domain classifier is proposed to extract the disentangled domain-irrelevant and domain-specific features.
Can Action be Imitated? Learn to Reconstruct and Transfer Human Dynamics from Videos
To achieve this, a novel Mesh-based Video Action Imitation (M-VAI) method is proposed by us.
Towards Bridging Event Captioner and Sentence Localizer for Weakly Supervised Dense Event Captioning
Dense Event Captioning (DEC) aims to jointly localize and describe multiple events of interest in untrimmed videos, which is an advancement of the conventional video captioning task (generating a single sentence description for a trimmed video).
Cross-domain Contrastive Learning for Unsupervised Domain Adaptation
In addition, we demonstrate that CDCL is a general framework and can be adapted to the data-free setting, where the source data are unavailable during training, with minimal modification.
VideoLT: Large-scale Long-tailed Video Recognition
In this paper, we introduce VideoLT, a large-scale long-tailed video recognition dataset, as a step toward real-world video recognition.
HMS: Hierarchical Modality Selection for Efficient Video Recognition
Conventional video recognition pipelines typically fuse multimodal features for improved performance.
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
This demands effective approaches that can detect perceptually convincing Deepfakes generated by advanced manipulation techniques.
What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Space
In this paper, we focus on image classification and propose a method to visualize and understand the class-wise knowledge (patterns) learned by DNNs under three different settings including natural, backdoor and adversarial.
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
WildDeepfake is a small dataset that can be used, in addition to existing datasets, to develop and test the effectiveness of deepfake detectors against real-world deepfakes.
Motion Guided Region Message Passing for Video Captioning
In this paper, we aim at designing a spatial information extraction and aggregation method for video captioning without the need of external object detectors.
Colonoscopy Polyp Detection: Domain Adaptation From Medical Report Images to Real-time Videos
However, there are several issues between the image-based training and video-based inference, including domain differences, lack of positive samples, and temporal smoothness.
Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition
Humans can easily recognize actions with only a few examples given, while the existing video recognition models still heavily rely on the large-scale labeled data inputs.
Multi-modal Cooking Workflow Construction for Food Recipes
Understanding food recipe requires anticipating the implicit causal effects of cooking actions, such that the recipe can be converted into a graph describing the temporal workflow of the recipe.
Learning Modality Interaction for Temporal Sentence Localization and Event Captioning in Videos
Inspired by the fact that there exist cross-modal interactions in the human brain, we propose a novel method for learning pairwise modality interactions in order to better exploit complementary information for each pair of modalities in videos and thus improve performances on both tasks.
Imbalanced Gradients: A New Cause of Overestimated Adversarial Robustness
Evaluating the robustness of a defense model is a challenging task in adversarial robustness research.
Long-Term Cloth-Changing Person Re-identification
Specifically, we consider that under cloth-changes, soft-biometrics such as body shape would be more reliable.
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt
Unfortunately, the representation learned by SketchRNN is primarily for the generation tasks, rather than the other tasks of recognition and retrieval of sketches.
Clean-Label Backdoor Attacks on Video Recognition Models
We propose the use of a universal adversarial trigger as the backdoor trigger to attack video recognition models, a situation where backdoor attacks are likely to be challenged by the above 4 strict conditions.
Learning to Augment Expressions for Few-shot Fine-grained Facial Expression Recognition
Considering the phenomenon of uneven data distribution and lack of samples is common in real-world scenarios, we further evaluate several tasks of few-shot expression learning by virtue of our F2ED, which are to recognize the facial expressions given only few training instances.
LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognition
This paper presents LiteEval, a simple yet effective coarse-to-fine framework for resource efficient video recognition, suitable for both online and offline scenarios.
Heuristic Black-box Adversarial Attacks on Video Recognition Models
To overcome this challenge, we propose a heuristic black-box attack model that generates adversarial perturbations only on the selected frames and regions.
Black-box Adversarial Attacks on Video Recognition Models
Using three benchmark video datasets, we demonstrate that V-BAD can craft both untargeted and targeted attacks to fool two state-of-the-art deep video recognition models.
A Multi-task Neural Approach for Emotion Attribution, Classification and Summarization
However, the sparsity of emotional expressions in the videos poses an obstacle to visual emotion analysis.
Instance-level Sketch-based Retrieval by Deep Triplet Classification Siamese Network
Additionally, to study the tasks of sketch-based hairstyle retrieval, this paper contributes a new instance-level photo-sketch dataset - Hairstyle Photo-Sketch dataset, which is composed of 3600 sketches and photos, and 2400 sketch-photo pairs.
Composite Binary Decomposition Networks
Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts.
Non-local NetVLAD Encoding for Video Classification
This paper describes our solution for the 2$^\text{nd}$ YouTube-8M video understanding challenge organized by Google AI.
Object Detection from Scratch with Deep Supervision
Thus, a better solution to handle these critical problems is to train object detectors from scratch, which motivates our proposed method.
NAIS: Neural Attentive Item Similarity Model for Recommendation
As such, the key to an item-based CF method is in the estimation of item similarities.
Recurrent Fusion Network for Image Captioning
In this paper, in order to exploit the complementary information from multiple encoders, we propose a novel Recurrent Fusion Network (RFNet) for tackling image captioning.
Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks
Recent studies on unsupervised image-to-image translation have made a remarkable progress by training a pair of generative adversarial networks with a cycle-consistent loss.
Cross-Domain Sentiment Classification with Target Domain Specific Information
And we introduce a few target domain labeled data for learning domain-specific information.
Multi-level Semantic Feature Augmentation for One-shot Learning
In semantic space, we search for related concepts, which are then projected back into the image feature spaces by the decoder portion of the TriNet.
Social Anchor-Unit Graph Regularized Tensor Completion for Large-Scale Image Retagging
Recent approaches simultaneously explore visual, user and tag information to improve the performance of image retagging by constructing and exploring an image-tag-user graph.
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
We propose an end-to-end deep learning architecture that produces a 3D shape in triangular mesh from a single color image.
Ranked #3 on
3D Object Reconstruction
on Data3D−R2N2
(Avg F1 metric)
Learning to score the figure skating sports videos
This paper targets at learning to score the figure skating sports videos.
Pose-Normalized Image Generation for Person Re-identification
Person Re-identification (re-id) faces two major challenges: the lack of cross-view paired training data and learning discriminative identity-sensitive and view-invariant features in the presence of large pose variations.
Dual Skipping Networks
Inspired by the recent neuroscience studies on the left-right asymmetry of the human brain in processing low and high spatial frequency information, this paper introduces a dual skipping network which carries out coarse-to-fine object categorization.
Recent Advances in Zero-shot Recognition
With the recent renaissance of deep convolution neural networks, encouraging breakthroughs have been achieved on the supervised recognition tasks, where each class has sufficient training data and fully annotated training data.
Multi-scale Deep Learning Architectures for Person Re-identification
Our model is able to learn deep discriminative feature representations at different scales and automatically determine the most suitable scales for matching.
DSOD: Learning Deeply Supervised Object Detectors from Scratch
State-of-the-art object objectors rely heavily on the off-the-shelf networks pre-trained on large-scale classification datasets like ImageNet, which incurs learning bias due to the difference on both the loss functions and the category distributions between classification and detection tasks.
Learning Fashion Compatibility with Bidirectional LSTMs
To this end, we propose to jointly learn a visual-semantic embedding and the compatibility relationships among fashion items in an end-to-end fashion.
Aggregating Frame-level Features for Large-Scale Video Classification
This paper introduces the system we developed for the Google Cloud & YouTube-8M Video Understanding Challenge, which can be considered as a multi-label classification problem defined on top of the large scale YouTube-8M Dataset.
Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classification
More specifically, we utilize three Convolutional Neural Networks (CNNs) operating on appearance, motion and audio signals to extract their corresponding features.
Weakly Supervised Dense Video Captioning
This paper focuses on a novel and challenging vision task, dense video captioning, which aims to automatically describe a video clip with multiple informative and diverse caption sentences.
Iterative Object and Part Transfer for Fine-Grained Recognition
On both datasets, we achieve better results than many state-of-the-art approaches, including a few using oracle (manually annotated) bounding boxes in the test images.
Deep Learning for Video Classification and Captioning
Accelerated by the tremendous increase in Internet bandwidth and storage space, video data has been generated, published and spread explosively, becoming an indispensable part of today's big data.
Harnessing Object and Scene Semantics for Large-Scale Video Understanding
Large-scale action recognition and video categorization are important problems in computer vision.
The THUMOS Challenge on Action Recognition for Videos "in the Wild"
Additionally, we include a comprehensive empirical study evaluating the differences in action recognition between trimmed and untrimmed videos, and how well methods trained on trimmed videos generalize to untrimmed videos.
Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization
Emotion is a key element in user-generated videos.
Fusing Multi-Stream Deep Networks for Video Classification
A multi-stream framework is proposed to fully utilize the rich multimodal information in videos.
Evaluating Two-Stream CNN for Video Classification
In this paper, we conduct an in-depth study to investigate important implementation options that may affect the performance of deep nets on video classification.
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
In this paper, we propose a hybrid deep learning framework for video classification, which is able to model static spatial information, short-term motion, as well as long-term temporal clues in the videos.
Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
In this paper, we study the challenging problem of categorizing videos according to high-level semantics such as the existence of a particular human action or a complex event.