Search Results for author:
Found 66 papers, 20 papers with code
Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation
Experiments show that our approach achieves a new state-of-the-art performance on VG and GQA datasets and makes a trade-off between the performance of tail predicates and head ones.
Adaptive Fine-Grained Predicates Learning for Scene Graph Generation
The performance of current Scene Graph Generation (SGG) models is severely hampered by hard-to-distinguish predicates, e. g., woman-on/standing on/walking on-beach.
KE-RCNN: Unifying Knowledge based Reasoning into Part-level Attribute Parsing
In this article, we propose a Knowledge Embedded RCNN (KE-RCNN) to identify attributes by leveraging rich knowledges, including implicit knowledge (e. g., the attribute ``above-the-hip'' for a shirt requires visual/geometry relations of shirt-hip) and explicit knowledge (e. g., the part of ``shorts'' cannot have the attribute of ``hoodie'' or ``lining'').
From Pixels to Objects: Cubic Visual Attention for Visual Question Answering
Existing visual attention models are generally planar, i. e., different channels of the last conv-layer feature map of an image share the same weight.
Structured Two-stream Attention Network for Video Question Answering
To date, visual question answering (VQA) (i. e., image QA and video QA) is still a holy grail in vision and language understanding, especially for video QA.
Thunder: Thumbnail based Fast Lightweight Image Denoising Network
To achieve promising results on removing noise from real-world images, most of existing denoising networks are formulated with complex network structure, making them impractical for deployment.
Support-set based Multi-modal Representation Enhancement for Video Captioning
Video captioning is a challenging task that necessitates a thorough comprehension of visual scenes.
Fine-Grained Predicates Learning for Scene Graph Generation
The performance of current Scene Graph Generation models is severely hampered by some hard-to-distinguish predicates, e. g., "woman-on/standing on/walking on-beach" or "woman-near/looking at/in front of-child".
Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression
Modeling latent variables with priors and hyperpriors is an essential problem in variational image compression.
Practical No-box Adversarial Attacks with Training-free Hybrid Image Transformation
In this paper, we move a step forward and show the existence of a \textbf{training-free} adversarial perturbation under the no-box threat model, which can be successfully used to attack different DNNs in real-time.
One-shot Scene Graph Generation
Specifically, the Relational Knowledge represents the prior knowledge of relationships between entities extracted from the visual content, e. g., the visual relationships "standing in", "sitting in", and "lying in" may exist between "dog" and "yard", while the Commonsense Knowledge encodes "sense-making" knowledge like "dog can guard yard".
Relation Regularized Scene Graph Generation
Inspired by this observation, in this article, we propose a relation regularized network (R2-Net), which can predict whether there is a relationship between two objects and encode this relation into object feature refinement and better SGG.
Meta Distribution Alignment for Generalizable Person Re-Identification
Domain Generalizable (DG) person ReID is a challenging task which trains a model on source domains yet generalizes well on target domains.
Domain Generalization
Generalizable Person Re-identification
+1
Semi-Supervised Video Paragraph Grounding With Contrastive Encoder
Video events grounding aims at retrieving the most relevant moments from an untrimmed video in terms of a given natural language query.
Fast Gradient Non-sign Methods
Based on the linearity hypothesis, under $\ell_\infty$ constraint, $sign$ operation applied to the gradients is a good choice for generating perturbations.
From General to Specific: Informative Scene Graph Generation via Balance Adjustment
The scene graph generation (SGG) task aims to detect visual relationship triplets, i. e., subject, predicate, object, in an image, providing a structural vision layout for scene understanding.
Adversarial Energy Disaggregation for Non-intrusive Load Monitoring
Energy disaggregation, also known as non-intrusive load monitoring (NILM), challenges the problem of separating the whole-home electricity usage into appliance-specific individual consumptions, which is a typical application of data analysis.
Multi-Stage Aggregated Transformer Network for Temporal Language Localization in Videos
Then for a moment candidate, we concatenate the starting/middle/ending representations of its starting/middle/ending elements respectively to form the final moment representation.
Staircase Sign Method for Boosting Adversarial Attacks
Crafting adversarial examples for the transfer-based attack is challenging and remains a research hot spot.
Patch-wise++ Perturbation for Adversarial Targeted Attacks
Specifically, we introduce an amplification factor to the step size in each iteration, and one pixel's overall gradient overflowing the $\epsilon$-constraint is properly assigned to its surrounding regions by a project kernel.
Dual ResGCN for Balanced Scene GraphGeneration
We propose to incorporate the prior about the co-occurrence of relation pairs into the graph to further help alleviate the class imbalance issue.
Universal Weighting Metric Learning for Cross-Modal Matching
Furthermore, we introduce a new polynomial loss under the universal weighting framework, which defines a weight function for the positive and negative informative pairs respectively.
Patch-wise Attack for Fooling Deep Neural Network
By adding human-imperceptible noise to clean images, the resultant adversarial examples can fool other unknown models.
Improving Target-driven Visual Navigation with Attention on 3D Spatial Relationships
On the other hand, TSE module is used to generate sub-targets which allow agent to learn from failures.
MetaMixUp: Learning Adaptive Interpolation Policy of MixUp with Meta-Learning
Since deep networks are capable of memorizing the entire dataset, the corrupted samples generated by vanilla MixUp with a badly chosen interpolation policy will degrade the performance of networks.
Cooperative Cross-Stream Network for Discriminative Action Representation
It extracts this complementary information of different modality from a connection block, which aims at exploring correlations of different stream features.
Ranked #12 on
Action Recognition
on UCF101
(using extra training data)
Temporal Reasoning Graph for Activity Recognition
In this paper, we propose an efficient temporal reasoning graph (TRG) to simultaneously capture the appearance features and temporal relation between video sequences at multiple time scales.
Ranked #37 on
Action Recognition
on Something-Something V1
Matching Images and Text with Multi-modal Tensor Fusion and Re-ranking
We propose a novel framework that achieves remarkable matching performance with acceptable model complexity.
Deep Recurrent Quantization for Generating Sequential Binary Codes
To the end, when the model is trained, a sequence of binary codes can be generated and the code length can be easily controlled by adjusting the number of recurrent iterations.
Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval
In this work, we propose a deep progressive quantization (DPQ) model, as an alternative to PQ, for large scale image retrieval.
One-Shot Image-to-Image Translation via Part-Global Learning with a Multi-adversarial Framework
It is well known that humans can learn and recognize objects effectively from several limited image samples.
Exact Adversarial Attack to Image Captioning via Structured Output Learning with Latent Variables
Due to the sequential dependencies among words in a caption, we formulate the generation of adversarial noises for targeted partial captions as a structured output learning problem with latent variables.
Exploring Auxiliary Context: Discrete Semantic Transfer Hashing for Scalable Image Retrieval
To address the problem, in this paper, we propose a novel hashing approach, dubbed as \emph{Discrete Semantic Transfer Hashing} (DSTH).
A Large-scale Varying-view RGB-D Action Dataset for Arbitrary-view Human Action Recognition
Besides, we propose a View-guided Skeleton CNN (VS-CNN) to tackle the problem of arbitrary-view action recognition.
Hierarchical LSTMs with Adaptive Attention for Visual Captioning
Also, a hierarchical LSTMs is designed to simultaneously consider both low-level visual information and high-level language context information to support the caption generation.
Highly-Economized Multi-View Binary Compression for Scalable Image Clustering
How to economically cluster large-scale multi-view images is a long-standing problem in computer vision.
Generative Domain-Migration Hashing for Sketch-to-Image Retrieval
The generative model learns a mapping that the distributions of sketches can be indistinguishable from the distribution of natural images using an adversarial loss, and simultaneously learns an inverse mapping based on the cycle consistency loss in order to enhance the indistinguishability.
The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers
Solving mathematical word problems (MWPs) automatically is challenging, primarily due to the semantic gap between human-readable words and machine-understandable logics.
Leveraging Weak Semantic Relevance for Complex Video Event Classification
In this paper, we propose a novel approach to automatically maximize the utility of weak semantic annotations (formalized as the semantic relevance of video shots to the target event) to facilitate video event classification.
From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning
In this paper, we propose a generative approach, referred to as multi-modal stochastic RNNs networks (MS-RNN), which models the uncertainty observed in the data using latent stochastic variables.
Matrix Tri-Factorization With Manifold Regularizations for Zero-Shot Learning
By additionally introducing manifold regularizations on visual data and semantic embeddings, the learned projection can effectively captures the geometrical manifold structure residing in both visual and semantic spaces.
Multi-Attention Network for One Shot Learning
One-shot learning is a challenging problem where the aim is to recognize a class identified by a single training image.
Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning
Specifically, the proposed framework utilizes the temporal attention for selecting specific frames to predict the related words, while the adjusted temporal attention is for deciding whether to depend on the visual information or the language context information.
Deep Region Hashing for Efficient Large-scale Instance Search from Images
Specifically, DRH is an end-to-end deep neural network which consists of object proposal, feature extraction, and hash code generation.
Recurrent Image Captioner: Describing Images with Spatial-Invariant Transformation and Attention Filtering
Along with the prosperity of recurrent neural network in modelling sequential data and the power of attention mechanism in automatically identify salient information, image captioning, a. k. a., image description, has been remarkably advanced in recent years.
Binary Subspace Coding for Query-by-Image Video Retrieval
The first approach, termed Inner-product Binary Coding (IBC), preserves the inner relationships of images and videos in a common Hamming space.
Exploiting Depth from Single Monocular Images for Object Detection and Semantic Segmentation
Augmenting RGB data with measured depth has been shown to improve the performance of a range of tasks in computer vision including object detection and semantic segmentation.
Where to Focus: Query Adaptive Matching for Instance Retrieval Using Convolutional Feature Maps
Instance retrieval requires one to search for images that contain a particular object within a large corpus.
Zero-Shot Hashing via Transferring Supervised Knowledge
Supervised knowledge e. g. semantic labels or pair-wise relationship) associated to data is capable of significantly improving the quality of hash codes and hash functions.
Bidirectional Long-Short Term Memory for Video Description
Video captioning has been attracting broad research attention in multimedia community.
What's Wrong With That Object? Identifying Images of Unusual Objects by Modelling the Detection Score Distribution
The key observation motivating our approach is that "regular object" images, "unusual object" images and "other objects" images exhibit different region-level scores in terms of both the score values and the spatial distributions.
A Survey on Learning to Hash
In this paper, we present a comprehensive survey of the learning to hash algorithms, categorize them according to the manners of preserving the similarities into: pairwise similarity preserving, multiwise similarity preserving, implicit similarity preserving, as well as quantization, and discuss their relations.
Learning Binary Codes and Binary Weights for Efficient Classification
The optimization alternatively proceeds over the binary classifiers and image hash codes.
Structured Learning of Binary Codes with Column Generation
Our column generation based method can be further generalized from the triplet loss to a general structured learning based framework that allows one to directly optimize multivariate performance measures.
Hi Detector, What's Wrong with that Object? Identifying Irregular Object From Images by Modelling the Detection Score Distribution
To address this problem, we propose a novel approach by inspecting the distribution of the detection scores at multiple image regions based on the detector trained from the "regular object" and "other objects".
Order-aware Convolutional Pooling for Video Based Action Recognition
Most video based action recognition approaches create the video-level representation by temporally pooling the features extracted at each frame.
Compositional Model based Fisher Vector Coding for Image Classification
To handle this limitation, in this paper we break the convention which assumes that a local feature is drawn from one of few Gaussian distributions.
Learning Binary Codes for Maximum Inner Product Search
Inspired by the latest advance in asymmetric hashing schemes, we propose an asymmetric binary code learning framework based on inner product fitting.
Optimal Graph Learning With Partial Tags and Multiple Features for Image and Video Annotation
In multimedia annotation, due to the time constraints and the tediousness of manual tagging, it is quite common to utilize both tagged and untagged data to improve the performance of supervised learning when only limited tagged training data are available.
Temporal Pyramid Pooling Based Convolutional Neural Networks for Action Recognition
One challenge is that video contains a varying number of frames which is incompatible to the standard input format of CNNs.
Hashing on Nonlinear Manifolds
In addition, a supervised inductive manifold hashing framework is developed by incorporating the label information, which is shown to greatly advance the semantic retrieval performance.
Hashing for Similarity Search: A Survey
Similarity search (nearest neighbor search) is a problem of pursuing the data items whose distances to a query item are the smallest from a large database.
Face Identification with Second-Order Pooling
Spatial pyramid pooling of features encoded by an over-complete dictionary has been the key component of many state-of-the-art image classification systems.
Face Image Classification by Pooling Raw Features
We propose a very simple, efficient yet surprisingly effective feature extraction method for face recognition (about 20 lines of Matlab code), which is mainly inspired by spatial pyramid pooling in generic image classification.
Optimized Cartesian $K$-Means
In OCKM, multiple sub codewords are used to encode the subvector of a data point in a subspace.
