Search Results for author:
Found 288 papers, 115 papers with code
Robust Region Grouping via Internal Patch Statistics
In this work, we present an efficient multi-scale low-rank representation for image segmentation.
PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors
By fusing complementary contrast measures in such a pixelwise adaptive manner, the detection effectiveness is significantly boosted.
Computational Baby Learning
Then the concept detector can be fine-tuned based on these new instances.
Correntropy Induced L2 Graph for Robust Subspace Clustering
In this paper, we study the robust subspace clustering problem, which aims to cluster the given possibly noisy data points into their underlying subspaces.
3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks
Our model thus advances existing approaches in two aspects: (i) it acts directly on the raw inputs (grayscale-depth data) to conduct recognition instead of relying on hand-crafted features, and (ii) the model structure can be dynamically adjusted accounting for the temporal variations of human activities, i. e. the network configuration is allowed to be partially activated during inference.
End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning
Sketch-based face recognition is an interesting task in vision and multimedia research, yet it is quite challenging due to the great difference between face photos and sketches.
Learning Latent Spatio-Temporal Compositional Model for Human Action Recognition
Action recognition is an important problem in multimedia understanding.
Human Re-identification by Matching Compositional Template with Cluster Sampling
This paper aims at a newly raising task in visual surveillance: re-identifying people at a distance by matching body information, given several reference examples.
Adaptive Scene Category Discovery with Generative Learning and Compositional Sampling
During the iterations of inference, the model of each category is analytically updated by a generative learning algorithm.
Discriminatively Trained And-Or Graph Models for Object Shape Detection
In this paper, we investigate a novel reconfigurable part-based model, namely And-Or graph model, to recognize object shapes in images.
Integrating Graph Partitioning and Matching for Trajectory Analysis in Video Surveillance
With this graph representation, we pose trajectory analysis as a joint task of spatial graph partitioning and temporal graph matching.
An Expressive Deep Model for Human Action Parsing from A Single Image
This paper aims at one newly raising task in vision and multimedia research: recognizing human actions from still images.
Towards a solid solution of real-time fire and flame detection
Although the object detection and recognition has received growing attention for decades, a robust fire and flame detection method is rarely explored.
Complex Background Subtraction by Pursuing Dynamic Spatio-Temporal Models
Although it has been widely discussed in video surveillance, background subtraction is still an open problem in the context of complex scenarios, e. g., dynamic backgrounds, illumination variations, and indistinct foreground objects.
Iterated Support Vector Machines for Distance Metric Learning
Distance metric learning aims to learn from the given training data a valid distance metric, with which the similarity between data samples can be more effectively evaluated for classification.
Dynamical And-Or Graph Learning for Object Shape Modeling and Detection
A discriminative learning algorithm, extended from the CCCP [23], is proposed to train the model in a dynamical manner: the model structure (e. g., the configuration of the leaf-nodes associated with the or-nodes) is automatically determined with optimizing the multi-layer parameters during the iteration.
Recognizing Focal Liver Lesions in Contrast-Enhanced Ultrasound with Discriminatively Trained Spatio-Temporal Model
The aim of this study is to provide an automatic computational framework to assist clinicians in diagnosing Focal Liver Lesions (FLLs) in Contrast-Enhancement Ultrasound (CEUS).
Learning Contour-Fragment-based Shape Model with And-Or Tree Representation
This paper proposes a simple yet effective method to learn the hierarchical object shape model consisting of local contour fragments, which represents a category of shapes in the form of an And-Or tree.
Clothing Co-Parsing by Joint Image Segmentation and Labeling
This paper aims at developing an integrated system of clothing co-parsing, in order to jointly parse a set of clothing images (unsegmented but annotated with tags) into semantic configurations.
Deep Joint Task Learning for Generic Object Extraction
We present a general joint task learning framework, in which each task (either object localization or object segmentation) is tackled via a multi-layer convolutional neural network, and the two networks work collaboratively to boost performance.
Incorporating Structural Alternatives and Sharing into Hierarchy for Multiclass Object Recognition and Detection
This paper proposes a reconfigurable model to recognize and detect multiclass (or multiview) objects with large variation in appearance.
Data-Driven Scene Understanding with Adaptively Retrieved Exemplars
In the first step, the references are selected by jointly matching their appearances with the target as well as the semantics (i. e. the assigned labels of the target and the references).
Deep Boosting: Layered Feature Mining for General Image Classification
Constructing effective representations is a critical but challenging problem in multimedia understanding.
Deep Human Parsing with Active Template Regression
The first CNN network is with max-pooling, and designed to predict the template coefficients for each label mask, while the second CNN network is without max-pooling to preserve sensitivity to label mask position and accurately predict the active shape parameters.
Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
Under the classic K Nearest Neighbor (KNN)-based nonparametric framework, the parametric Matching Convolutional Neural Network (M-CNN) is proposed to predict the matching confidence and displacements of the best matched region in the testing image for a particular semantic region in one KNN image.
F-SVM: Combination of Feature Transformation and SVM Learning via Convex Relaxation
The generalization error bound of support vector machine (SVM) depends on the ratio of radius and margin, while standard SVM only considers the maximization of the margin but ignores the minimization of the radius.
PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Edge-Preserving Coherence
In this paper, we propose a unified framework called PISA, which stands for Pixelwise Image Saliency Aggregating various bottom-up cues and priors.
SOLD: Sub-Optimal Low-rank Decomposition for Efficient Video Segmentation
In particular, the affinity matrix with the rank fixed can be decomposed into two sub-matrices of low rank, and then we iteratively optimize them with closed-form solutions.
Discriminative Learning of Iteration-Wise Priors for Blind Deconvolution
The maximum a posterior (MAP)-based blind deconvolution framework generally involves two stages: blur kernel estimation and non-blind restoration.
Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks
This paper investigates how to rapidly and accurately localize facial landmarks in unconstrained, cluttered environments rather than in the well segmented face images.
Deep Boosting: Joint Feature Selection and Analysis Dictionary Learning in Hierarchy
In each layer, we construct a dictionary of filters by combining the filters from the lower layer, and iteratively optimize the image representation with a joint discriminative-generative formulation, i. e. minimization of empirical classification error plus regularization of analysis image generation over training images.
Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification
Furthermore, each bit of our hashing codes is unequally weighted so that we can manipulate the code lengths by truncating the insignificant bits.
Proposal-free Network for Instance-level Object Segmentation
Instance-level object segmentation is an important yet under-explored task.
DISC: Deep Image Saliency Computing via Progressive Representation Learning
Our DISC framework is capable of uniformly highlighting the objects-of-interest from complex background while preserving well object details.
Semantic Object Parsing with Local-Global Long Short-Term Memory
The long chains of sequential computation by stacked LG-LSTM layers also enable each pixel to sense a much larger region for inference benefiting from the memorization of previous dependencies in all positions along all dimensions.
Towards Computational Baby Learning: A Weakly-Supervised Approach for Object Detection
Then the concept detector can be fine-tuned based on these new instances.
Human Parsing With Contextualized Convolutional Neural Network
In this work, we address the human parsing task with a novel Contextualized Convolutional Neural Network (Co-CNN) architecture, which well integrates the cross-layer context, global image-level context, within-super-pixel context and cross-super-pixel neighborhood context into a unified network.
A Deep Structured Model with Radius-Margin Bound for 3D Human Activity Recognition
Understanding human activity is very challenging even with the recently developed 3D/depth sensors.
Deep Feature Learning with Relative Distance Comparison for Person Re-identification
Identifying the same individual across different scenes is an important yet difficult task in intelligent video surveillance.
Ranked #9 on
Person Re-Identification
on SYSU-30k
(using extra training data)
Learning Support Correlation Filters for Visual Tracking
Sampling and budgeting training examples are two essential factors in tracking algorithms based on support vector machines (SVMs) as a trade-off between accuracy and efficiency.
Character Proposal Network for Robust Text Extraction
Maximally stable extremal regions (MSER), which is a popular method to generate character proposals/candidates, has shown superior performance in scene text detection.
Semantic Object Parsing with Graph LSTM
By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data.
Geometric Scene Parsing with Hierarchical LSTM
This paper addresses the problem of geometric scene parsing, i. e. simultaneously labeling geometric surfaces (e. g. sky, ground and vertical plane) and determining the interaction relations (e. g. layering, supporting, siding and affinity) between main regions.
Joint Detection and Identification Feature Learning for Person Search
Existing person re-identification benchmarks and methods mainly focus on matching cropped pedestrian images between queries and candidates.
Ranked #9 on
Person Re-Identification
on CUHK03
Deep Structured Scene Parsing by Learning with Image Descriptions
This paper addresses a fundamental problem of scene understanding: How to parse the scene image into a structured configuration (i. e., a semantic object hierarchy with object interaction relations) that finely accords with human perception.
DARI: Distance metric And Representation Integration for Person Verification
The past decade has witnessed the rapid development of feature representation learning and distance metric learning, whereas the two steps are often discussed separately.
Ranked #7 on
Person Re-Identification
on SYSU-30k
(using extra training data)
LSTM-CF: Unifying Context Modeling and Fusion with LSTMs for RGB-D Scene Labeling
Another long short-term memorized fusion layer is set up to integrate the contexts along the vertical direction from different channels, and perform bi-directional propagation of the fused vertical contexts along the horizontal direction to obtain true 2D global contexts.
Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning
Cross-domain visual data matching is one of the fundamental problems in many real-world vision tasks, e. g., matching persons across ID photos and surveillance videos.
Joint Learning of Single-Image and Cross-Image Representations for Person Re-Identification
Person re-identification has been usually solved as either the matching of single-image representation (SIR) or the classification of cross-image representation (CIR).
Dictionary Pair Classifier Driven Convolutional Neural Networks for Object Detection
Feature representation and object category classification are two key components of most object detection methods.
Is Faster R-CNN Doing Well for Pedestrian Detection?
Detecting pedestrian has been arguably addressed as a special topic beyond general object detection.
Ranked #19 on
Pedestrian Detection
on Caltech
Local- and Holistic- Structure Preserving Image Super Resolution via Deep Joint Component Learning
Recently, machine learning based single image super resolution (SR) approaches focus on jointly learning representations for high-resolution (HR) and low-resolution (LR) image patch pairs to improve the quality of the super-resolved images.
Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning
In this paper, we propose a novel inference-embedded multi-task learning framework for predicting human pose from still depth images, which is implemented with a deep architecture of neural networks.
Learning to Segment Object Candidates via Recursive Neural Networks
To avoid the exhaustive search over locations and scales, current state-of-the-art object detection systems usually involve a crucial component generating a batch of candidate object proposals from images.
Cost-Effective Active Learning for Deep Image Classification
In this paper, we propose a novel active learning framework, which is capable of building a competitive classifier with optimal feature representation via a limited amount of labeled training instances in an incremental learning manner.
Active Self-Paced Learning for Cost-Effective and Progressive Face Identification
By naturally combining two recently rising techniques: active learning (AL) and self-paced learning (SPL), our framework is capable of automatically annotating new instances and incorporating them into training under weak expert re-certification.
Progressively Diffused Networks for Semantic Image Segmentation
This paper introduces Progressively Diffused Networks (PDNs) for unifying multi-scale context modeling with deep feature learning, by taking semantic image segmentation as an exemplar application.
Interpretable Structure-Evolving LSTM
Instead of learning LSTM models over the pre-fixed structures, we propose to further learn the intermediate interpretable multi-level graph structures in a progressive and stochastic way from data during the LSTM network optimization.
Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing
Human parsing has recently attracted a lot of research interests due to its huge application potentials.
Ranked #13 on
Semantic Segmentation
on LIP val
Instance-Level Salient Object Segmentation
Image saliency detection has recently witnessed rapid progress due to deep convolutional neural networks.
Ranked #15 on
RGB Salient Object Detection
on DUTS-TE
(max F-measure metric)
Learning Object Interactions and Descriptions for Semantic Image Segmentation
This work significantly increases segmentation accuracy of CNNs by learning from an Image Descriptions in the Wild (IDW) dataset.
Knowledge-Guided Recurrent Neural Network Learning for Task-Oriented Action Prediction
This paper aims at task-oriented action prediction, i. e., predicting a sequence of actions towards accomplishing a specific task under a certain scene, which is a new problem in computer vision research.
Structure-Preserving Image Super-resolution via Contextualized Multi-task Learning
Single image super resolution (SR), which refers to reconstruct a higher-resolution (HR) image from the observed low-resolution (LR) image, has received substantial attention due to its tremendous application potentials.
Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning
Aiming at improving performance of visual classification in a cost-effective manner, this paper proposes an incremental semi-supervised learning paradigm called Deep Co-Space (DCS).
Recurrent 3D Pose Sequence Machines
3D human articulated pose recovery from monocular image sequences is very challenging due to the diverse appearances, viewpoints, occlusions, and also the human 3D pose is inherently ambiguous from the monocular imagery.
Ranked #20 on
3D Human Pose Estimation
on HumanEva-I
Attention-Aware Face Hallucination via Deep Reinforcement Learning
Face hallucination is a domain-specific super-resolution problem with the goal to generate high-resolution (HR) faces from low-resolution (LR) input images.
Hierarchical Scene Parsing by Weakly Supervised Learning with Image Descriptions
Rather than relying on elaborative annotations (e. g., manually labeled semantic maps and relations), we train our deep model in a weakly-supervised learning manner by leveraging the descriptive sentences of the training images.
Deep Dual Learning for Semantic Image Segmentation
The estimated labelmaps that capture accurate object classes and boundaries are used as ground truths in training to boost performance.
Visual Tracking via Dynamic Graph Learning
First, the graph is initialized by assigning binary weights of some image patches to indicate the object and background patches according to the predicted bounding box.
Learning to Segment Human by Watching YouTube
An intuition on human segmentation is that when a human is moving in a video, the video-context (e. g., appearance and motion clues) may potentially infer reasonable mask information for the whole human body.
Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning
Sketch portrait generation benefits a wide range of applications such as digital entertainment and law enforcement.
Multi-label Image Recognition by Recurrently Discovering Attentional Regions
This paper proposes a novel deep architecture to address multi-label image recognition, a fundamental and practical task towards general visual understanding.
LSTM Pose Machines
Such suboptimal results are mainly attributed to the inability of imposing sequential geometric consistency, handling severe image quality degradation (e. g. motion blur and occlusion) as well as the inability of capturing the temporal correlation among video frames.
Ranked #3 on
Pose Estimation
on J-HMDB
Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networks
In this work, aiming at a general and comprehensive way for neural network acceleration, we develop a Wavelet-like Auto-Encoder (WAE) that decomposes the original input image into two low-resolution channels (sub-images) and incorporate the WAE into the classification neural networks for joint training.
Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition
Recognizing multiple labels of images is a fundamental but challenging task in computer vision, and remarkable progress has been attained by localizing semantic-aware image regions and predicting their labels with deep convolutional neural networks.
Context-Aware Semantic Inpainting
Our proposed GAN-based framework consists of a fully convolutional design for the generator which helps to better preserve spatial structures and a joint loss function with a revised perceptual loss to capture high-level semantics in the context.
Structured Inhomogeneous Density Map Learning for Crowd Counting
In this paper, we aim at tackling the problem of crowd counting in extremely high-density scenes, which contain hundreds, or even thousands of people.
Batch Kalman Normalization: Towards Training Deep Neural Networks with Micro-Batches
As an indispensable component, Batch Normalization (BN) has successfully improved the training of deep neural networks (DNNs) with mini-batches, by normalizing the distribution of the internal representation for each hidden layer.
Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift
Motivated by the theoretical results in \cite{mansour2009domain}, the target distribution can be represented as the weighted combination of source distributions, and, the multi-source unsupervised domain adaptation via DCTN is then performed as two alternating steps: i) It deploys multi-way adversarial learning to minimize the discrepancy between the target and each of the multiple source domains, which also obtains the source-specific perplexity scores to denote the possibilities that a target sample belongs to different source domains.
Multi-Source Unsupervised Domain Adaptation
Unsupervised Domain Adaptation
Single View Stereo Matching
The resulting model outperforms all the previous monocular depth estimation methods as well as the stereo block matching method in the challenging KITTI dataset by only using a small number of real training data.
Ranked #43 on
Monocular Depth Estimation
on KITTI Eigen split
Weakly Supervised Salient Object Detection Using Image Labels
Our algorithm is based on alternately exploiting a graphical model and training a fully convolutional network for model updating.
Zoom and Learn: Generalizing Deep Stereo Matching to Novel Domains
By feeding real stereo pairs of different domains to stereo models pre-trained with synthetic data, we see that: i) a pre-trained model does not generalize well to the new domain, producing artifacts at boundaries and ill-posed regions; however, ii) feeding an up-sampled stereo pair leads to a disparity map with extra details.
Towards Human-Machine Cooperation: Self-supervised Sample Mining for Object Detection
Though quite challenging, leveraging large-scale unlabeled or partially labeled images in a cost-effective way has increasingly attracted interests for its great importance to computer vision.
Visual Question Reasoning on General Dependency Tree
This network comprises of two collaborative modules: i) an adversarial attention module to exploit the local visual evidence for each word parsed from the question; ii) a residual composition module to compose the previously mined evidence.
Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark
To further explore and take advantage of the semantic correlation of these two tasks, we propose a novel joint human parsing and pose estimation network to explore efficient context modeling, which can simultaneously predict parsing and pose with extremely high quality.
Ranked #10 on
Semantic Segmentation
on LIP val
Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
We investigate a novel approach for image restoration by reinforcement learning.
Learning Warped Guidance for Blind Face Restoration
For better recovery of fine facial details, we modify the problem setting by taking both the degraded observation and a high-quality guided image of the same identity as input to our guided face restoration network (GFRNet).
Ranked #1 on
Image Super-Resolution
on WebFace - 8x upscaling
Multi-level Wavelet-CNN for Image Restoration
With the modified U-Net architecture, wavelet transform is introduced to reduce the size of feature maps in the contracting subnetwork.
Ranked #2 on
Grayscale Image Denoising
on Set12 sigma25
DRPose3D: Depth Ranking in 3D Human Pose Estimation
In this paper, we propose a two-stage depth ranking based method (DRPose3D) to tackle the problem of 3D human pose estimation.
Interpretable Video Captioning via Trajectory Structured Localization
Automatically describing open-domain videos with natural language are attracting increasing interest in the field of artificial intelligence.
Flow Guided Recurrent Neural Encoder for Video Salient Object Detection
Image saliency detection has recently witnessed significant progress due to deep convolutional neural networks.
Ranked #2 on
Video Salient Object Detection
on DAVSOD-Difficult20
(using extra training data)
Cost-effective Object Detection: Active Sample Mining with Switchable Selection Criteria
The proposed process can be compatible with mini-batch based training (i. e., using a batch of unlabeled or partially labeled data as a one-time input) for object detection.
Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition
Humans can naturally understand an image in depth with the aid of rich knowledge accumulated from daily lives or professions.
Fine-Grained Image Classification
Fine-Grained Image Recognition
+1
Deep Reasoning with Knowledge Graph for Social Relationship Understanding
And this structured knowledge can be efficiently integrated into the deep neural network architecture to promote social relationship understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which a propagation mechanism is learned to propagate node message through the graph to explore the interaction between persons of interest and the contextual objects.
Ranked #2 on
Visual Social Relationship Recognition
on PIPA
Crowd Counting using Deep Recurrent Spatial-Aware Network
Crowd counting from unconstrained scene images is a crucial task in many real-world applications like urban surveillance and management, but it is greatly challenged by the camera's perspective that causes huge appearance variations in people's scales and rotations.
SCAN: Self-and-Collaborative Attention Network for Video Person Re-identification
To address the above issues, we present a novel and practical deep architecture for video person re-identification termed Self-and-Collaborative Attention Network (SCAN).
Toward Characteristic-Preserving Image-based Virtual Try-On Network
Second, to alleviate boundary artifacts of warped clothes and make the results more realistic, we employ a Try-On Module that learns a composition mask to integrate the warped clothes and the rendered image to ensure smoothness.
Instance-level Human Parsing via Part Grouping Network
Instance-level human parsing towards real-world human analysis scenarios is still under-explored due to the absence of sufficient data resources and technical difficulty in parsing multiple instances in a single pass.
Ranked #6 on
Human Part Segmentation
on CIHP
Adaptive Temporal Encoding Network for Video Instance-level Human Parsing
Beyond the existing single-person and multiple-person human parsing tasks in static images, this paper makes the first attempt to investigate a more realistic video instance-level human parsing that simultaneously segments out each person instance and parses each instance into more fine-grained parts (e. g., head, leg, dress).
Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining
Single image rain streaks removal has recently witnessed substantial progress due to the development of deep convolutional neural networks.
Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding
In this work, we investigate simultaneously predicting categories of different levels in the hierarchy and integrating this structured correlation information into the deep neural network by developing a novel Hierarchical Semantic Embedding (HSE) framework.
Ranked #52 on
Fine-Grained Image Classification
on CUB-200-2011
Fine-Grained Image Classification
Fine-Grained Image Recognition
+1
Neural Task Planning with And-Or Graph Representations
This paper focuses on semantic task planning, i. e., predicting a sequence of actions toward accomplishing a specific task under a certain scene, which is a new problem in computer vision research.
Attentive Crowd Flow Machines
Traffic flow prediction is crucial for urban traffic management and public safety.
Monocular Depth Estimation with Affinity, Vertical Pooling, and Label Enhancement
While significant progress has been made in monocular depth estimation with Convolutional Neural Networks (CNNs) extracting absolute features, such as edges and textures, the depth constraint of neighboring pixels, namely relative features, has been mostly ignored by recent methods.
Generative Semantic Manipulation with Mask-Contrasting GAN
Despite the promising results on paired/unpaired image-to-image translation achieved by Generative Adversarial Networks (GANs), prior works often only transfer the low-level information (e. g. color or texture changes), but fail to manipulate high-level semantic meanings (e. g., geometric structure or content) of different object regions.
Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks
We consider the single image super-resolution problem in a more general case that the low-/high-resolution pairs and the down-sampling process are unavailable.
Interpretable Visual Question Answering by Reasoning on Dependency Trees
Collaborative reasoning for understanding image-question pairs is a very critical but underexplored topic in interpretable visual question answering systems.
Teaching to Teach by Structured Dark Knowledge
We evaluate T2T across different learners, teachers, and tasks, which significantly demonstrates that structured knowledge can be inherited by the teachers to further benefit learners' training.
PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report
This paper reviews the first challenge on efficient perceptual image enhancement with the focus on deploying deep learning models on smartphones.
Learning Deep Representations for Semantic Image Parsing: a Comprehensive Overview
Semantic image parsing, which refers to the process of decomposing images into semantic regions and constructing the structure representation of the input, has recently aroused widespread interest in the field of computer vision.
Hybrid Knowledge Routed Modules for Large-scale Object Detection
The dominant object detection approaches treat the recognition of each region separately and overlook crucial semantic correlations between objects in one scene.
Cross-Modal Attentional Context Learning for RGB-D Object Detection
In this paper, we address this problem by developing a Cross-Modal Attentional Context (CMAC) learning framework, which enables the full exploitation of the context information from both RGB and depth data.
Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation
Domain adaptation enables the learner to safely generalize into novel environments by mitigating domain shifts across distributions.
Ranked #7 on
Domain Adaptation
on ImageCLEF-DA
Kalman Normalization: Normalizing Internal Representations Across Network Layers
In this paper, we present a novel normalization method, called Kalman Normalization (KN), for improving and accelerating the training of DNNs, particularly under the context of micro-batches.
Symbolic Graph Reasoning Meets Convolutions
To cooperate with local convolutions, each SGR is constituted by three modules: a) a primal local-to-semantic voting module where the features of all symbolic nodes are generated by voting from local representations; b) a graph reasoning module propagates information over knowledge graph to achieve global semantic coherency; c) a dual semantic-to-local mapping module learns new associations of the evolved symbolic nodes with local representations, and accordingly enhances local features.
Ranked #81 on
Semantic Segmentation
on ADE20K val
FRAME Revisited: An Interpretation View Based on Particle Evolution
FRAME (Filters, Random fields, And Maximum Entropy) is an energy-based descriptive model that synthesizes visual realism by capturing mutual patterns from structural input signals.
Facial Landmark Machines: A Backbone-Branches Architecture with Progressive Representation Learning
In this paper, we propose a novel cascaded backbone-branches fully convolutional neural network~(BB-FCN) for rapidly and accurately localizing facial landmarks in unconstrained and cluttered settings.
NADPEx: An on-policy temporally consistent exploration method for deep reinforcement learning
Reinforcement learning agents need exploratory behaviors to escape from local optima.
SNAS: Stochastic Neural Architecture Search
In experiments on CIFAR-10, SNAS takes less epochs to find a cell architecture with state-of-the-art accuracy than non-differentiable evolution-based and reinforcement-learning-based NAS, which is also transferable to ImageNet.
Ranked #25 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-10
3D Human Pose Machines with Self-supervised Learning
Driven by recent computer vision and robotic applications, recovering 3D human poses has become increasingly important and attracted growing interests.
Ranked #263 on
3D Human Pose Estimation
on Human3.6M
End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis
Besides the challenges for conversational dialogue systems (e. g. topic transition coherency and question understanding), automatic medical diagnosis further poses more critical requirements for the dialogue rationality in the context of medical knowledge and symptom-disease relations.
Knowledge-Embedded Routing Network for Scene Graph Generation
More specifically, we show that the statistical correlations between objects appearing in images and their relationships, can be explicitly represented by a structured knowledge graph, and a routing mechanism is learned to propagate messages through the graph to explore their interactions.
Ranked #9 on
Scene Graph Generation
on Visual Genome
Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation
Recent studies have shown remarkable advances in 3D human pose estimation from monocular images, with the help of large-scale in-door 3D datasets and sophisticated network architectures.
Adaptively Connected Neural Networks
This paper presents a novel adaptively connected neural network (ACNet) to improve the traditional convolutional neural networks (CNNs) {in} two aspects.
Ranked #1 on
Document Classification
on Cora
Weakly Supervised Person Re-ID: Differentiable Graphical Learning and A New Benchmark
Learning a Re-ID model with bag-level annotation is called the weakly supervised Re-ID problem.
Ranked #2 on
Person Re-Identification
on SYSU-30k
Graphonomy: Universal Human Parsing via Graph Transfer Learning
By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity.
Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition
Specifically, by analyzing the symbiosis and mutual exclusion of AUs in various facial expressions, we organize the facial AUs in the form of structured knowledge-graph and integrate a Gated Graph Neural Network (GGNN) in a multi-scale CNN framework to propagate node information through the graph for generating enhanced AU representation.
Face Hallucination by Attentive Sequence Optimization with Reinforcement Learning
Face hallucination is a domain-specific super-resolution problem that aims to generate a high-resolution (HR) face image from a low-resolution~(LR) input.
Contextualized Spatial-Temporal Network for Taxi Origin-Destination Demand Prediction
Finally, a GCC module is applied to model the correlation between all regions by computing a global correlation feature as a weighted sum of all regional features, with the weights being calculated as the similarity between the corresponding region pairs.
Blending-target Domain Adaptation by Adversarial Meta-Adaptation Networks
(Unsupervised) Domain Adaptation (DA) seeks for classifying target instances when solely provided with source labeled and target unlabeled examples for training.
Ranked #3 on
Multi-target Domain Adaptation
on Office-Home
Multivariate-Information Adversarial Ensemble for Scalable Joint Distribution Matching
A broad range of cross-$m$-domain generation researches boil down to matching a joint distribution by deep generative models (DGMs).
Learning Compact Target-Oriented Feature Representations for Visual Tracking
Many state-of-the-art trackers usually resort to the pretrained convolutional neural network (CNN) model for correlation filtering, in which deep features could usually be redundant, noisy and less discriminative for some certain instances, and the tracking performance might thus be affected.
Semi-Supervised Video Salient Object Detection Using Pseudo-Labels
Specifically, we present an effective video saliency detector that consists of a spatial refinement network and a spatiotemporal module.
Ranked #1 on
Video Salient Object Detection
on VOS-T
(using extra training data)
Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
Recognizing multiple labels of images is a practical and challenging task, and significant progress has been made by searching semantic-aware regions and modeling label dependency.
Ranked #8 on
Multi-Label Classification
on PASCAL VOC 2007
Crowd Counting with Deep Structured Scale Integration Network
Automatic estimation of the number of people in unconstrained crowded scenes is a challenging task and one major difficulty stems from the huge scale variation of people.
Fashion Retrieval via Graph Reasoning Networks on a Similarity Pyramid
To address this issue, we propose a novel Graph Reasoning Network (GRNet) on a Similarity Pyramid, which learns similarities between a query and a gallery cloth by using both global and local representations in multiple scales.
Ranked #4 on
Image Retrieval
on DeepFashion - Consumer-to-shop
(Rank-1 metric)
Dynamic Spatial-Temporal Representation Learning for Traffic Flow Prediction
Specifically, the first ConvLSTM unit takes normal traffic flow features as input and generates a hidden state at each time-step, which is further fed into the connected convolutional layer for spatial attention map inference.
Explainable High-order Visual Question Reasoning: A New Benchmark and Knowledge-routed Network
Explanation and high-order reasoning capabilities are crucial for real-world visual question answering with diverse levels of inference complexity (e. g., what is the dog that is near the girl playing with?)
Meta R-CNN : Towards General Solver for Instance-level Few-shot Learning
Resembling the rapid learning capability of human, few-shot learning empowers vision systems to understand new concepts by training with few samples.
Ranked #19 on
Few-Shot Object Detection
on MS-COCO (30-shot)
Layout-Graph Reasoning for Fashion Landmark Detection
Each Layout-Graph Reasoning(LGR) layer aims to map feature representations into structural graph nodes via a Map-to-Node module, performs reasoning over structural graph nodes to achieve global layout coherency via a layout-graph reasoning module, and then maps graph nodes back to enhance feature representations via a Node-to-Map module.
A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models
To solve this problem, we derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation.
Knowledge Graph Transfer Network for Few-Shot Recognition
In this work, we represent the semantic correlations in the form of structured knowledge graph and integrate this graph into deep neural networks to promote few-shot learning by a novel Knowledge Graph Transfer Network (KGTN).
Blockwisely Supervised Neural Architecture Search with Knowledge Distillation
Moreover, we find that the knowledge of a network model lies not only in the network parameters but also in the network architecture.
Ranked #1 on
Neural Architecture Search
on CIFAR-100
An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation
In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations.
Physical-Virtual Collaboration Modeling for Intra-and Inter-Station Metro Ridership Prediction
To address this problem, we model a metro system as graphs with various topologies and propose a unified Physical-Virtual Collaboration Graph Network (PVCGN), which can effectively learn the complex ridership patterns from the tailor-designed graphs.
Tree-Structured Policy based Progressive Reinforcement Learning for Temporally Language Grounding in Video
Temporally language grounding in untrimmed videos is a newly-raised task in video understanding.
Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread
The proposed algorithm for the first time achieves competitive accuracy and high inference efficiency simultaneously with a single CPU thread.
DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution
Previous image SR methods fail to exhibit similar performance on Real-SR as the image data is not aligned inherently.
Towards Causality-Aware Inferring: A Sequential Discriminative Approach for Medical Diagnosis
To eliminate this bias and inspired by the propensity score matching technique with causal diagram, we propose a propensity-based patient simulator to effectively answer unrecorded inquiry by drawing knowledge from the other records; Bias (ii) inherently comes along with the passively collected data, and is one of the key obstacles for training the agent towards "learning how" rather than "remembering what".
Linguistically Driven Graph Capsule Network for Visual Question Reasoning
Inspired by the property of a capsule network that can carve a tree structure inside a regular convolutional neural network (CNN), we propose a hierarchical compositional reasoning model called the "Linguistically driven Graph Capsule Network", where the compositional process is guided by the linguistic parse tree.
Efficient Crowd Counting via Structured Knowledge Transfer
Crowd counting is an application-oriented task and its inference efficiency is crucial for real-world applications.
Transferable, Controllable, and Inconspicuous Adversarial Attacks on Person Re-identification With Deep Mis-Ranking
To examine the robustness of ReID systems is rather important because the insecurity of ReID systems may cause severe losses, e. g., the criminals may use the adversarial perturbations to cheat the CCTV systems.
Bidirectional Graph Reasoning Network for Panoptic Segmentation
We introduce a Bidirectional Graph Reasoning Network (BGRNet), which incorporates graph structure into the conventional panoptic segmentation network to mine the intra-modular and intermodular relations within and between foreground things and background stuff classes.
Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning
Channel pruning is a popular technique for compressing convolutional neural networks (CNNs), where various pruning criteria have been proposed to remove the redundant filters.
NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results
This paper reviews the NTIRE 2020 challenge on perceptual extreme super-resolution with focus on proposed solutions and results.
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
Many algorithms try to predict model performance of the pruned sub-nets by introducing various evaluation methods.
Ranked #6 on
Network Pruning
on ImageNet
Fine-Grained Image Captioning with Global-Local Discriminative Objective
This is primarily due to (i) the conservative characteristic of traditional training objectives that drives the model to generate correct but hardly discriminative captions for similar images and (ii) the uneven word distribution of the ground-truth captions, which encourages generating highly frequent words/phrases while suppressing the less frequent but more concrete ones.
Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition
However, most of these works focus on holistic feature adaptation, and they ignore local features that are more transferable across different datasets.
Cross-Domain Facial Expression Recognition
Facial Expression Recognition (FER)
Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning
Although each declares to achieve superior performance, fair comparisons are lacking due to the inconsistent choices of the source/target datasets and feature extractors.
Ranked #1 on
Cross-Domain Facial Expression Recognition
on Source: AFE, Target: CK+, JAFFE, SFEW2.0, FER2013, ExpW
Cross-Domain Facial Expression Recognition
Domain Adaptation
+3
Component Divide-and-Conquer for Real-World Image Super-Resolution
Learning an SR model with conventional pixel-wise loss usually is easily dominated by flat regions and edges, and fails to infer realistic details of complex textures.
Unsupervised Multi-view Clustering by Squeezing Hybrid Knowledge from Cross View and Each View
To ensure that we achieve effective sparse representation and clustering performance on the original data matrix, adaptive graph regularization and unsupervised clustering constraints are also incorporated in the proposed model to preserve the internal structural features of the data.
Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition
In this paper, we propose a novel framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) by adaptively transferring and distilling the knowledge from multiple wearable sensors.
Online Alternate Generator against Adversarial Attacks
The field of computer vision has witnessed phenomenal progress in recent years partially due to the development of deep convolutional neural networks.
Collaborative Training between Region Proposal Localization and Classification for Domain Adaptive Object Detection
Domain adaptation for object detection tries to adapt the detector from labeled datasets to unlabeled ones for better performance.
Reinforcement Learning for Weakly Supervised Temporal Grounding of Natural Language in Untrimmed Videos
Temporal grounding of natural language in untrimmed videos is a fundamental yet challenging multimedia task facilitating cross-media visual content retrieval.
Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition
The framework exploits prior knowledge to guide adaptive information propagation among different categories to facilitate multi-label analysis and reduce the dependency of training samples.
Few-Shot Learning
Multi-label Image Recognition with Partial Labels
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results
This paper introduces the real image Super-Resolution (SR) challenge that was part of the Advances in Image Manipulation (AIM) workshop, held in conjunction with ECCV 2020.
GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems
Capitalized on the topic-level dialogue graph, we propose a new evaluation metric GRADE, which stands for Graph-enhanced Representations for Automatic Dialogue Evaluation.
Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems
A practical automatic textual math word problems (MWPs) solver should be able to solve various textual MWPs while most existing works only focused on one-unknown linear MWPs.
Ranked #10 on
Math Word Problem Solving
on ALG514
A Hamiltonian Monte Carlo Method for Probabilistic Adversarial Attack and Learning
Although deep convolutional neural networks (CNNs) have demonstrated remarkable performance on multiple computer vision tasks, researches on adversarial learning have shown that deep models are vulnerable to adversarial examples, which are crafted by adding visually imperceptible perturbations to the input images.
Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation
In this work, we propose an efficient, cooperative and highly automated framework to simultaneously search for all main components including backbone, segmentation branches, and feature fusion module in a unified panoptic segmentation pipeline based on the prevailing one-shot Network Architecture Search (NAS) paradigm.
Continuous Transition: Improving Sample Efficiency for Continuous Control Problems via MixUp
Although deep reinforcement learning (RL) has been successfully applied to a variety of robotic control tasks, it's still challenging to apply it to real-world tasks, due to the poor sample efficiency.
Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting
Extensive experiments conducted on the RGBT-CC benchmark demonstrate the effectiveness of our framework for RGBT crowd counting.
Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding
Specifically, we generate the question-answer pair based on both the Visual Genome scene graph and an external knowledge base with controlled programs to disentangle the knowledge from other biases.
Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation
Besides, we develop a Graph-Evolving Meta-Learning (GEML) framework that learns to evolve the commonsense graph for reasoning disease-symptom correlations in a new disease, which effectively alleviates the needs of a large number of dialogues.
Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognition
When sampling tasks in MML-ASR, AMS adaptively determines the task sampling probability for each source language.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement
It is crucial since the quality of the evidence is the key to answering commonsense questions, and even determines the upper bound on the QA systems performance.
AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition
In this work, we explore the correlations among the action units and facial expressions, and devise an AU-Expression Knowledge Constrained Representation Learning (AUE-CRL) framework to learn the AU representations without AU annotations and adaptively use representations to facilitate facial expression recognition.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
CAT-SAC: Soft Actor-Critic with Curiosity-Aware Entropy Temperature
The curiosity is added to the target entropy to increase the entropy temperature for unfamiliar states and decrease the target entropy for familiar states.
Linguistically Routing Capsule Network for Out-of-Distribution Visual Question Answering
The experimental results show that our proposed method can improve current VQA models on OOD split without losing performance on the in-domain test data.
Erasure for Advancing: Dynamic Self-Supervised Learning for Commonsense Reasoning
To solve this issue, we propose a novel DynamIc Self-sUperviSed Erasure (DISUSE) which adaptively erases redundant and artifactual clues in the context and questions to learn and establish the correct corresponding pair relations between the questions and their clues.
Towards a Reliable and Robust Dialogue System for Medical Automatic Diagnosis
To this end, we propose a novel DSMAD agent, INS-DS (Introspective Diagnosis System) comprising of two separate yet cooperative modules, i. e., an inquiry module for proposing symptom-inquiries and an introspective module for deciding when to inform a disease.
Adversarial Training using Contrastive Divergence
To protect the security of machine learning models against adversarial examples, adversarial training becomes the most popular and powerful strategy against various adversarial attacks by injecting adversarial examples into training data.
Temporal Contrastive Graph Learning for Video Action Recognition and Retrieval
To model multi-scale temporal dependencies, our TCGL integrates the prior knowledge about the frame and snippet orders into graph structures, i. e., the intra-/inter- snippet temporal contrastive graphs.
Unifying Relational Sentence Generation and Retrieval for Medical Image Report Composition
Beyond generating long and topic-coherent paragraphs in traditional captioning tasks, the medical image report composition task poses more task-oriented challenges by requiring both the highly-accurate medical term diagnosis and multiple heterogeneous forms of information including impression and findings.
Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer
Prior highly-tuned image parsing models are usually studied in a certain domain with a specific set of semantic labels and can hardly be adapted into other scenarios (e. g., sharing discrepant label granularity) without extensive re-training.
Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition
In this work, we prove that dynamically adapting network architectures tailored for each domain task along with weight finetuning benefits in both efficiency and effectiveness, compared to the existing image recognition pipeline that only tunes the weights regardless of the architecture.
Solving Inefficiency of Self-supervised Representation Learning
In this paper, we reveal two contradictory phenomena in contrastive learning that we call under-clustering and over-clustering problems, which are major obstacles to learning efficiency.
Ranked #1 on
Self-Supervised Person Re-Identification
on SYSU-30k
GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning
Therefore, we propose a Geometric Question Answering dataset GeoQA, containing 4, 998 geometric problems with corresponding annotated programs, which illustrate the solving process of the given problems.
Ranked #4 on
Mathematical Reasoning
on PGPS9K
Towards Quantifiable Dialogue Coherence Evaluation
To address these limitations, we propose Quantifiable Dialogue Coherence Evaluation (QuantiDCE), a novel framework aiming to train a quantifiable dialogue coherence metric that can reflect the actual human rating standards.
Prototypical Graph Contrastive Learning
However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i. e., the negatives likely having the same semantic structure with the query, leading to performance degradation.
Online Metro Origin-Destination Prediction via Heterogeneous Information Aggregation
In this work, we proposed a novel neural network module termed Heterogeneous Information Aggregation Machine (HIAM), which fully exploits heterogeneous information of historical data (e. g., incomplete OD matrices, unfinished order vectors, and DO matrices) to jointly learn the evolutionary patterns of OD and DO ridership.
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks
Previous math word problem solvers following the encoder-decoder paradigm fail to explicitly incorporate essential math symbolic constraints, leading to unexplainable and unreasonable predictions.
Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
Specifically, we propose a Dynamic Reinforced Instruction Attacker (DR-Attacker), which learns to mislead the navigator to move to the wrong target by destroying the most instructive information in instructions at different timesteps.
Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video
In this paper, we introduce a novel task, referred to as Weakly-Supervised Spatio-Temporal Anomaly Detection (WSSTAD) in surveillance video.
Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learning
Open-set semi-supervised learning (open-set SSL) investigates a challenging but practical scenario where out-of-distribution (OOD) samples are contained in the unlabeled data.
Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift
We attribute this ranking correlation problem to the supernet training consistency shift, including feature shift and parameter shift.
Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition
Specifically, we unify a pre-trained acoustic model (wav2vec 2. 0) and a language model (BERT) into an end-to-end trainable framework.
