Search Results for author:
Found 285 papers, 114 papers with code
Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation
Domain adaptation enables the learner to safely generalize into novel environments by mitigating domain shifts across distributions.
Ranked #6 on
Domain Adaptation
on ImageCLEF-DA
Joint Detection and Identification Feature Learning for Person Search
Existing person re-identification benchmarks and methods mainly focus on matching cropped pedestrian images between queries and candidates.
Ranked #9 on
Person Re-Identification
on CUHK03
Identity-Preserving Talking Face Generation with Landmark and Appearance Priors
Prior landmark characteristics of the speaker's face are employed to make the generated landmarks coincide with the facial outline of the speaker.
Toward Characteristic-Preserving Image-based Virtual Try-On Network
Second, to alleviate boundary artifacts of warped clothes and make the results more realistic, we employ a Try-On Module that learns a composition mask to integrate the warped clothes and the rendered image to ensure smoothness.
3D Human Pose Machines with Self-supervised Learning
Driven by recent computer vision and robotic applications, recovering 3D human poses has become increasingly important and attracted growing interests.
Ranked #263 on
3D Human Pose Estimation
on Human3.6M
Instance-level Human Parsing via Part Grouping Network
Instance-level human parsing towards real-world human analysis scenarios is still under-explored due to the absence of sufficient data resources and technical difficulty in parsing multiple instances in a single pass.
Ranked #6 on
Human Part Segmentation
on CIHP
Look into Person: Joint Body Parsing & Pose Estimation Network and A New Benchmark
To further explore and take advantage of the semantic correlation of these two tasks, we propose a novel joint human parsing and pose estimation network to explore efficient context modeling, which can simultaneously predict parsing and pose with extremely high quality.
Ranked #10 on
Semantic Segmentation
on LIP val
Control-A-Video: Controllable Text-to-Video Generation with Diffusion Models
Based on a pre-trained conditional text-to-image (T2I) diffusion model, our model aims to generate videos conditioned on a sequence of control signals, such as edge or depth maps.
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
Many algorithms try to predict model performance of the pruned sub-nets by introducing various evaluation methods.
Ranked #5 on
Network Pruning
on ImageNet
Graphonomy: Universal Human Parsing via Graph Transfer Learning
By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity.
Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer
Prior highly-tuned image parsing models are usually studied in a certain domain with a specific set of semantic labels and can hardly be adapted into other scenarios (e. g., sharing discrepant label granularity) without extensive re-training.
Single View Stereo Matching
The resulting model outperforms all the previous monocular depth estimation methods as well as the stereo block matching method in the challenging KITTI dataset by only using a small number of real training data.
Ranked #42 on
Monocular Depth Estimation
on KITTI Eigen split
LSTM Pose Machines
Such suboptimal results are mainly attributed to the inability of imposing sequential geometric consistency, handling severe image quality degradation (e. g. motion blur and occlusion) as well as the inability of capturing the temporal correlation among video frames.
Ranked #3 on
Pose Estimation
on J-HMDB
Blockwisely Supervised Neural Architecture Search with Knowledge Distillation
Moreover, we find that the knowledge of a network model lies not only in the network parameters but also in the network architecture.
Ranked #1 on
Neural Architecture Search
on CIFAR-100
DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions
Addressing this problem, we modularize a large search space into blocks with small search spaces and develop a family of models with the distilling neural architecture (DNA) techniques.
Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing
Human parsing has recently attracted a lot of research interests due to its huge application potentials.
Ranked #13 on
Semantic Segmentation
on LIP val
Multi-level Wavelet-CNN for Image Restoration
With the modified U-Net architecture, wavelet transform is introduced to reduce the size of feature maps in the contracting subnetwork.
Ranked #2 on
Grayscale Image Denoising
on Set12 sigma25
Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
We investigate a novel approach for image restoration by reinforcement learning.
Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation
To this end, we study LLMs on the popular Oogiri game which needs participants to have good creativity and strong associative thinking for responding unexpectedly and humorously to the given image, text, or both, and thus is suitable for LoT study.
Component Divide-and-Conquer for Real-World Image Super-Resolution
Learning an SR model with conventional pixel-wise loss usually is easily dominated by flat regions and edges, and fails to infer realistic details of complex textures.
Weakly Supervised Person Re-ID: Differentiable Graphical Learning and A New Benchmark
Learning a Re-ID model with bag-level annotation is called the weakly supervised Re-ID problem.
Ranked #2 on
Person Re-Identification
on SYSU-30k
Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
Recognizing multiple labels of images is a practical and challenging task, and significant progress has been made by searching semantic-aware regions and modeling label dependency.
Ranked #8 on
Multi-Label Classification
on PASCAL VOC 2007
Deep Human Parsing with Active Template Regression
The first CNN network is with max-pooling, and designed to predict the template coefficients for each label mask, while the second CNN network is without max-pooling to preserve sensitivity to label mask position and accurately predict the active shape parameters.
SNAS: Stochastic Neural Architecture Search
In experiments on CIFAR-10, SNAS takes less epochs to find a cell architecture with state-of-the-art accuracy than non-differentiable evolution-based and reinforcement-learning-based NAS, which is also transferable to ImageNet.
Ranked #25 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-10
Adaptively Connected Neural Networks
This paper presents a novel adaptively connected neural network (ACNet) to improve the traditional convolutional neural networks (CNNs) {in} two aspects.
Ranked #1 on
Document Classification
on Cora
Knowledge-Embedded Routing Network for Scene Graph Generation
More specifically, we show that the statistical correlations between objects appearing in images and their relationships, can be explicitly represented by a structured knowledge graph, and a routing mechanism is learned to propagate messages through the graph to explore their interactions.
Ranked #9 on
Scene Graph Generation
on Visual Genome
Learning Warped Guidance for Blind Face Restoration
For better recovery of fine facial details, we modify the problem setting by taking both the degraded observation and a high-quality guided image of the same identity as input to our guided face restoration network (GFRNet).
Ranked #1 on
Image Super-Resolution
on WebFace - 8x upscaling
Cross-Modal Causal Intervention for Medical Report Generation
Medical report generation (MRG) is essential for computer-aided diagnosis and medication guidance, which can relieve the heavy burden of radiologists by automatically generating the corresponding medical reports according to the given radiology image.
Visual Causal Scene Refinement for Video Question Answering
Our VCSR involves two essential modules: i) the Question-Guided Refiner (QGR) module, which refines consecutive video frames guided by the question semantics to obtain more representative segment features for causal front-door intervention; ii) the Causal Scene Separator (CSS) module, which discovers a collection of visual causal and non-causal scenes based on the visual-linguistic causal relevance and estimates the causal effect of the scene-separating intervention in a contrastive learning manner.
CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal Reasoning
We present CausalVLR (Causal Visual-Linguistic Reasoning), an open-source toolbox containing a rich set of state-of-the-art causal relation discovery and causal inference methods for various visual-linguistic reasoning tasks, such as VQA, image/video captioning, medical report generation, model generalization and robustness, etc.
Towards CausalGPT: A Multi-Agent Approach for Faithful Knowledge Reasoning via Promoting Causal Consistency in LLMs
Despite advancements in LLMs, knowledge-based reasoning remains a longstanding issue due to the fragility of knowledge recall and inference.
DreamEditor: Text-Driven 3D Scene Editing with Neural Fields
Neural fields have achieved impressive advancements in view synthesis and scene reconstruction.
Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning
Although each declares to achieve superior performance, fair comparisons are lacking due to the inconsistent choices of the source/target datasets and feature extractors.
Ranked #1 on
Cross-Domain Facial Expression Recognition
on Source: AFE, Target: CK+, JAFFE, SFEW2.0, FER2013, ExpW
Cross-Domain Facial Expression Recognition
Domain Adaptation
+3
Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition
However, most of these works focus on holistic feature adaptation, and they ignore local features that are more transferable across different datasets.
Cross-Domain Facial Expression Recognition
Facial Expression Recognition (FER)
SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models
Our approach can make text-to-image diffusion models easier to use with better user experience, which demonstrates our approach has the potential for further advancing the development of user-friendly text-to-image generation models by bridging the semantic gap between simple narrative prompts and complex keyword-based prompts.
Hybrid Knowledge Routed Modules for Large-scale Object Detection
The dominant object detection approaches treat the recognition of each region separately and overlook crucial semantic correlations between objects in one scene.
Transferable, Controllable, and Inconspicuous Adversarial Attacks on Person Re-identification With Deep Mis-Ranking
To examine the robustness of ReID systems is rather important because the insecurity of ReID systems may cause severe losses, e. g., the criminals may use the adversarial perturbations to cheat the CCTV systems.
Towards Real-World Burst Image Super-Resolution: Benchmark and Method
Despite substantial advances, single-image super-resolution (SISR) is always in a dilemma to reconstruct high-quality images with limited information from one input image, especially in realistic scenarios.
SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training
Moreover, these methods ignore how to utilize the fine-grained dependencies among different skeleton joints to pre-train an efficient skeleton sequence learning model that can generalize well across different datasets.
Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding
In this work, we investigate simultaneously predicting categories of different levels in the hierarchy and integrating this structured correlation information into the deep neural network by developing a novel Hierarchical Semantic Embedding (HSE) framework.
Ranked #51 on
Fine-Grained Image Classification
on CUB-200-2011
Fine-Grained Image Classification
Fine-Grained Image Recognition
+1
Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding
Specifically, we generate the question-answer pair based on both the Visual Genome scene graph and an external knowledge base with controlled programs to disentangle the knowledge from other biases.
Adaptive Temporal Encoding Network for Video Instance-level Human Parsing
Beyond the existing single-person and multiple-person human parsing tasks in static images, this paper makes the first attempt to investigate a more realistic video instance-level human parsing that simultaneously segments out each person instance and parses each instance into more fine-grained parts (e. g., head, leg, dress).
Semi-Supervised Video Salient Object Detection Using Pseudo-Labels
Specifically, we present an effective video saliency detector that consists of a spatial refinement network and a spatiotemporal module.
Ranked #1 on
Video Salient Object Detection
on VOS-T
(using extra training data)
Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering
Existing visual question answering methods often suffer from cross-modal spurious correlations and oversimplified event-level reasoning processes that fail to capture event temporality, causality, and dynamics spanning over the video.
Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networks
In this work, aiming at a general and comprehensive way for neural network acceleration, we develop a Wavelet-like Auto-Encoder (WAE) that decomposes the original input image into two low-resolution channels (sub-images) and incorporate the WAE into the classification neural networks for joint training.
Towards Quantifiable Dialogue Coherence Evaluation
To address these limitations, we propose Quantifiable Dialogue Coherence Evaluation (QuantiDCE), a novel framework aiming to train a quantifiable dialogue coherence metric that can reflect the actual human rating standards.
End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis
Besides the challenges for conversational dialogue systems (e. g. topic transition coherency and question understanding), automatic medical diagnosis further poses more critical requirements for the dialogue rationality in the context of medical knowledge and symptom-disease relations.
GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems
Capitalized on the topic-level dialogue graph, we propose a new evaluation metric GRADE, which stands for Graph-enhanced Representations for Automatic Dialogue Evaluation.
Dynamic Spatial-Temporal Representation Learning for Traffic Flow Prediction
Specifically, the first ConvLSTM unit takes normal traffic flow features as input and generates a hidden state at each time-step, which is further fed into the connected convolutional layer for spatial attention map inference.
Adversarially-Aware Robust Object Detector
In this work, we empirically explore the model training for adversarial robustness in object detection, which greatly attributes to the conflict between learning clean images and adversarial images.
OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup
Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes), which has attracted increasing attention from the multimedia and computer vision communities.
Real-World Image Super-Resolution by Exclusionary Dual-Learning
Real-world image super-resolution is a practical image restoration problem that aims to obtain high-quality images from in-the-wild input, has recently received considerable attention with regard to its tremendous application potentials.
Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting
Extensive experiments conducted on the RGBT-CC benchmark demonstrate the effectiveness of our framework for RGBT crowd counting.
Deep Reasoning with Knowledge Graph for Social Relationship Understanding
And this structured knowledge can be efficiently integrated into the deep neural network architecture to promote social relationship understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which a propagation mechanism is learned to propagate node message through the graph to explore the interaction between persons of interest and the contextual objects.
Ranked #2 on
Visual Social Relationship Recognition
on PIPA
Physical-Virtual Collaboration Modeling for Intra-and Inter-Station Metro Ridership Prediction
To address this problem, we model a metro system as graphs with various topologies and propose a unified Physical-Virtual Collaboration Graph Network (PVCGN), which can effectively learn the complex ridership patterns from the tailor-designed graphs.
Efficient Crowd Counting via Structured Knowledge Transfer
Crowd counting is an application-oriented task and its inference efficiency is crucial for real-world applications.
ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection
Besides, we also observe the theoretical benefits of the LSC coefficient scaling of UNet in the stableness of hidden features and gradient and also robustness.
Tree-Structured Policy based Progressive Reinforcement Learning for Temporally Language Grounding in Video
Temporally language grounding in untrimmed videos is a newly-raised task in video understanding.
Blending-target Domain Adaptation by Adversarial Meta-Adaptation Networks
(Unsupervised) Domain Adaptation (DA) seeks for classifying target instances when solely provided with source labeled and target unlabeled examples for training.
Ranked #3 on
Multi-target Domain Adaptation
on Office-Home
GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning
Therefore, we propose a Geometric Question Answering dataset GeoQA, containing 4, 998 geometric problems with corresponding annotated programs, which illustrate the solving process of the given problems.
Ranked #4 on
Mathematical Reasoning
on PGPS9K
Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning
We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source-pretrained model is adapted to the target domain without access to the source data.
Ranked #4 on
Source-Free Domain Adaptation
on VisDA-2017
Solving Inefficiency of Self-supervised Representation Learning
In this paper, we reveal two contradictory phenomena in contrastive learning that we call under-clustering and over-clustering problems, which are major obstacles to learning efficiency.
Ranked #1 on
Self-Supervised Person Re-Identification
on SYSU-30k
Cost-Effective Active Learning for Deep Image Classification
In this paper, we propose a novel active learning framework, which is capable of building a competitive classifier with optimal feature representation via a limited amount of labeled training instances in an incremental learning manner.
Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation
Besides, we develop a Graph-Evolving Meta-Learning (GEML) framework that learns to evolve the commonsense graph for reasoning disease-symptom correlations in a new disease, which effectively alleviates the needs of a large number of dialogues.
Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift
Motivated by the theoretical results in \cite{mansour2009domain}, the target distribution can be represented as the weighted combination of source distributions, and, the multi-source unsupervised domain adaptation via DCTN is then performed as two alternating steps: i) It deploys multi-way adversarial learning to minimize the discrepancy between the target and each of the multiple source domains, which also obtains the source-specific perplexity scores to denote the possibilities that a target sample belongs to different source domains.
Multi-Source Unsupervised Domain Adaptation
Unsupervised Domain Adaptation
Structured Semantic Transfer for Multi-Label Recognition with Partial Labels
To reduce the annotation cost, we propose a structured semantic transfer (SST) framework that enables training multi-label recognition models with partial labels, i. e., merely some labels are known while other labels are missing (also called unknown labels) per image.
Unsupervised Domain Adaptive Salient Object Detection Through Uncertainty-Aware Pseudo-Label Learning
To relieve the burden of labor-intensive labeling, deep unsupervised SOD methods have been proposed to exploit noisy labels generated by handcrafted saliency methods.
Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels
However, these algorithms depend on sufficient multi-label annotations to train the models, leading to poor performance especially with low known label proportion.
Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels
Multi-label image recognition with partial labels (MLR-PL), in which some labels are known while others are unknown for each image, may greatly reduce the cost of annotation and thus facilitate large-scale MLR.
Dual-Perspective Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels
Specifically, an instance-perspective representation blending (IPRB) module is designed to blend the representations of the known labels in an image with the representations of the corresponding unknown labels in another image to complement these unknown labels.
Image Classification
Multi-label Image Recognition with Partial Labels
DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution
Previous image SR methods fail to exhibit similar performance on Real-SR as the image data is not aligned inherently.
TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning
To overcome these limitations, we take advantage of the multi-scale temporal dependencies within videos and proposes a novel video self-supervised learning framework named Temporal Contrastive Graph Learning (TCGL), which jointly models the inter-snippet and intra-snippet temporal dependencies for temporal representation learning with a hybrid graph contrastive learning strategy.
Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks
We consider the single image super-resolution problem in a more general case that the low-/high-resolution pairs and the down-sampling process are unavailable.
Zoom and Learn: Generalizing Deep Stereo Matching to Novel Domains
By feeding real stereo pairs of different domains to stereo models pre-trained with synthetic data, we see that: i) a pre-trained model does not generalize well to the new domain, producing artifacts at boundaries and ill-posed regions; however, ii) feeding an up-sampled stereo pair leads to a disparity map with extra details.
Prototypical Graph Contrastive Learning
However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i. e., the negatives likely having the same semantic structure with the query, leading to performance degradation.
Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition
In this paper, we propose a novel framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) by adaptively transferring and distilling the knowledge from multiple wearable sensors.
Collaborative Training between Region Proposal Localization and Classification for Domain Adaptive Object Detection
Domain adaptation for object detection tries to adapt the detector from labeled datasets to unlabeled ones for better performance.
Symbolic Graph Reasoning Meets Convolutions
To cooperate with local convolutions, each SGR is constituted by three modules: a) a primal local-to-semantic voting module where the features of all symbolic nodes are generated by voting from local representations; b) a graph reasoning module propagates information over knowledge graph to achieve global semantic coherency; c) a dual semantic-to-local mapping module learns new associations of the evolved symbolic nodes with local representations, and accordingly enhances local features.
Ranked #81 on
Semantic Segmentation
on ADE20K val
Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation
In this work, we propose an efficient, cooperative and highly automated framework to simultaneously search for all main components including backbone, segmentation branches, and feature fusion module in a unified panoptic segmentation pipeline based on the prevailing one-shot Network Architecture Search (NAS) paradigm.
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks
Previous math word problem solvers following the encoder-decoder paradigm fail to explicitly incorporate essential math symbolic constraints, leading to unexplainable and unreasonable predictions.
Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift
We attribute this ranking correlation problem to the supernet training consistency shift, including feature shift and parameter shift.
On Fast Simulation of Dynamical System with Neural Vector Enhanced Numerical Solver
The large-scale simulation of dynamical systems is critical in numerous scientific and engineering disciplines.
UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression
Naturally, we also present a unified multi-task Geometric Transformer framework, Geoformer, to tackle calculation and proving problems simultaneously in the form of sequence generation, which finally shows the reasoning ability can be improved on both two tasks by unifying formulation.
Ranked #3 on
Mathematical Reasoning
on PGPS9K
Dual Adversarial Adaptation for Cross-Device Real-World Image Super-Resolution
To address this issue, we propose an unsupervised domain adaptation mechanism for real-world SR, named Dual ADversarial Adaptation (DADA), which only requires LR images in the target domain with available real paired data from a source camera.
LSTM-CF: Unifying Context Modeling and Fusion with LSTMs for RGB-D Scene Labeling
Another long short-term memorized fusion layer is set up to integrate the contexts along the vertical direction from different channels, and perform bi-directional propagation of the fused vertical contexts along the horizontal direction to obtain true 2D global contexts.
Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems
A practical automatic textual math word problems (MWPs) solver should be able to solve various textual MWPs while most existing works only focused on one-unknown linear MWPs.
Ranked #10 on
Math Word Problem Solving
on ALG514
SIRST-5K: Exploring Massive Negatives Synthesis with Self-supervised Learning for Robust Infrared Small Target Detection
The quality, quantity, and diversity of the infrared dataset are critical to the detection of small targets.
Recognizing Focal Liver Lesions in Contrast-Enhanced Ultrasound with Discriminatively Trained Spatio-Temporal Model
The aim of this study is to provide an automatic computational framework to assist clinicians in diagnosing Focal Liver Lesions (FLLs) in Contrast-Enhancement Ultrasound (CEUS).
AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition
In this work, we explore the correlations among the action units and facial expressions, and devise an AU-Expression Knowledge Constrained Representation Learning (AUE-CRL) framework to learn the AU representations without AU annotations and adaptively use representations to facilitate facial expression recognition.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Online Metro Origin-Destination Prediction via Heterogeneous Information Aggregation
In this work, we proposed a novel neural network module termed Heterogeneous Information Aggregation Machine (HIAM), which fully exploits heterogeneous information of historical data (e. g., incomplete OD matrices, unfinished order vectors, and DO matrices) to jointly learn the evolutionary patterns of OD and DO ridership.
Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Training
During inference, instead of changing the motion generator, our method reformulates the input text into a masked motion as the prompt for the motion generator to ``reconstruct'' the motion.
AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis
Through extensive experiments across various datasets and scenes, we demonstrate the effectiveness of our approach in facilitating better interaction between LiDAR and camera modalities within a unified neural field.
Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
Specifically, we propose a Dynamic Reinforced Instruction Attacker (DR-Attacker), which learns to mislead the navigator to move to the wrong target by destroying the most instructive information in instructions at different timesteps.
Robust Real-World Image Super-Resolution against Adversarial Attacks
Since the frequency masking may not only destroys the adversarial perturbations but also affects the sharp details in a clean image, we further develop an adversarial sample classifier based on the frequency domain of images to determine if applying the proposed mask module.
Masked Images Are Counterfactual Samples for Robust Fine-tuning
In this paper, based on causal analysis of the aforementioned problems, we propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
Knowledge Graph Transfer Network for Few-Shot Recognition
In this work, we represent the semantic correlations in the form of structured knowledge graph and integrate this graph into deep neural networks to promote few-shot learning by a novel Knowledge Graph Transfer Network (KGTN).
ADASR: An Adversarial Auto-Augmentation Framework for Hyperspectral and Multispectral Data Fusion
Deep learning-based hyperspectral image (HSI) super-resolution, which aims to generate high spatial resolution HSI (HR-HSI) by fusing hyperspectral image (HSI) and multispectral image (MSI) with deep neural networks (DNNs), has attracted lots of attention.
Cost-effective Object Detection: Active Sample Mining with Switchable Selection Criteria
The proposed process can be compatible with mini-batch based training (i. e., using a batch of unlabeled or partially labeled data as a one-time input) for object detection.
Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism
To make ROSETTA automatically determine which experience is available and useful, a prototypical task correlation guided Gating Diversity Controller(GDC) is introduced to adaptively adjust the diversity of gates for the new task based on class-specific prototypes.
Spatial-Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation
In this work, we propose a spatial-temporal knowledge-embedded transformer (STKET) that incorporates the prior spatial-temporal knowledge into the multi-head cross-attention mechanism to learn more representative relationship representations.
LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning
To address this issue and make a step towards interpretable MWP solving, we first construct a high-quality MWP dataset named InterMWP which consists of 11, 495 MWPs and annotates interpretable logical formulas based on algebraic knowledge as the grounded linguistic logic of each solution equation.
Semantic-Aware Auto-Encoders for Self-Supervised Representation Learning
Inspired by perceptual learning that could use cross-view learning to perceive concepts and semantics, we propose a novel AE that could learn semantic-aware representation via cross-view image reconstruction.
Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima
This strategy can implicitly enhance the model's robustness during the optimization process, mitigating instability risks arising from multimodal information inputs.
Multivariate-Information Adversarial Ensemble for Scalable Joint Distribution Matching
A broad range of cross-$m$-domain generation researches boil down to matching a joint distribution by deep generative models (DGMs).
Fine-Grained Image Captioning with Global-Local Discriminative Objective
This is primarily due to (i) the conservative characteristic of traditional training objectives that drives the model to generate correct but hardly discriminative captions for similar images and (ii) the uneven word distribution of the ground-truth captions, which encourages generating highly frequent words/phrases while suppressing the less frequent but more concrete ones.
Continuous Transition: Improving Sample Efficiency for Continuous Control Problems via MixUp
Although deep reinforcement learning (RL) has been successfully applied to a variety of robotic control tasks, it's still challenging to apply it to real-world tasks, due to the poor sample efficiency.
NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes
Specifically, we introduce cooperative learning between visible and infrared images captured by different sensors.
Contrastive Transformer Learning with Proximity Data Generation for Text-Based Person Search
Moreover, we propose a proximity data generation (PDG) module to automatically produce more diverse data for cross-modal training.
Adaptive Global-Local Representation Learning and Selection for Cross-Domain Facial Expression Recognition
Specifically, the framework consists of separate global-local adversarial learning modules that learn domain-invariant global and local features independently.
Cross-Domain Facial Expression Recognition
Model Optimization
+2
IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-sensing Images
IDF-CR consists of a pixel space cloud removal module (Pixel-CR) and a latent space iterative noise diffusion network (IND).
Road Network Guided Fine-Grained Urban Traffic Flow Inference
Furthermore, we take the road network feature as a query to capture the long-range spatial distribution of traffic flow with a transformer architecture.
DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback
1) The travel time of a vehicle is delayed feedback on the effectiveness of TSC policy at each traffic intersection since it is obtained after the vehicle has left the road network.
Towards Human-Machine Cooperation: Self-supervised Sample Mining for Object Detection
Though quite challenging, leveraging large-scale unlabeled or partially labeled images in a cost-effective way has increasingly attracted interests for its great importance to computer vision.
DRPose3D: Depth Ranking in 3D Human Pose Estimation
In this paper, we propose a two-stage depth ranking based method (DRPose3D) to tackle the problem of 3D human pose estimation.
Visual Tracking via Dynamic Graph Learning
First, the graph is initialized by assigning binary weights of some image patches to indicate the object and background patches according to the predicted bounding box.
Visual Question Reasoning on General Dependency Tree
This network comprises of two collaborative modules: i) an adversarial attention module to exploit the local visual evidence for each word parsed from the question; ii) a residual composition module to compose the previously mined evidence.
Weakly Supervised Salient Object Detection Using Image Labels
Our algorithm is based on alternately exploiting a graphical model and training a fully convolutional network for model updating.
Deep Structured Scene Parsing by Learning with Image Descriptions
This paper addresses a fundamental problem of scene understanding: How to parse the scene image into a structured configuration (i. e., a semantic object hierarchy with object interaction relations) that finely accords with human perception.
Learning to Segment Human by Watching YouTube
An intuition on human segmentation is that when a human is moving in a video, the video-context (e. g., appearance and motion clues) may potentially infer reasonable mask information for the whole human body.
Batch Kalman Normalization: Towards Training Deep Neural Networks with Micro-Batches
As an indispensable component, Batch Normalization (BN) has successfully improved the training of deep neural networks (DNNs) with mini-batches, by normalizing the distribution of the internal representation for each hidden layer.
Hierarchical Scene Parsing by Weakly Supervised Learning with Image Descriptions
Rather than relying on elaborative annotations (e. g., manually labeled semantic maps and relations), we train our deep model in a weakly-supervised learning manner by leveraging the descriptive sentences of the training images.
Structured Inhomogeneous Density Map Learning for Crowd Counting
In this paper, we aim at tackling the problem of crowd counting in extremely high-density scenes, which contain hundreds, or even thousands of people.
Context-Aware Semantic Inpainting
Our proposed GAN-based framework consists of a fully convolutional design for the generator which helps to better preserve spatial structures and a joint loss function with a revised perceptual loss to capture high-level semantics in the context.
Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition
Recognizing multiple labels of images is a fundamental but challenging task in computer vision, and remarkable progress has been attained by localizing semantic-aware image regions and predicting their labels with deep convolutional neural networks.
Multi-label Image Recognition by Recurrently Discovering Attentional Regions
This paper proposes a novel deep architecture to address multi-label image recognition, a fundamental and practical task towards general visual understanding.
Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning
Sketch portrait generation benefits a wide range of applications such as digital entertainment and law enforcement.
Attention-Aware Face Hallucination via Deep Reinforcement Learning
Face hallucination is a domain-specific super-resolution problem with the goal to generate high-resolution (HR) faces from low-resolution (LR) input images.
Recurrent 3D Pose Sequence Machines
3D human articulated pose recovery from monocular image sequences is very challenging due to the diverse appearances, viewpoints, occlusions, and also the human 3D pose is inherently ambiguous from the monocular imagery.
Ranked #20 on
3D Human Pose Estimation
on HumanEva-I
Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning
Aiming at improving performance of visual classification in a cost-effective manner, this paper proposes an incremental semi-supervised learning paradigm called Deep Co-Space (DCS).
Structure-Preserving Image Super-resolution via Contextualized Multi-task Learning
Single image super resolution (SR), which refers to reconstruct a higher-resolution (HR) image from the observed low-resolution (LR) image, has received substantial attention due to its tremendous application potentials.
Knowledge-Guided Recurrent Neural Network Learning for Task-Oriented Action Prediction
This paper aims at task-oriented action prediction, i. e., predicting a sequence of actions towards accomplishing a specific task under a certain scene, which is a new problem in computer vision research.
Active Self-Paced Learning for Cost-Effective and Progressive Face Identification
By naturally combining two recently rising techniques: active learning (AL) and self-paced learning (SPL), our framework is capable of automatically annotating new instances and incorporating them into training under weak expert re-certification.
Instance-Level Salient Object Segmentation
Image saliency detection has recently witnessed rapid progress due to deep convolutional neural networks.
Ranked #15 on
RGB Salient Object Detection
on DUTS-TE
(max F-measure metric)
Interpretable Structure-Evolving LSTM
Instead of learning LSTM models over the pre-fixed structures, we propose to further learn the intermediate interpretable multi-level graph structures in a progressive and stochastic way from data during the LSTM network optimization.
Progressively Diffused Networks for Semantic Image Segmentation
This paper introduces Progressively Diffused Networks (PDNs) for unifying multi-scale context modeling with deep feature learning, by taking semantic image segmentation as an exemplar application.
Learning to Segment Object Candidates via Recursive Neural Networks
To avoid the exhaustive search over locations and scales, current state-of-the-art object detection systems usually involve a crucial component generating a batch of candidate object proposals from images.
Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning
In this paper, we propose a novel inference-embedded multi-task learning framework for predicting human pose from still depth images, which is implemented with a deep architecture of neural networks.
Is Faster R-CNN Doing Well for Pedestrian Detection?
Detecting pedestrian has been arguably addressed as a special topic beyond general object detection.
Ranked #19 on
Pedestrian Detection
on Caltech
Local- and Holistic- Structure Preserving Image Super Resolution via Deep Joint Component Learning
Recently, machine learning based single image super resolution (SR) approaches focus on jointly learning representations for high-resolution (HR) and low-resolution (LR) image patch pairs to improve the quality of the super-resolved images.
Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning
Cross-domain visual data matching is one of the fundamental problems in many real-world vision tasks, e. g., matching persons across ID photos and surveillance videos.
DARI: Distance metric And Representation Integration for Person Verification
The past decade has witnessed the rapid development of feature representation learning and distance metric learning, whereas the two steps are often discussed separately.
Ranked #7 on
Person Re-Identification
on SYSU-30k
(using extra training data)
Unconstrained Facial Landmark Localization with Backbone-Branches Fully-Convolutional Networks
This paper investigates how to rapidly and accurately localize facial landmarks in unconstrained, cluttered environments rather than in the well segmented face images.
Geometric Scene Parsing with Hierarchical LSTM
This paper addresses the problem of geometric scene parsing, i. e. simultaneously labeling geometric surfaces (e. g. sky, ground and vertical plane) and determining the interaction relations (e. g. layering, supporting, siding and affinity) between main regions.
Semantic Object Parsing with Graph LSTM
By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data.
Character Proposal Network for Robust Text Extraction
Maximally stable extremal regions (MSER), which is a popular method to generate character proposals/candidates, has shown superior performance in scene text detection.
Learning Support Correlation Filters for Visual Tracking
Sampling and budgeting training examples are two essential factors in tracking algorithms based on support vector machines (SVMs) as a trade-off between accuracy and efficiency.
Deep Feature Learning with Relative Distance Comparison for Person Re-identification
Identifying the same individual across different scenes is an important yet difficult task in intelligent video surveillance.
Ranked #9 on
Person Re-Identification
on SYSU-30k
(using extra training data)
DISC: Deep Image Saliency Computing via Progressive Representation Learning
Our DISC framework is capable of uniformly highlighting the objects-of-interest from complex background while preserving well object details.
A Deep Structured Model with Radius-Margin Bound for 3D Human Activity Recognition
Understanding human activity is very challenging even with the recently developed 3D/depth sensors.
Semantic Object Parsing with Local-Global Long Short-Term Memory
The long chains of sequential computation by stacked LG-LSTM layers also enable each pixel to sense a much larger region for inference benefiting from the memorization of previous dependencies in all positions along all dimensions.
Proposal-free Network for Instance-level Object Segmentation
Instance-level object segmentation is an important yet under-explored task.
Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification
Furthermore, each bit of our hashing codes is unequally weighted so that we can manipulate the code lengths by truncating the insignificant bits.
Deep Boosting: Joint Feature Selection and Analysis Dictionary Learning in Hierarchy
In each layer, we construct a dictionary of filters by combining the filters from the lower layer, and iteratively optimize the image representation with a joint discriminative-generative formulation, i. e. minimization of empirical classification error plus regularization of analysis image generation over training images.
PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Edge-Preserving Coherence
In this paper, we propose a unified framework called PISA, which stands for Pixelwise Image Saliency Aggregating various bottom-up cues and priors.
Computational Baby Learning
Then the concept detector can be fine-tuned based on these new instances.
F-SVM: Combination of Feature Transformation and SVM Learning via Convex Relaxation
The generalization error bound of support vector machine (SVM) depends on the ratio of radius and margin, while standard SVM only considers the maximization of the margin but ignores the minimization of the radius.
End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning
Sketch-based face recognition is an interesting task in vision and multimedia research, yet it is quite challenging due to the great difference between face photos and sketches.
Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
Under the classic K Nearest Neighbor (KNN)-based nonparametric framework, the parametric Matching Convolutional Neural Network (M-CNN) is proposed to predict the matching confidence and displacements of the best matched region in the testing image for a particular semantic region in one KNN image.
Data-Driven Scene Understanding with Adaptively Retrieved Exemplars
In the first step, the references are selected by jointly matching their appearances with the target as well as the semantics (i. e. the assigned labels of the target and the references).
Incorporating Structural Alternatives and Sharing into Hierarchy for Multiclass Object Recognition and Detection
This paper proposes a reconfigurable model to recognize and detect multiclass (or multiview) objects with large variation in appearance.
Deep Joint Task Learning for Generic Object Extraction
We present a general joint task learning framework, in which each task (either object localization or object segmentation) is tackled via a multi-layer convolutional neural network, and the two networks work collaboratively to boost performance.
Dynamical And-Or Graph Learning for Object Shape Modeling and Detection
A discriminative learning algorithm, extended from the CCCP [23], is proposed to train the model in a dynamical manner: the model structure (e. g., the configuration of the leaf-nodes associated with the or-nodes) is automatically determined with optimizing the multi-layer parameters during the iteration.
Clothing Co-Parsing by Joint Image Segmentation and Labeling
This paper aims at developing an integrated system of clothing co-parsing, in order to jointly parse a set of clothing images (unsegmented but annotated with tags) into semantic configurations.
Learning Contour-Fragment-based Shape Model with And-Or Tree Representation
This paper proposes a simple yet effective method to learn the hierarchical object shape model consisting of local contour fragments, which represents a category of shapes in the form of an And-Or tree.
Deep Boosting: Layered Feature Mining for General Image Classification
Constructing effective representations is a critical but challenging problem in multimedia understanding.
An Expressive Deep Model for Human Action Parsing from A Single Image
This paper aims at one newly raising task in vision and multimedia research: recognizing human actions from still images.
Towards a solid solution of real-time fire and flame detection
Although the object detection and recognition has received growing attention for decades, a robust fire and flame detection method is rarely explored.
Integrating Graph Partitioning and Matching for Trajectory Analysis in Video Surveillance
With this graph representation, we pose trajectory analysis as a joint task of spatial graph partitioning and temporal graph matching.
Adaptive Scene Category Discovery with Generative Learning and Compositional Sampling
During the iterations of inference, the model of each category is analytically updated by a generative learning algorithm.
Iterated Support Vector Machines for Distance Metric Learning
Distance metric learning aims to learn from the given training data a valid distance metric, with which the similarity between data samples can be more effectively evaluated for classification.
Complex Background Subtraction by Pursuing Dynamic Spatio-Temporal Models
Although it has been widely discussed in video surveillance, background subtraction is still an open problem in the context of complex scenarios, e. g., dynamic backgrounds, illumination variations, and indistinct foreground objects.
Discriminatively Trained And-Or Graph Models for Object Shape Detection
In this paper, we investigate a novel reconfigurable part-based model, namely And-Or graph model, to recognize object shapes in images.
3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks
Our model thus advances existing approaches in two aspects: (i) it acts directly on the raw inputs (grayscale-depth data) to conduct recognition instead of relying on hand-crafted features, and (ii) the model structure can be dynamically adjusted accounting for the temporal variations of human activities, i. e. the network configuration is allowed to be partially activated during inference.
Learning Latent Spatio-Temporal Compositional Model for Human Action Recognition
Action recognition is an important problem in multimedia understanding.
Human Re-identification by Matching Compositional Template with Cluster Sampling
This paper aims at a newly raising task in visual surveillance: re-identifying people at a distance by matching body information, given several reference examples.
Correntropy Induced L2 Graph for Robust Subspace Clustering
In this paper, we study the robust subspace clustering problem, which aims to cluster the given possibly noisy data points into their underlying subspaces.
Crowd Counting using Deep Recurrent Spatial-Aware Network
Crowd counting from unconstrained scene images is a crucial task in many real-world applications like urban surveillance and management, but it is greatly challenged by the camera's perspective that causes huge appearance variations in people's scales and rotations.
Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition
Humans can naturally understand an image in depth with the aid of rich knowledge accumulated from daily lives or professions.
Fine-Grained Image Classification
Fine-Grained Image Recognition
+1
SCAN: Self-and-Collaborative Attention Network for Video Person Re-identification
To address the above issues, we present a novel and practical deep architecture for video person re-identification termed Self-and-Collaborative Attention Network (SCAN).
Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining
Single image rain streaks removal has recently witnessed substantial progress due to the development of deep convolutional neural networks.
Neural Task Planning with And-Or Graph Representations
This paper focuses on semantic task planning, i. e., predicting a sequence of actions toward accomplishing a specific task under a certain scene, which is a new problem in computer vision research.
Attentive Crowd Flow Machines
Traffic flow prediction is crucial for urban traffic management and public safety.
Interpretable Visual Question Answering by Reasoning on Dependency Trees
Collaborative reasoning for understanding image-question pairs is a very critical but underexplored topic in interpretable visual question answering systems.
PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report
This paper reviews the first challenge on efficient perceptual image enhancement with the focus on deploying deep learning models on smartphones.
Learning Deep Representations for Semantic Image Parsing: a Comprehensive Overview
Semantic image parsing, which refers to the process of decomposing images into semantic regions and constructing the structure representation of the input, has recently aroused widespread interest in the field of computer vision.
Cross-Modal Attentional Context Learning for RGB-D Object Detection
In this paper, we address this problem by developing a Cross-Modal Attentional Context (CMAC) learning framework, which enables the full exploitation of the context information from both RGB and depth data.
FRAME Revisited: An Interpretation View Based on Particle Evolution
FRAME (Filters, Random fields, And Maximum Entropy) is an energy-based descriptive model that synthesizes visual realism by capturing mutual patterns from structural input signals.
Facial Landmark Machines: A Backbone-Branches Architecture with Progressive Representation Learning
In this paper, we propose a novel cascaded backbone-branches fully convolutional neural network~(BB-FCN) for rapidly and accurately localizing facial landmarks in unconstrained and cluttered settings.
NADPEx: An on-policy temporally consistent exploration method for deep reinforcement learning
Reinforcement learning agents need exploratory behaviors to escape from local optima.
Kalman Normalization: Normalizing Internal Representations Across Network Layers
In this paper, we present a novel normalization method, called Kalman Normalization (KN), for improving and accelerating the training of DNNs, particularly under the context of micro-batches.
Flow Guided Recurrent Neural Encoder for Video Salient Object Detection
Image saliency detection has recently witnessed significant progress due to deep convolutional neural networks.
Ranked #2 on
Video Salient Object Detection
on DAVSOD-Difficult20
(using extra training data)
Interpretable Video Captioning via Trajectory Structured Localization
Automatically describing open-domain videos with natural language are attracting increasing interest in the field of artificial intelligence.
Monocular Depth Estimation with Affinity, Vertical Pooling, and Label Enhancement
While significant progress has been made in monocular depth estimation with Convolutional Neural Networks (CNNs) extracting absolute features, such as edges and textures, the depth constraint of neighboring pixels, namely relative features, has been mostly ignored by recent methods.
Generative Semantic Manipulation with Mask-Contrasting GAN
Despite the promising results on paired/unpaired image-to-image translation achieved by Generative Adversarial Networks (GANs), prior works often only transfer the low-level information (e. g. color or texture changes), but fail to manipulate high-level semantic meanings (e. g., geometric structure or content) of different object regions.
Robust Region Grouping via Internal Patch Statistics
In this work, we present an efficient multi-scale low-rank representation for image segmentation.
PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors
By fusing complementary contrast measures in such a pixelwise adaptive manner, the detection effectiveness is significantly boosted.
Discriminative Learning of Iteration-Wise Priors for Blind Deconvolution
The maximum a posterior (MAP)-based blind deconvolution framework generally involves two stages: blur kernel estimation and non-blind restoration.
SOLD: Sub-Optimal Low-rank Decomposition for Efficient Video Segmentation
In particular, the affinity matrix with the rank fixed can be decomposed into two sub-matrices of low rank, and then we iteratively optimize them with closed-form solutions.
Joint Learning of Single-Image and Cross-Image Representations for Person Re-Identification
Person re-identification has been usually solved as either the matching of single-image representation (SIR) or the classification of cross-image representation (CIR).
