Search Results for author: Qingming Huang

Found 148 papers, 88 papers with code

Weakly-Supervised Crowd Counting Learns from Sorting rather than Locations

no code implementations ECCV 2020 Yifan Yang, Guorong Li, Zhe Wu, Li Su, Qingming Huang, Nicu Sebe

We propose a soft-label sorting network along with the counting network, which sorts the given images by their crowd numbers.

Crowd Counting

Interpretable Visual Reasoning via Probabilistic Formulation under Natural Supervision

no code implementations ECCV 2020 Xinzhe Han, Shuhui Wang, Chi Su, Weigang Zhang, Qingming Huang, Qi Tian

In this paper, we rethink implicit reasoning process in VQA, and propose a new formulation which maximizes the log-likelihood of joint distribution for the observed question and predicted answer.

Question Answering Visual Question Answering +1

Context-aware Difference Distilling for Multi-change Captioning

no code implementations31 May 2024 Yunbin Tu, Liang Li, Li Su, Zheng-Jun Zha, Chenggang Yan, Qingming Huang

Given an image pair, CARD first decouples context features that aggregate all similar/dissimilar semantics, termed common/difference context features.

Decoder

Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection

1 code implementation16 May 2024 Feiran Li, Qianqian Xu, Shilong Bao, Zhiyong Yang, Runmin Cong, Xiaochun Cao, Qingming Huang

This paper explores the size-invariance of evaluation metrics in Salient Object Detection (SOD), especially when multiple targets of diverse sizes co-exist in the same image.

Object object-detection +2

ReconBoost: Boosting Can Achieve Modality Reconcilement

1 code implementation15 May 2024 Cong Hua, Qianqian Xu, Shilong Bao, Zhiyong Yang, Qingming Huang

This paper explores a novel multi-modal alternating learning paradigm pursuing a reconciliation between the exploitation of uni-modal features and the exploration of cross-modal interactions.

Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition

1 code implementation13 May 2024 Zhiyong Yang, Qianqian Xu, Zitai Wang, Sicong Li, Boyu Han, Shilong Bao, Xiaochun Cao, Qingming Huang

Traditional methods predominantly use a Mixture-of-Expert (MoE) approach, targeting a few fixed test label distributions that exhibit substantial global variations.

Image Classification Long-tail Learning +1

Retrieval Enhanced Zero-Shot Video Captioning

no code implementations11 May 2024 Yunchuan Ma, Laiyun Qing, Guorong Li, Yuankai Qi, Quan Z. Sheng, Qingming Huang

Despite the significant progress of fully-supervised video captioning, zero-shot methods remain much less explored.

Retrieval Test-time Adaptation +3

Uncertainty-boosted Robust Video Activity Anticipation

1 code implementation29 Apr 2024 Zhaobo Qi, Shuhui Wang, Weigang Zhang, Qingming Huang

Video activity anticipation aims to predict what will happen in the future, embracing a broad application prospect ranging from robot vision and autonomous driving.

Autonomous Driving

A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes

no code implementations12 Mar 2024 Ting Yu, Xiaojun Lin, Shuhui Wang, Weiguo Sheng, Qingming Huang, Jun Yu

Three-Dimensional (3D) dense captioning is an emerging vision-language bridging task that aims to generate multiple detailed and accurate descriptions for 3D scenes.

3D dense captioning Dense Captioning

Query-guided Prototype Evolution Network for Few-Shot Segmentation

no code implementations11 Mar 2024 Runmin Cong, Hang Xiong, Jinpeng Chen, Wei zhang, Qingming Huang, Yao Zhao

To address this, we present the Query-guided Prototype Evolution Network (QPENet), a new method that integrates query features into the generation process of foreground and background prototypes, thereby yielding customized prototypes attuned to specific queries.

Segmentation

StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing

no code implementations20 Feb 2024 Gaoxiang Cong, Yuankai Qi, Liang Li, Amin Beheshti, Zhedong Zhang, Anton Van Den Hengel, Ming-Hsuan Yang, Chenggang Yan, Qingming Huang

It contains three main components: (1) A multimodal style adaptor operating at the phoneme level to learn pronunciation style from the reference audio, and generate intermediate representations informed by the facial emotion presented in the video; (2) An utterance-level style learning module, which guides both the mel-spectrogram decoding and the refining processes from the intermediate embeddings to improve the overall style expression; And (3) a phoneme-guided lip aligner to maintain lip sync.

Voice Cloning

Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization

no code implementations30 Jan 2024 Henglei Lv, Jiayu Xiao, Liang Li, Qingming Huang

To this end, we propose Pick-and-Draw, a training-free semantic guidance approach to boost identity consistency and generative diversity for personalization methods.

Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Video

1 code implementation15 Jan 2024 Zhaobo Qi, Yibo Yuan, Xiaowen Ruan, Shuhui Wang, Weigang Zhang, Qingming Huang

Temporal Sentence Grounding in Video (TSGV) is troubled by dataset bias issue, which is caused by the uneven temporal distribution of the target moments for samples with similar semantic components in input videos or query texts.

Sentence Temporal Sentence Grounding

Weakly Supervised Video Individual Counting

no code implementations CVPR 2024 Xinyan Liu, Guorong Li, Yuankai Qi, Ziheng Yan, Zhenjun Han, Anton Van Den Hengel, Ming-Hsuan Yang, Qingming Huang

To provide a more realistic reflection of the underlying practical challenge we introduce a weakly supervised VIC task wherein trajectory labels are not provided.

Contrastive Learning Video Individual Counting

ADA-GAD: Anomaly-Denoised Autoencoders for Graph Anomaly Detection

1 code implementation22 Dec 2023 Junwei He, Qianqian Xu, Yangbangyan Jiang, Zitai Wang, Qingming Huang

We pretrain graph autoencoders on these augmented graphs at multiple levels, which enables the graph autoencoders to capture normal patterns.

Fraud Detection Graph Anomaly Detection

Subject-Oriented Video Captioning

no code implementations20 Dec 2023 Yunchuan Ma, Chang Teng, Yuankai Qi, Guorong Li, Laiyu Qing, Qi Wu, Qingming Huang

To address this problem, we propose a new video captioning task, subject-oriented video captioning, which allows users to specify the describing target via a bounding box.

Video Captioning

Weakly Supervised Video Individual CountingWeakly Supervised Video Individual Counting

1 code implementation10 Dec 2023 Xinyan Liu, Guorong Li, Yuankai Qi, Ziheng Yan, Zhenjun Han, Anton Van Den Hengel, Ming-Hsuan Yang, Qingming Huang

% To provide a more realistic reflection of the underlying practical challenge, we introduce a weakly supervised VIC task, wherein trajectory labels are not provided.

Contrastive Learning Video Individual Counting

Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection

1 code implementation4 Dec 2023 Chen Zhang, Guorong Li, Yuankai Qi, Hanhua Ye, Laiyun Qing, Ming-Hsuan Yang, Qingming Huang

To address these limitations, we propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection, which learns multi-scale temporal features.

Anomaly Detection Video Anomaly Detection

DRAUC: An Instance-wise Distributionally Robust AUC Optimization Framework

1 code implementation NeurIPS 2023 Siran Dai, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, Qingming Huang

To tackle this challenge, methodically we propose an instance-wise surrogate loss of Distributionally Robust AUC (DRAUC) and build our optimization framework on top of it.

R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation

no code implementations13 Oct 2023 Jiayu Xiao, Henglei Lv, Liang Li, Shuhui Wang, Qingming Huang

Recent text-to-image (T2I) diffusion models have achieved remarkable progress in generating high-quality images given text-prompts as input.

Text-to-Image Generation

Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges

no code implementations12 Oct 2023 Peifeng Gao, Qianqian Xu, Yibo Yang, Peisong Wen, Huiyang Shao, Zhiyong Yang, Bernard Ghanem, Qingming Huang

While there have been extensive studies on optimization characteristics showing the global optimality of neural collapse, little research has been done on the generalization behaviors during the occurrence of NC.

Open-Set Knowledge-Based Visual Question Answering with Inference Paths

1 code implementation12 Oct 2023 Jingru Gan, Xinzhe Han, Shuhui Wang, Qingming Huang

Given an image and an associated textual question, the purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases.

Knowledge Graphs Multi-class Classification +2

A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment for Imbalanced Learning

1 code implementation NeurIPS 2023 Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, Qingming Huang

However, existing generalization analysis of such losses is still coarse-grained and fragmented, failing to explain some empirical results.

Learning node representation via Motif Coarsening

1 code implementation journal 2023 Junyu Chen, Qianqian Xu, Zhiyong Yang, Ke Ma, Xiaochun Cao, Qingming Huang

For the motif-based node representation learning process, we propose a Motif Coarsening strategy for incorporating motif structure into the graph representation learning process.

Graph Representation Learning

AUC-Oriented Domain Adaptation: From Theory to Algorithm

1 code implementation TPAMI 2023 Zhiyong Yang, Qianqian Xu, Shilong Bao, Peisong Wen, Xiaochun Cao, Qingming Huang

We propose a new result that not only addresses the interdependency issue but also brings a much sharper bound with weaker assumptions about the loss function.

Disease Prediction Fraud Detection +1

Revisiting AUC-oriented Adversarial Training with Loss-Agnostic Perturbations

2 code implementations TPAMI 2023 Zhiyong Yang, Qianqian Xu, Wenzheng Hou, Shilong Bao, Yuan He, Xiaochun Cao, Qingming Huang

On top of this, we can show that: 1) Under mild conditions, AdAUC can be optimized equivalently with score-based or instance-wise-loss-based perturbations, which is compatible with most of the popular adversarial example generation methods.

When Measures are Unreliable: Imperceptible Adversarial Perturbations toward Top-$k$ Multi-Label Learning

1 code implementation27 Jul 2023 Yuchen Sun, Qianqian Xu, Zitai Wang, Qingming Huang

However, existing adversarial attacks toward multi-label learning only pursue the traditional visual imperceptibility but ignore the new perceptible problem coming from measures such as Precision@$k$ and mAP@$k$.

Adversarial Attack Multi-Label Learning

PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminators

1 code implementation15 Jun 2023 Runmin Cong, Wenyu Yang, Wei zhang, Chongyi Li, Chun-Le Guo, Qingming Huang, Sam Kwong

Among existing UIE methods, Generative Adversarial Networks (GANs) based methods perform well in visual aesthetics, while the physical model-based methods have better scene adaptability.

Quantization UIE

Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training

no code implementations13 May 2023 Ke Zhang, Yan Yang, Jun Yu, Hanliang Jiang, Jianping Fan, Qingming Huang, Weidong Han

To address this limitation, we propose a unified Med-VLP framework based on Multi-task Paired Masking with Alignment (MPMA) to integrate the cross-modal alignment task into the joint image-text reconstruction framework to achieve more comprehensive cross-modal interaction, while a Global and Local Alignment (GLA) module is designed to assist self-supervised paradigm in obtaining semantic representations with rich domain knowledge.

A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalization

no code implementations18 Apr 2023 Peifeng Gao, Qianqian Xu, Peisong Wen, Huiyang Shao, Zhiyong Yang, Qingming Huang

Out of curiosity about the symmetry of Grassmannian Frame, we conduct experiments to explore if models with different Grassmannian Frames have different performance.

Neighborhood Contrastive Transformer for Change Captioning

1 code implementation6 Mar 2023 Yunbin Tu, Liang Li, Li Su, Ke Lu, Qingming Huang

Change captioning is to describe the semantic change between a pair of similar images in natural language.

Decoder Image Captioning

Stable Attribute Group Editing for Reliable Few-shot Image Generation

1 code implementation1 Feb 2023 Guanqi Ding, Xinzhe Han, Shuhui Wang, Xin Jin, Dandan Tu, Qingming Huang

SAGE takes use of all given few-shot images and estimates a class center embedding based on the category-relevant attribute dictionary.

Attribute Classification +1

Building Bridge Across the Time: Disruption and Restoration of Murals In the Wild

no code implementations ICCV 2023 Huiyang Shao, Qianqian Xu, Peisong Wen, Peifeng Gao, Zhiyong Yang, Qingming Huang

Finally, experimental results support the effectiveness of the proposed framework in terms of both mural synthesis and restoration.

Image Restoration

Text-Driven Generative Domain Adaptation with Spectral Consistency Regularization

1 code implementation ICCV 2023 Zhenhuan Liu, Liang Li, Jiayu Xiao, Zheng-Jun Zha, Qingming Huang

The experiments demonstrate the effectiveness of our method to preserve the diversity of source domain and generate high fidelity target images.

Domain Adaptation

Learning to Dub Movies via Hierarchical Prosody Models

1 code implementation CVPR 2023 Gaoxiang Cong, Liang Li, Yuankai Qi, ZhengJun Zha, Qi Wu, Wenyu Wang, Bin Jiang, Ming-Hsuan Yang, Qingming Huang

Given a piece of text, a video clip and a reference audio, the movie dubbing (also known as visual voice clone V2C) task aims to generate speeches that match the speaker's emotion presented in the video using the desired speaker voice as reference.

Progressive Multi-resolution Loss for Crowd Counting

1 code implementation8 Dec 2022 Ziheng Yan, Yuankai Qi, Guorong Li, Xinyan Liu, Weigang Zhang, Qingming Huang, Ming-Hsuan Yang

Crowd counting is usually handled in a density map regression fashion, which is supervised via a L2 loss between the predicted density map and ground truth.

Crowd Counting

Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective

1 code implementation CVPR 2022 Yunrui Zhao, Qianqian Xu, Yangbangyan Jiang, Peisong Wen, Qingming Huang

Positive-Unlabeled (PU) learning tries to learn binary classifiers from a few labeled positive examples with many unlabeled ones.

OpenAUC: Towards AUC-Oriented Open-Set Recognition

1 code implementation22 Oct 2022 Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, Qingming Huang

In this paper, a systematic analysis reveals that most existing metrics are essentially inconsistent with the aforementioned goal of OSR: (1) For metrics extended from close-set classification, such as Open-set F-score, Youden's index, and Normalized Accuracy, a poor open-set prediction can escape from a low performance score with a superior close-set prediction.

Novelty Detection Open Set Learning

Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspective

2 code implementations9 Oct 2022 Yao Zhu, Yuefeng Chen, Xiaodan Li, Kejiang Chen, Yuan He, Xiang Tian, Bolun Zheng, Yaowu Chen, Qingming Huang

We conduct comprehensive transferable attacks against multiple DNNs to demonstrate the effectiveness of the proposed method.

Does Thermal Really Always Matter for RGB-T Salient Object Detection?

2 code implementations9 Oct 2022 Runmin Cong, Kepu Zhang, Chen Zhang, Feng Zheng, Yao Zhao, Qingming Huang, Sam Kwong

In addition, considering the role of thermal modality, we set up different cross-modality interaction mechanisms in the encoding phase and the decoding phase.

object-detection Object Detection +2

CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection

3 code implementations6 Oct 2022 Runmin Cong, Qinwei Lin, Chen Zhang, Chongyi Li, Xiaochun Cao, Qingming Huang, Yao Zhao

Focusing on the issue of how to effectively capture and utilize cross-modality information in RGB-D salient object detection (SOD) task, we present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement.

Decoder object-detection +2

The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm

1 code implementation NeurIPS 2023 Shilong Bao, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, Qingming Huang

Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems (RS), closing the gap between metric learning and Collaborative Filtering.

Collaborative Filtering Metric Learning +1

MaxMatch: Semi-Supervised Learning with Worst-Case Consistency

no code implementations26 Sep 2022 Yangbangyan Jiang, Xiaodan Li, Yuefeng Chen, Yuan He, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, Qingming Huang

In recent years, great progress has been made to incorporate unlabeled data to overcome the inefficiently supervised problem via semi-supervised learning (SSL).

A Tale of HodgeRank and Spectral Method: Target Attack Against Rank Aggregation Is the Fixed Point of Adversarial Game

1 code implementation13 Sep 2022 Ke Ma, Qianqian Xu, Jinshan Zeng, Guorong Li, Xiaochun Cao, Qingming Huang

From the perspective of the dynamical system, the attack behavior with a target ranking list is a fixed point belonging to the composition of the adversary and the victim.

Information Retrieval Retrieval

Optimizing Partial Area Under the Top-k Curve: Theory and Practice

1 code implementation3 Sep 2022 Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, Qingming Huang

Finally, the experimental results on four benchmark datasets validate the effectiveness of our proposed framework.

Multi-Attention Network for Compressed Video Referring Object Segmentation

1 code implementation26 Jul 2022 Weidong Chen, Dexiang Hong, Yuankai Qi, Zhenjun Han, Shuhui Wang, Laiyun Qing, Qingming Huang, Guorong Li

To address this problem, we propose a multi-attention network which consists of dual-path dual-attention module and a query-based cross-modal Transformer module.

Object Referring Expression Segmentation +4

Entity-enhanced Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding

1 code implementation18 Jul 2022 Xuejing Liu, Liang Li, Shuhui Wang, Zheng-Jun Zha, Zechao Li, Qi Tian, Qingming Huang

Second, most previous weakly supervised REG methods ignore the discriminative location and context of the referent, causing difficulties in distinguishing the target from other same-category objects.

Attribute Referring Expression +2

Geometry Interaction Knowledge Graph Embeddings

1 code implementation24 Jun 2022 Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, Qingming Huang

Knowledge graph (KG) embeddings have shown great power in learning representations of entities and relations for link prediction tasks.

Knowledge Graph Completion Knowledge Graph Embeddings +1

ER: Equivariance Regularizer for Knowledge Graph Completion

1 code implementation24 Jun 2022 Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Qingming Huang

To address this issue, we propose a new regularizer, namely, Equivariance Regularizer (ER), which can suppress overfitting by leveraging the implicit semantic information.

Knowledge Graph Completion

AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems

no code implementations ICML 2022 Wenzheng Hou, Qianqian Xu, Zhiyong Yang, Shilong Bao, Yuan He, Qingming Huang

Our analysis differs from the existing studies since the algorithm is asked to generate adversarial examples by calculating the gradient of a min-max problem.

Optimizing Two-way Partial AUC with an End-to-end Framework

1 code implementation TPAMI 2022 Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao, Qingming Huang

The critical challenge along this course lies in the difficulty of performing gradient-based optimization with end-to-end stochastic training, even with a proper choice of surrogate loss.

Vocal Bursts Valence Prediction

Automatic Relation-aware Graph Network Proliferation

1 code implementation CVPR 2022 Shaofei Cai, Liang Li, Xinzhe Han, Jiebo Luo, Zheng-Jun Zha, Qingming Huang

However, the currently used graph search space overemphasizes learning node features and neglects mining hierarchical relational information.

Graph Classification Graph Learning +5

Global-and-Local Collaborative Learning for Co-Salient Object Detection

2 code implementations19 Apr 2022 Runmin Cong, Ning Yang, Chongyi Li, Huazhu Fu, Yao Zhao, Qingming Huang, Sam Kwong

In this paper, we propose a global-and-local collaborative learning architecture, which includes a global correspondence modeling (GCM) and a local correspondence modeling (LCM) to capture comprehensive inter-image corresponding relationship among different images from the global and local perspectives.

8k Co-Salient Object Detection +2

CenterNet++ for Object Detection

2 code implementations18 Apr 2022 Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, Qi Tian

Our approach, named CenterNet, detects each object as a triplet keypoints (top-left and bottom-right corners and the center keypoint).

Object object-detection +1

Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment

2 code implementations CVPR 2022 Jiayu Xiao, Liang Li, Chaofei Wang, Zheng-Jun Zha, Qingming Huang

A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption.

Generative Adversarial Network

General Greedy De-bias Learning

1 code implementation20 Dec 2021 Xinzhe Han, Shuhui Wang, Chi Su, Qingming Huang, Qi Tian

Existing de-bias learning frameworks try to capture specific dataset bias by annotations but they fail to handle complicated OOD scenarios.

Image Classification Question Answering +1

When False Positive is Intolerant: End-to-End Optimization with Low FPR for Multipartite Ranking

no code implementations NeurIPS 2021 Peisong Wen, Qianqian Xu, Zhiyong Yang, Yuan He, Qingming Huang

To leverage high performance under low FPRs, we consider an alternative metric for multipartite ranking evaluating the True Positive Rate (TPR) at a given FPR, denoted as TPR@FPR.

Hierarchical Modular Network for Video Captioning

1 code implementation CVPR 2022 Hanhua Ye, Guorong Li, Yuankai Qi, Shuhui Wang, Qingming Huang, Ming-Hsuan Yang

(II) Predicate level, which learns the actions conditioned on highlighted objects and is supervised by the predicate in captions.

Representation Learning Sentence +1

Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysis

1 code implementation23 Nov 2021 Zhaobo Qi, Shuhui Wang, Chi Su, Li Su, Weigang Zhang, Qingming Huang

Based on TDC, we propose the temporal dynamic concept modeling network (TDCMN) to learn an accurate and complete concept representation for efficient untrimmed video analysis.

Image Categorization

DVCFlow: Modeling Information Flow Towards Human-like Video Captioning

no code implementations19 Nov 2021 Xu Yan, Zhengcong Fei, Shuhui Wang, Qingming Huang, Qi Tian

Dense video captioning (DVC) aims to generate multi-sentence descriptions to elucidate the multiple events in the video, which is challenging and demands visual consistency, discoursal coherence, and linguistic diversity.

Dense Video Captioning Sentence

Learning Meta-path-aware Embeddings for Recommender Systems

1 code implementation ACM MM 2021 2021 Qianxiu Hao, Qianqian Xu, Zhiyong Yang, Qingming Huang

Heterogeneous information networks (HINs) have become a popular tool to capture complicated user-item relationships in recommendation problems in recent years.

Recommendation Systems

Semi-Autoregressive Image Captioning

1 code implementation11 Oct 2021 Xu Yan, Zhengcong Fei, Zekang Li, Shuhui Wang, Qingming Huang, Qi Tian

Non-autoregressive image captioning with continuous iterative refinement, which eliminates the sequential dependence in a sentence generation, can achieve comparable performance to the autoregressive counterparts with a considerable acceleration.

Decoder Image Captioning +1

Pareto Optimality for Fairness-constrained Collaborative Filtering

2 code implementations MM '21: Proceedings of the 29th ACM International Conference on Multimedia 2021 Qianxiu Hao, Qianqian Xu, Zhiyong Yang, Qingming Huang

To balance overall recommendation performance and fairness, prevalent solutions apply fairness constraints or regularizations to enforce equality of certain performance across different subgroups.

Collaborative Filtering Fairness

Edge-featured Graph Neural Architecture Search

no code implementations3 Sep 2021 Shaofei Cai, Liang Li, Xinzhe Han, Zheng-Jun Zha, Qingming Huang

Recently, researchers study neural architecture search (NAS) to reduce the dependence of human expertise and explore better GNN architectures, but they over-emphasize entity features and ignore latent relation information concealed in the edges.

Neural Architecture Search

Greedy Gradient Ensemble for Robust Visual Question Answering

1 code implementation ICCV 2021 Xinzhe Han, Shuhui Wang, Chi Su, Qingming Huang, Qi Tian

Language bias is a critical issue in Visual Question Answering (VQA), where models often exploit dataset biases for the final decision without considering the image information.

Question Answering Visual Question Answering

When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC.

1 code implementation ICML 2021 Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao, Qingming Huang

The critical challenge along this course lies in the difficulty of performing gradient-based optimization with end-to-end stochastic training, even with a proper choice of surrogate loss.

Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation

1 code implementation13 Jul 2021 Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, Qi Tian

Due to the domain discrepancy in visual domain adaptation, the performance of source model degrades when bumping into the high data density near decision boundary in target domain.

Domain Adaptation

Poisoning Attack against Estimating from Pairwise Comparisons

1 code implementation5 Jul 2021 Ke Ma, Qianqian Xu, Jinshan Zeng, Xiaochun Cao, Qingming Huang

In this paper, to the best of our knowledge, we initiate the first systematic investigation of data poisoning attacks on pairwise ranking algorithms, which can be formalized as the dynamic and static games between the ranker and the attacker and can be modeled as certain kinds of integer programming problems.

Data Poisoning

When False Positive is Intolerant: End-to-End Optimization with Low FPR for Multipartite Ranking

no code implementations NeurIPS 2021 Peisong Wen, Qianqian Xu, Zhiyong Yang, Yuan He, Qingming Huang

To leverage high performance under low FPRs, we consider an alternative metric for multipartite ranking evaluating the True Positive Rate (TPR) at a given FPR, denoted as TPR@FPR.

Location-Sensitive Visual Recognition with Cross-IOU Loss

1 code implementation11 Apr 2021 Kaiwen Duan, Lingxi Xie, Honggang Qi, Song Bai, Qingming Huang, Qi Tian

Object detection, instance segmentation, and pose estimation are popular visual recognition tasks which require localizing the object by internal or boundary landmarks.

2D Human Pose Estimation Instance Segmentation +5

Rethinking Graph Neural Architecture Search from Message-passing

1 code implementation CVPR 2021 Shaofei Cai, Liang Li, Jincan Deng, Beichen Zhang, Zheng-Jun Zha, Li Su, Qingming Huang

Inspired by the strong searching capability of neural architecture search (NAS) in CNN, this paper proposes Graph Neural Architecture Search (GNAS) with novel-designed search space.

feature selection Neural Architecture Search

Viewpoint and Scale Consistency Reinforcement for UAV Vehicle Re-Identification

1 code implementation IJCV 2021 Shangzhi Teng, Shiliang Zhang, Qingming Huang, Nicu Sebe

Moreover, our method also achieves competitive performance compared with recent works on existing vehicle ReID datasets including VehicleID, VeRi-776 and VERI-Wild.

Vehicle Re-Identification

Exploiting Sample Correlation for Crowd Counting With Multi-Expert Network

no code implementations ICCV 2021 Xinyan Liu, Guorong Li, Zhenjun Han, Weigang Zhang, Yifan Yang, Qingming Huang, Nicu Sebe

Specifically, we propose a task-driven similarity metric based on sample's mutual enhancement, referred as co-fine-tune similarity, which can find a more efficient subset of data for training the expert network.

Crowd Counting

Heuristic Domain Adaptation

1 code implementation NeurIPS 2020 Shuhao Cui, Xuan Jin, Shuhui Wang, Yuan He, Qingming Huang

In visual domain adaptation (DA), separating the domain-specific characteristics from the domain-invariant representations is an ill-posed problem.

Domain Adaptation

Semantic Editing On Segmentation Map Via Multi-Expansion Loss

no code implementations16 Oct 2020 Jianfeng He, Xuchao Zhang, Shuo Lei, Shuhui Wang, Qingming Huang, Chang-Tien Lu, Bei Xiao

Each MEx area has the mask area of the generation as the majority and the boundary of original context as the minority.

Image Inpainting Segmentation

Label Decoupling Framework for Salient Object Detection

1 code implementation CVPR 2020 Jun Wei, Shuhui Wang, Zhe Wu, Chi Su, Qingming Huang, Qi Tian

Though remarkable progress has been achieved, we observe that the closer the pixel is to the edge, the more difficult it is to be predicted, because edge pixels have a very imbalance distribution.

Object object-detection +3

State-Relabeling Adversarial Active Learning

1 code implementation CVPR 2020 Beichen Zhang, Liang Li, Shijie Yang, Shuhui Wang, Zheng-Jun Zha, Qingming Huang

In this paper, we propose a state relabeling adversarial active learning model (SRAAL), that leverages both the annotation and the labeled/unlabeled state information for deriving the most informative unlabeled samples.

Active Learning

Gradually Vanishing Bridge for Adversarial Domain Adaptation

2 code implementations CVPR 2020 Shuhao Cui, Shuhui Wang, Junbao Zhuo, Chi Su, Qingming Huang, Qi Tian

On the discriminator, GVB contributes to enhance the discriminating ability, and balance the adversarial training process.

Unsupervised Domain Adaptation

Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations

2 code implementations CVPR 2020 Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, Qi Tian

We find by theoretical analysis that the prediction discriminability and diversity could be separately measured by the Frobenius-norm and rank of the batch output matrix.

Domain Adaptation

DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection

1 code implementation19 Mar 2020 Zuyao Chen, Runmin Cong, Qianqian Xu, Qingming Huang

There are two main issues in RGB-D salient object detection: (1) how to effectively integrate the complementarity from the cross-modal RGB-D data; (2) how to prevent the contamination effect from the unreliable depth map.

object-detection RGB-D Salient Object Detection +3

Global Context-Aware Progressive Aggregation Network for Salient Object Detection

2 code implementations2 Mar 2020 Zuyao Chen, Qianqian Xu, Runmin Cong, Qingming Huang

Deep convolutional neural networks have achieved competitive performance in salient object detection, in which how to learn effective and comprehensive features plays a critical role.

Dichotomous Image Segmentation object-detection +1

DM2C: Deep Mixed-Modal Clustering

1 code implementation NeurIPS 2019 Yangbangyan Jiang, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, Qingming Huang

Instead of transforming all the samples into a joint modality-independent space, our framework learns the mappings across individual modal spaces by virtue of cycle-consistency.

Clustering

Generalized Block-Diagonal Structure Pursuit: Learning Soft Latent Task Assignment against Negative Transfer

1 code implementation NeurIPS 2019 Zhiyong Yang, Qianqian Xu, Yangbangyan Jiang, Xiaochun Cao, Qingming Huang

Different from most of the previous work, pursuing the Block-Diagonal structure of LTAM (assigning latent tasks to output tasks) alleviates negative transfer via collaboratively grouping latent tasks and output tasks such that inter-group knowledge transfer and sharing is suppressed.

Attribute Multi-Task Learning

F3Net: Fusion, Feedback and Focus for Salient Object Detection

4 code implementations26 Nov 2019 Jun Wei, Shuhui Wang, Qingming Huang

Furthermore, different from binary cross entropy, the proposed PPA loss doesn't treat pixels equally, which can synthesize the local structure information of a pixel to guide the network to focus more on local details.

Dichotomous Image Segmentation Object +2

Collaborative Preference Embedding against Sparse Labels

1 code implementation ACM MM 2019 Shilong Bao, Qianqian Xu, Ke Ma, Zhiyong Yang, Xiaochun Cao, Qingming Huang

From the margin theory point-of-view, we then propose a generalization enhancement scheme for sparse and insufficient labels via optimizing the margin distribution.

Collaborative Filtering Decision Making +3

iSplit LBI: Individualized Partial Ranking with Ties via Split LBI

1 code implementation NeurIPS 2019 Qianqian Xu, Xinwei Sun, Zhiyong Yang, Xiaochun Cao, Qingming Huang, Yuan YAO

In this paper, instead of learning a global ranking which is agreed with the consensus, we pursue the tie-aware partial ranking from an individualized perspective.

Learning fragment self-attention embeddings for image-text matching

1 code implementation ACMMM 2019 Yiling Wu, Shuhui Wang, Guoli Song, Qingming Huang

In this paper, we propose Self-Attention Embeddings (SAEM) to exploit fragment relations in images or texts by self-attention mechanism, and aggregate fragment information into visual and textual embeddings.

Image-text matching Sentence +1

Knowledge-guided Pairwise Reconstruction Network for Weakly Supervised Referring Expression Grounding

1 code implementation5 Sep 2019 Xuejing Liu, Liang Li, Shuhui Wang, Zheng-Jun Zha, Li Su, Qingming Huang

Weakly supervised referring expression grounding (REG) aims at localizing the referential entity in an image according to linguistic query, where the mapping between the image region (proposal) and the query is unknown in the training stage.

Object Referring Expression +2

Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding

1 code implementation ICCV 2019 Xuejing Liu, Liang Li, Shuhui Wang, Zheng-Jun Zha, Dechao Meng, Qingming Huang

It builds the correspondence between image region proposal and query in an adaptive manner: adaptive grounding and collaborative reconstruction.

Attribute Referring Expression +1

Learning Personalized Attribute Preference via Multi-task AUC Optimization

no code implementations18 Jun 2019 Zhiyong Yang, Qianqian Xu, Xiaochun Cao, Qingming Huang

Traditionally, most of the existing attribute learning methods are trained based on the consensus of annotations aggregated from a limited number of annotators.

Attribute

Multimodal Transformer with Multi-View Visual Representation for Image Captioning

no code implementations20 May 2019 Jun Yu, Jing Li, Zhou Yu, Qingming Huang

Despite the success of existing studies, current methods only model the co-attention that characterizes the inter-modal interactions while neglecting the self-attention that characterizes the intra-modal interactions.

Decoder Image Captioning +2

Unsupervised Open Domain Recognition by Semantic Discrepancy Minimization

1 code implementation CVPR 2019 Junbao Zhuo, Shuhui Wang, Shuhao Cui, Qingming Huang

We address the unsupervised open domain recognition (UODR) problem, where categories in labeled source domain S is only a subset of those in unlabeled target domain T. The task is to correctly classify all samples in T including known and unknown categories.

Classification General Classification

CenterNet: Keypoint Triplets for Object Detection

20 code implementations ICCV 2019 Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, Qi Tian

In object detection, keypoint-based approaches often suffer a large number of incorrect object bounding boxes, arguably due to the lack of an additional look into the cropped regions.

Object object-detection +1

Spatiotemporal CNN for Video Object Segmentation

1 code implementation CVPR 2019 Kai Xu, Longyin Wen, Guorong Li, Liefeng Bo, Qingming Huang

Specifically, the temporal coherence branch pretrained in an adversarial fashion from unlabeled video data, is designed to capture the dynamic appearance and motion cues of video sequences to guide object segmentation.

Object Segmentation +5

Deep Robust Subjective Visual Property Prediction in Crowdsourcing

no code implementations CVPR 2019 Qianqian Xu, Zhiyong Yang, Yangbangyan Jiang, Xiaochun Cao, Qingming Huang, Yuan YAO

The problem of estimating subjective visual properties (SVP) of images (e. g., Shoes A is more comfortable than B) is gaining rising attention.

Property Prediction

HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images

no code implementations16 Nov 2018 Runmin Cong, Jianjun Lei, Huazhu Fu, Qingming Huang, Xiaochun Cao, Nam Ling

In this paper, we propose a novel co-saliency detection method for RGBD images based on hierarchical sparsity reconstruction and energy function refinement.

Co-Salient Object Detection

Person Re-Identification by Semantic Region Representation and Topology Constraint

no code implementations20 Aug 2018 Jianjun Lei, Lijie Niu, Huazhu Fu, Bo Peng, Qingming Huang, Chunping Hou

In this paper, we propose a novel person re-identification method, which consists of a reliable representation called Semantic Region Representation (SRR), and an effective metric learning with Mapping Space Topology Constraint (MSTC).

Metric Learning Person Re-Identification

Weakly Supervised Bilinear Attention Network for Fine-Grained Visual Classification

no code implementations6 Aug 2018 Tao Hu, Jizheng Xu, Cong Huang, Honggang Qi, Qingming Huang, Yan Lu

Besides, we propose attention regularization and attention dropout to weakly supervise the generating process of attention maps.

Classification Fine-Grained Image Classification +1

A Margin-based MLE for Crowdsourced Partial Ranking

no code implementations29 Jul 2018 Qianqian Xu, Jiechao Xiong, Xinwei Sun, Zhiyong Yang, Xiaochun Cao, Qingming Huang, Yuan YAO

A preference order or ranking aggregated from pairwise comparison data is commonly understood as a strict total order.

RAM: A Region-Aware Deep Model for Vehicle Re-Identification

no code implementations25 Jun 2018 Xiaobin Liu, Shiliang Zhang, Qingming Huang, Wen Gao

Specifically, in addition to extracting global features, RAM also extracts features from a series of local regions.

Vehicle Re-Identification

The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking

no code implementations ECCV 2018 Dawei Du, Yuankai Qi, Hongyang Yu, Yifan Yang, Kaiwen Duan, Guorong Li, Weigang Zhang, Qingming Huang, Qi Tian

Selected from 10 hours raw videos, about 80, 000 representative frames are fully annotated with bounding boxes as well as up to 14 kinds of attributes (e. g., weather condition, flying altitude, camera view, vehicle category, and occlusion) for three fundamental computer vision tasks: object detection, single object tracking, and multiple object tracking.

Multiple Object Tracking Object +3

Facial Landmarks Detection by Self-Iterative Regression based Landmarks-Attention Network

no code implementations18 Mar 2018 Tao Hu, Honggang Qi, Jizheng Xu, Qingming Huang

Only one self-iterative regressor is trained to learn the descent directions for samples from coarse stages to fine stages, and parameters are iteratively updated by the same regressor.

Ranked #16 on Face Alignment on 300W (NME_inter-pupil (%, Common) metric)

Face Alignment regression

Review of Visual Saliency Detection with Comprehensive Information

no code implementations9 Mar 2018 Runmin Cong, Jianjun Lei, Huazhu Fu, Ming-Ming Cheng, Weisi Lin, Qingming Huang

With the acquisition technology development, more comprehensive information, such as depth cue, inter-image correspondence, or temporal relationship, is available to extend image saliency detection to RGBD saliency detection, co-saliency detection, or video saliency detection.

Co-Salient Object Detection Video Saliency Detection

From Social to Individuals: a Parsimonious Path of Multi-level Models for Crowdsourced Preference Aggregation

no code implementations8 Mar 2018 Qianqian Xu, Jiechao Xiong, Xiaochun Cao, Qingming Huang, Yuan YAO

In crowdsourced preference aggregation, it is often assumed that all the annotators are subject to a common preference or social utility function which generates their comparison behaviors in experiments.

Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

2 code implementations4 Dec 2017 Jun Yu, Xingxin Xu, Fei Gao, Shengjie Shi, Meng Wang, DaCheng Tao, Qingming Huang

Experimental results show that our method is capable of generating both visually comfortable and identity-preserving face sketches/photos over a wide range of challenging data.

 Ranked #1 on Face Sketch Synthesis on CUFS (FID metric)

Face Sketch Synthesis Generative Adversarial Network

From Common to Special: When Multi-Attribute Learning Meets Personalized Opinions

no code implementations18 Nov 2017 Zhiyong Yang, Qianqian Xu, Xiaochun Cao, Qingming Huang

However, both categories ignore the joint effect of the two mentioned factors: the personal diversity with respect to the global consensus; and the intrinsic correlation among multiple attributes.

Attribute feature selection

HodgeRank with Information Maximization for Crowdsourced Pairwise Ranking Aggregation

no code implementations16 Nov 2017 Qianqian Xu, Jiechao Xiong, Xi Chen, Qingming Huang, Yuan YAO

Recently, crowdsourcing has emerged as an effective paradigm for human-powered large scale problem solving in various domains.

An Iterative Co-Saliency Framework for RGBD Images

no code implementations4 Nov 2017 Runmin Cong, Jianjun Lei, Huazhu Fu, Weisi Lin, Qingming Huang, Xiaochun Cao, Chunping Hou

In this paper, we propose an iterative RGBD co-saliency framework, which utilizes the existing single saliency maps as the initialization, and generates the final RGBD cosaliency map by using a refinement-cycle model.

Co-Salient Object Detection

Co-saliency Detection for RGBD Images Based on Multi-constraint Feature Matching and Cross Label Propagation

no code implementations14 Oct 2017 Runmin Cong, Jianjun Lei, Huazhu Fu, Qingming Huang, Xiaochun Cao, Chunping Hou

Different from the most existing co-saliency methods focusing on RGB images, this paper proposes a novel co-saliency detection model for RGBD images, which utilizes the depth information to enhance identification of co-saliency.

Co-Salient Object Detection

Exploring Outliers in Crowdsourced Ranking for QoE

no code implementations18 Jul 2017 Qianqian Xu, Ming Yan, Chendi Huang, Jiechao Xiong, Qingming Huang, Yuan YAO

Outlier detection is a crucial part of robust evaluation for crowdsourceable assessment of Quality of Experience (QoE) and has attracted much attention in recent years.

Outlier Detection

Online Asymmetric Similarity Learning for Cross-Modal Retrieval

no code implementations CVPR 2017 Yiling Wu, Shuhui Wang, Qingming Huang

In this paper, we propose an online learning method to learn the similarity function between heterogeneous modalities by preserving the relative similarity in the training data, which is modeled as a set of bi-directional hinge loss constraints on the cross-modal training triplets.

Cross-Modal Retrieval Retrieval +2

Hedged Deep Tracking

no code implementations CVPR 2016 Yuankai Qi, Shengping Zhang, Lei Qin, Hongxun Yao, Qingming Huang, Jongwoo Lim, Ming-Hsuan Yang

In recent years, several methods have been developed to utilize hierarchical features learned from a deep convolutional neural network (CNN) for visual tracking.

Visual Tracking

Geometric Hypergraph Learning for Visual Tracking

no code implementations18 Mar 2016 Dawei Du, Honggang Qi, Longyin Wen, Qi Tian, Qingming Huang, Siwei Lyu

Graph based representation is widely used in visual tracking field by finding correct correspondences between target parts in consecutive frames.

Visual Tracking

Similarity Gaussian Process Latent Variable Model for Multi-Modal Data Analysis

no code implementations ICCV 2015 Guoli Song, Shuhui Wang, Qingming Huang, Qi Tian

Data from real applications involve multiple modalities representing content with the same semantics and deliver rich information from complementary aspects.

Retrieval

Evaluating Visual Properties via Robust HodgeRank

no code implementations15 Aug 2014 Qianqian Xu, Jiechao Xiong, Xiaochun Cao, Qingming Huang, Yuan YAO

In this paper we study the problem of how to estimate such visual properties from a ranking perspective with the help of the annotators from online crowdsourcing platforms.

Graph Sampling Outlier Detection

Multi-level Discriminative Dictionary Learning towards Hierarchical Visual Categorization

no code implementations CVPR 2013 Li Shen, Shuhui Wang, Gang Sun, Shuqiang Jiang, Qingming Huang

For each internode of the hierarchical category structure, a discriminative dictionary and a set of classification models are learnt for visual categorization, and the dictionaries in different layers are learnt to exploit the discriminative visual properties of different granularity.

Dictionary Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.