Search Results for author: WangMeng Zuo

Found 234 papers, 148 papers with code

Self-Supervised Video Desmoking for Laparoscopic Surgery

1 code implementation17 Mar 2024 Renlong Wu, Zhilu Zhang, Shuohao Zhang, Longfei Gou, Haobin Chen, Lei Zhang, Hao Chen, WangMeng Zuo

On the other hand, in order to enhance the desmoking performance, we further feed the valuable information from PS frame into models, where a masking strategy and a regularization term are presented to avoid trivial solutions.

Learning Hierarchical Color Guidance for Depth Map Super-Resolution

no code implementations12 Mar 2024 Runmin Cong, Ronghui Sheng, Hao Wu, Yulan Guo, Yunchao Wei, WangMeng Zuo, Yao Zhao, Sam Kwong

On the one hand, the low-level detail embedding module is designed to supplement high-frequency color information of depth features in a residual mask manner at the low-level stages.

Depth Map Super-Resolution

A self-supervised CNN for image watermark removal

1 code implementation9 Mar 2024 Chunwei Tian, Menghua Zheng, Tiancai Jiao, WangMeng Zuo, Yanning Zhang, Chia-Wen Lin

Popular convolutional neural networks mainly use paired images in a supervised way for image watermark removal.

VideoElevator: Elevating Video Generation Quality with Versatile Text-to-Image Diffusion Models

1 code implementation8 Mar 2024 Yabo Zhang, Yuxiang Wei, Xianhui Lin, Zheng Hui, Peiran Ren, Xuansong Xie, Xiangyang Ji, WangMeng Zuo

Different from conventional T2V sampling (i. e., temporal and spatial modeling), VideoElevator explicitly decomposes each sampling step into temporal motion refining and spatial quality elevating.

Video Generation

PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis

1 code implementation4 Mar 2024 Zhengyao Lv, Yuxiang Wei, WangMeng Zuo, Kwan-Yee K. Wong

Extensive experiments demonstrate that our approach performs favorably in terms of visual quality, semantic consistency, and layout alignment.

Image Generation

ConSept: Continual Semantic Segmentation via Adapter-based Vision Transformer

no code implementations26 Feb 2024 Bowen Dong, Guanglei Yang, WangMeng Zuo, Lei Zhang

Empirical investigations on the adaptation of existing frameworks to vanilla ViT reveal that incorporating visual adapters into ViTs or fine-tuning ViTs with distillation terms is advantageous for enhancing the segmentation capability of novel classes.

Continual Semantic Segmentation Segmentation +1

A Heterogeneous Dynamic Convolutional Neural Network for Image Super-resolution

1 code implementation24 Feb 2024 Chunwei Tian, Xuanyu Zhang, Jia Ren, WangMeng Zuo, Yanning Zhang, Chia-Wen Lin

The lower network utilizes a symmetric architecture to enhance relations of different layers to mine more structural information, which is complementary with a upper network for image super-resolution.

Image Super-Resolution

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

1 code implementation7 Feb 2024 Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, WangMeng Zuo, Dahua Lin, Yu Qiao, Jing Shao

In the rapidly evolving landscape of Large Language Models (LLMs), ensuring robust safety measures is paramount.

Multiple-choice

A Comprehensive Survey on 3D Content Generation

1 code implementation2 Feb 2024 Jian Liu, Xiaoshui Huang, Tianyu Huang, Lu Chen, Yuenan Hou, Shixiang Tang, Ziwei Liu, Wanli Ouyang, WangMeng Zuo, Junjun Jiang, Xianming Liu

Recent years have witnessed remarkable advances in artificial intelligence generated content(AIGC), with diverse input modalities, e. g., text, image, video, audio and 3D.

Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learning

1 code implementation3 Jan 2024 Zitong Huang, Ze Chen, Zhixing Chen, Erjin Zhou, Xinxing Xu, Rick Siow Mong Goh, Yong liu, ChunMei Feng, WangMeng Zuo

When progressing to a new session, pseudo-features are sampled from old-class distributions combined with training images of the current session to optimize the prompt, thus enabling the model to learn new knowledge while retaining old knowledge.

Few-Shot Class-Incremental Learning Incremental Learning +1

Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks

1 code implementation1 Jan 2024 Zhilu Zhang, Shuohao Zhang, Renlong Wu, Zifei Yan, WangMeng Zuo

It is highly desired but challenging to acquire high-quality photos with clear content in low-light environments.

Deblurring Denoising +2

FILP-3D: Enhancing 3D Few-shot Class-incremental Learning with Pre-trained Vision-Language Models

1 code implementation28 Dec 2023 Wan Xu, Tianyu Huang, Tianyu Qu, Guanglei Yang, Yiwen Guo, WangMeng Zuo

Few-shot class-incremental learning (FSCIL) aims to mitigate the catastrophic forgetting issue when a model is incrementally trained on limited data.

Dimensionality Reduction Few-Shot Class-Incremental Learning +2

Improving Image Restoration through Removing Degradations in Textual Representations

1 code implementation28 Dec 2023 Jingbo Lin, Zhilu Zhang, Yuxiang Wei, Dongwei Ren, Dongsheng Jiang, WangMeng Zuo

To address the cross-modal assistance, we propose to map the degraded images into textual representations for removing the degradations, and then convert the restored textual representations into a guidance image for assisting image restoration.

Deblurring Denoising +2

Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation

1 code implementation26 Dec 2023 Zixian Guo, Yuxiang Wei, Ming Liu, Zhilong Ji, Jinfeng Bai, Yiwen Guo, WangMeng Zuo

Parameter-efficient fine-tuning (PEFT) methods have provided an effective way for adapting large vision-language models to specific tasks or scenarios.

VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering

1 code implementation19 Dec 2023 Chun-Mei Feng, Yang Bai, Tao Luo, Zhen Li, Salman Khan, WangMeng Zuo, Xinxing Xu, Rick Siow Mong Goh, Yong liu

By feeding the retrieved image and question to the VQA model, one can find the images inconsistent with relative caption when the answer by VQA is inconsistent with the answer in the QA pair.

Image Retrieval Question Answering +2

Decoupled Textual Embeddings for Customized Image Generation

1 code implementation19 Dec 2023 Yufei Cai, Yuxiang Wei, Zhilong Ji, Jinfeng Bai, Hu Han, WangMeng Zuo

To decouple irrelevant attributes (i. e., background and pose) from the subject embedding, we further present several attribute mappers that encode each image as several image-specific subject-unrelated embeddings.

Attribute Disentanglement +2

DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior

1 code implementation11 Dec 2023 Tianyu Huang, Yihan Zeng, Zhilu Zhang, Wan Xu, Hang Xu, Songcen Xu, Rynson W. H. Lau, WangMeng Zuo

The priors are then regarded as input conditions to maintain reasonable geometries, in which conditional LoRA and weighted score are further proposed to optimize detailed textures.

Text to 3D

Learning with Noisy Labels Using Collaborative Sample Selection and Contrastive Semi-Supervised Learning

no code implementations24 Oct 2023 Qing Miao, Xiaohe Wu, Chao Xu, Yanli Ji, WangMeng Zuo, Yiwen Guo, Zhaopeng Meng

By incorporating auxiliary information from CLIP and utilizing prompt fine-tuning, we effectively eliminate noisy samples from the clean set and mitigate confirmation bias during training.

Learning with noisy labels

Learning Real-World Image De-Weathering with Imperfect Supervision

1 code implementation23 Oct 2023 Xiaohui Liu, Zhilu Zhang, Xiaohe Wu, Chaoyu Feng, Xiaotao Wang, Lei Lei, WangMeng Zuo

Real-world image de-weathering aims at removing various undesirable weather-related artifacts.

Pseudo Label

A cross Transformer for image denoising

1 code implementation16 Oct 2023 Chunwei Tian, Menghua Zheng, WangMeng Zuo, Shichao Zhang, Yanning Zhang, Chia-Wen Ling

To avoid loss of key information, PB uses three heterogeneous networks to implement multiple interactions of multi-level features to broadly search for extra information for improving the adaptability of an obtained denoiser for complex scenes.

Image Denoising

DualAug: Exploiting Additional Heavy Augmentation with OOD Data Rejection

1 code implementation12 Oct 2023 Zehao Wang, Yiwen Guo, Qizhang Li, Guanglei Yang, WangMeng Zuo

Most existing data augmentation methods tend to find a compromise in augmenting the data, \textit{i. e.}, increasing the amplitude of augmentation carefully to avoid degrading some data too much and doing harm to the model performance.

Data Augmentation Image Classification +1

Rethinking the BERT-like Pretraining for DNA Sequences

no code implementations11 Oct 2023 Chaoqi Liang, Weiqiang Bai, Lifeng Qiao, Yuchen Ren, Jianle Sun, Peng Ye, Hongliang Yan, Xinzhu Ma, WangMeng Zuo, Wanli Ouyang

To address this research gap, we first conducted a series of exploratory experiments and gained several insightful observations: 1) In the fine-tuning phase of downstream tasks, when using K-mer overlapping tokenization instead of K-mer non-overlapping tokenization, both overlapping and non-overlapping pretraining weights show consistent performance improvement. 2) During the pre-training process, using K-mer overlapping tokenization quickly produces clear K-mer embeddings and reduces the loss to a very low level, while using K-mer non-overlapping tokenization results in less distinct embeddings and continuously decreases the loss.

Sentence-level Prompts Benefit Composed Image Retrieval

1 code implementation9 Oct 2023 Yang Bai, Xinxing Xu, Yong liu, Salman Khan, Fahad Khan, WangMeng Zuo, Rick Siow Mong Goh, Chun-Mei Feng

Composed image retrieval (CIR) is the task of retrieving specific images by using a query that involves both a reference image and a relative caption.

Attribute Composed Image Retrieval (CoIR) +2

Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes

1 code implementation3 Oct 2023 Zhilu Zhang, Haoyu Wang, Shuai Liu, Xiaotao Wang, Lei Lei, WangMeng Zuo

The color component is estimated from aligned multi-exposure images, while the structure one is generated through a structure-focused network that is supervised by the color component and an input reference (\eg, medium-exposure) image.

HDR Reconstruction

Beyond Image Borders: Learning Feature Extrapolation for Unbounded Image Composition

1 code implementation ICCV 2023 Xiaoyu Liu, Ming Liu, Junyi Li, Shuai Liu, Xiaotao Wang, Lei Lei, WangMeng Zuo

In this paper, we circumvent this issue by presenting a joint framework for both unbounded recommendation of camera view and image composition (i. e., UNIC).

Image Cropping

MetaF2N: Blind Image Super-Resolution by Learning Efficient Model Adaptation from Faces

1 code implementation ICCV 2023 Zhicun Yin, Ming Liu, Xiaoming Li, Hui Yang, Longan Xiao, WangMeng Zuo

To evaluate our proposed MetaF2N, we have collected a real-world low-quality dataset with one or multiple faces in each image, and our MetaF2N achieves superior performance on both synthetic and real-world datasets.

Image Generation Image Super-Resolution +1

Aggregating Long-term Sharp Features via Hybrid Transformers for Video Deblurring

1 code implementation13 Sep 2023 Dongwei Ren, Wei Shang, Yi Yang, WangMeng Zuo

To aggregate long-term sharp features from detected sharp frames, we utilize a global Transformer with multi-scale matching capability.

Deblurring

Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration

no code implementations4 Sep 2023 Yuanshuo Cheng, Mingwen Shao, Yecong Wan, Yuanjian Qiao, WangMeng Zuo, Deyu Meng

To empower the framework for eliminating diverse degradations, we devise a Sequence-wise Adaptive Degradation Estimator (SADE) to estimate degradation features for the input corrupted video.

Video Restoration

Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models

no code implementations31 Aug 2023 Minheng Ni, Yabo Zhang, Kailai Feng, Xiaoming Li, Yiwen Guo, WangMeng Zuo

In this work, we introduce a novel Referring Diffusional segmentor (Ref-Diff) for this task, which leverages the fine-grained multi-modal information from generative models.

Image Segmentation Instance Segmentation +2

VQ-Font: Few-Shot Font Generation with Structure-Aware Enhancement and Quantization

1 code implementation27 Aug 2023 Mingshuai Yao, Yabo Zhang, Xianhui Lin, Xiaoming Li, WangMeng Zuo

In this paper, we propose a VQGAN-based framework (i. e., VQ-Font) to enhance glyph fidelity through token prior refinement and structure-aware enhancement.

Font Generation Quantization

UniM$^2$AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving

1 code implementation21 Aug 2023 Jian Zou, Tianyu Huang, Guanglei Yang, Zhenhua Guo, WangMeng Zuo

The extension makes it possible to back-project the informative features, obtained by fusing features from both modalities, into their native modalities to reconstruct the multiple masked inputs.

3D Object Detection Autonomous Driving +1

Rethinking Client Drift in Federated Learning: A Logit Perspective

no code implementations20 Aug 2023 Yunlu Yan, Chun-Mei Feng, Mang Ye, WangMeng Zuo, Ping Li, Rick Siow Mong Goh, Lei Zhu, C. L. Philip Chen

Concretely, FedCSD introduces a class prototype similarity distillation to align the local logits with the refined global logits that are weighted by the similarity between local logits and the global prototype.

Federated Learning

Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning

1 code implementation ICCV 2023 Chun-Mei Feng, Kai Yu, Yong liu, Salman Khan, WangMeng Zuo

In this paper, we focus on a particular setting of learning adaptive prompts on the fly for each test sample from an unseen new domain, which is known as test-time prompt tuning (TPT).

Data Augmentation

Data-free Black-box Attack based on Diffusion Model

no code implementations24 Jul 2023 Mingwen Shao, Lingzhuang Meng, Yuanjian Qiao, Lixu Zhang, WangMeng Zuo

Since the training data for the target model in a data-free black-box attack is not available, most recent schemes utilize GANs to generate data for training substitute model.

Improving Transferability of Adversarial Examples via Bayesian Attacks

no code implementations21 Jul 2023 Qizhang Li, Yiwen Guo, Xiaochen Yang, WangMeng Zuo, Hao Chen

Our ICLR work advocated for enhancing transferability in adversarial examples by incorporating a Bayesian formulation into model parameters, which effectively emulates the ensemble of infinitely many deep neural networks, while, in this paper, we introduce a novel extension by incorporating the Bayesian formulation into the model input as well, enabling the joint diversification of both the model input and model parameters.

RBSR: Efficient and Flexible Recurrent Network for Burst Super-Resolution

1 code implementation30 Jun 2023 Renlong Wu, Zhilu Zhang, Shuohao Zhang, Hongzhi Zhang, WangMeng Zuo

The main challenge of BurstSR is to effectively combine the complementary information from input frames, while existing methods still struggle with it.

Super-Resolution

Self-supervised Learning to Bring Dual Reversed Rolling Shutter Images Alive

2 code implementations ICCV 2023 Wei Shang, Dongwei Ren, Chaoyu Feng, Xiaotao Wang, Lei Lei, WangMeng Zuo

In this paper, we propose a Self-supervised learning framework for Dual reversed RS distortions Correction (SelfDRSC), where a DRSC network can be learned to generate a high framerate GS video only based on dual RS images with reversed distortions.

Self-Supervised Learning

Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesis

1 code implementation CVPR 2023 Yuxiang Wei, Zhilong Ji, Xiaohe Wu, Jinfeng Bai, Lei Zhang, WangMeng Zuo

Despite the progress in semantic image synthesis, it remains a challenging problem to generate photo-realistic parts from input semantic map.

Image Generation Object

ControlVideo: Training-free Controllable Text-to-Video Generation

1 code implementation22 May 2023 Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, WangMeng Zuo, Qi Tian

Text-driven diffusion models have unlocked unprecedented abilities in image generation, whereas their video counterpart still lags behind due to the excessive training cost of temporal modeling.

Image Generation Text-to-Video Generation +1

Improving Adversarial Transferability via Intermediate-level Perturbation Decay

2 code implementations NeurIPS 2023 Qizhang Li, Yiwen Guo, WangMeng Zuo, Hao Chen

In particular, the proposed method, named intermediate-level perturbation decay (ILPD), encourages the intermediate-level perturbation to be in an effective adversarial direction and to possess a great magnitude simultaneously.

Learning Federated Visual Prompt in Null Space for MRI Reconstruction

1 code implementation CVPR 2023 Chun-Mei Feng, Bangjun Li, Xinxing Xu, Yong liu, Huazhu Fu, WangMeng Zuo

Federated Magnetic Resonance Imaging (MRI) reconstruction enables multiple hospitals to collaborate distributedly without aggregating local data, thereby protecting patient privacy.

MRI Reconstruction

Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time

2 code implementations CVPR 2023 Wei Shang, Dongwei Ren, Yi Yang, Hongzhi Zhang, Kede Ma, WangMeng Zuo

Moreover, on the seemingly implausible x16 interpolation task, our method outperforms existing methods by more than 1. 5 dB in terms of PSNR.

Contrastive Learning Deblurring +2

Learning Generative Structure Prior for Blind Text Image Super-resolution

1 code implementation CVPR 2023 Xiaoming Li, WangMeng Zuo, Chen Change Loy

To restrict the generative space of StyleGAN so that it obeys the structure of characters yet remains flexible in handling different font styles, we store the discrete features for each character in a codebook.

Image Super-Resolution

Towards Universal Vision-language Omni-supervised Segmentation

no code implementations12 Mar 2023 Bowen Dong, Jiaxi Gu, Jianhua Han, Hang Xu, WangMeng Zuo

To improve the open-world segmentation ability, we leverage omni-supervised data (i. e., panoptic segmentation data, object detection data, and image-text pairs data) into training, thus enriching the open-world segmentation ability and achieving better segmentation accuracy.

Instance Segmentation object-detection +4

ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation

1 code implementation ICCV 2023 Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, WangMeng Zuo

In addition to the unprecedented ability in imaginary creation, large text-to-image models are expected to take customized concepts in image generation.

Text-to-Image Generation

Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examples

1 code implementation10 Feb 2023 Qizhang Li, Yiwen Guo, WangMeng Zuo, Hao Chen

In this paper, by contrast, we opt for the diversity in substitute models and advocate to attack a Bayesian model for achieving desirable transferability.

Physics-Guided ISO-Dependent Sensor Noise Modeling for Extreme Low-Light Photography

no code implementations CVPR 2023 Yue Cao, Ming Liu, Shuai Liu, Xiaotao Wang, Lei Lei, WangMeng Zuo

Although deep neural networks have achieved astonishing performance in many vision tasks, existing learning-based methods are far inferior to the physical model-based solutions in extreme low-light sensor noise modeling.

Image Denoising

NUWA-LIP: Language-Guided Image Inpainting With Defect-Free VQGAN

no code implementations CVPR 2023 Minheng Ni, Xiaoming Li, WangMeng Zuo

Language-guided image inpainting aims to fill the defective regions of an image under the guidance of text while keeping the non-defective regions unchanged.

Image Inpainting

Position-Aware Contrastive Alignment for Referring Image Segmentation

no code implementations27 Dec 2022 Bo Chen, Zhiwei Hu, Zhilong Ji, Jinfeng Bai, WangMeng Zuo

The main challenge of this task is to understand the visual and linguistic content simultaneously and to find the referred object accurately among all instances in the image.

Image Segmentation Position +1

Human Co-Parsing Guided Alignment for Occluded Person Re-identification

1 code implementation IEEE Transactions on Image Processing 2022 Shuguang Dou, Cairong Zhao, Xinyang Jiang, Shanshan Zhang, Wei-Shi Zheng, WangMeng Zuo

Most supervised methods propose to train an extra human parsing model aside from the ReID model with cross-domain human parts annotation, suffering from expensive annotation cost and domain gap; Unsupervised methods integrate a feature clustering-based human parsing process into the ReID model, but lacking supervision signals brings less satisfactory segmentation results.

Human Parsing Person Re-Identification

HS-Diffusion: Semantic-Mixing Diffusion for Head Swapping

1 code implementation13 Dec 2022 Qinghe Wang, Lijie Liu, Miao Hua, Pengfei Zhu, WangMeng Zuo, QinGhua Hu, Huchuan Lu, Bing Cao

We blend the semantic layouts of source head and source body, and then inpaint the transition region by the semantic layout generator, achieving a coarse-grained head swapping.

Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution

1 code implementation10 Dec 2022 Ruohao Wang, Xiaohui Liu, Zhilu Zhang, Xiaohe Wu, Chun-Mei Feng, Lei Zhang, WangMeng Zuo

On the other hand, alignment algorithms in existing VSR methods perform poorly for real-world videos, leading to unsatisfactory results.

Optical Flow Estimation Video Super-Resolution

Relationship Quantification of Image Degradations

no code implementations8 Dec 2022 Wenxin Wang, Boyun Li, Yuanbiao Gou, Peng Hu, WangMeng Zuo, Xi Peng

To tackle the first challenge, we proposed a Degradation Relationship Index (DRI) which is defined as the mean drop rate difference in the validation loss between two models which are respectively trained using the anchor degradation and the mixture of the anchor and the auxiliary degradations.

Denoising Image Dehazing +2

Learning Single Image Defocus Deblurring with Misaligned Training Pairs

2 code implementations26 Nov 2022 Yu Li, Dongwei Ren, Xinya Shu, WangMeng Zuo

First, in the deblurring module, a bi-directional optical flow-based deformation is introduced to tolerate spatial misalignment between deblurred and ground-truth images.

Deblurring Image Defocus Deblurring +1

Self-Supervised Image Restoration with Blurry and Noisy Pairs

1 code implementation14 Nov 2022 Zhilu Zhang, Rongjian Xu, Ming Liu, Zifei Yan, WangMeng Zuo

By learning in a collaborative manner, the deblurring and denoising tasks in our method can benefit each other.

Deblurring Denoising +1

Learning Dual Memory Dictionaries for Blind Face Restoration

1 code implementation15 Oct 2022 Xiaoming Li, Shiguang Zhang, Shangchen Zhou, Lei Zhang, WangMeng Zuo

Generally, it is a challenging and intractable task to improve the photo-realistic performance of blind restoration and adaptively handle the generic and specific restoration scenarios with a single unified model.

Blind Face Restoration

ImaginaryNet: Learning Object Detectors without Real Images and Annotations

1 code implementation13 Oct 2022 Minheng Ni, Zitong Huang, Kailai Feng, WangMeng Zuo

Given a class label, the language model is used to generate a full description of a scene with a target object, and the text-to-image model deployed to generate a photo-realistic image.

Image Generation Language Modelling +3

From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution

1 code implementation3 Oct 2022 Xiaoming Li, Chaofeng Chen, Xianhui Lin, WangMeng Zuo, Lei Zhang

Notably, LQ face images, which may have the same degradation process as natural images, can be robustly restored with photo-realistic textures by exploiting their strong structural priors.

Image Generation Image Super-Resolution

CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training

1 code implementation ICCV 2023 Tianyu Huang, Bowen Dong, Yunhan Yang, Xiaoshui Huang, Rynson W. H. Lau, Wanli Ouyang, WangMeng Zuo

To address this issue, we propose CLIP2Point, an image-depth pre-training method by contrastive learning to transfer CLIP to the 3D domain, and adapt it to point cloud classification.

Contrastive Learning Few-Shot Learning +4

LPT: Long-tailed Prompt Tuning for Image Classification

1 code implementation3 Oct 2022 Bowen Dong, Pan Zhou, Shuicheng Yan, WangMeng Zuo

For better effectiveness, we divide prompts into two groups: 1) a shared prompt for the whole long-tailed dataset to learn general features and to adapt a pretrained model into target domain; and 2) group-specific prompts to gather group-specific features for the samples which have similar features and also to empower the pretrained model with discrimination ability.

 Ranked #1 on Long-tail Learning on CIFAR-100-LT (ρ=100) (using extra training data)

Classification Image Classification +1

A heterogeneous group CNN for image super-resolution

1 code implementation26 Sep 2022 Chunwei Tian, Yanning Zhang, WangMeng Zuo, Chia-Wen Lin, David Zhang, Yixuan Yuan

To prevent loss of original information, a multi-level enhancement mechanism guides a CNN to achieve a symmetric architecture for promoting expressive ability of HGSRCNN.

Image Super-Resolution

Multi-stage image denoising with the wavelet transform

1 code implementation26 Sep 2022 Chunwei Tian, Menghua Zheng, WangMeng Zuo, Bob Zhang, Yanning Zhang, David Zhang

In this paper, we propose a multi-stage image denoising CNN with the wavelet transform (MWDCNN) via three stages, i. e., a dynamic convolutional block (DCB), two cascaded wavelet transform and enhancement blocks (WEBs) and a residual block (RB).

Image Denoising

Learning Hierarchical Dynamics with Spatial Adjacency for Image Enhancement

1 code implementation ACMMM 2022 Yudong Liang, Bin Wang, Wenqi Ren, Jiaying Liu, Wenjian Wang, WangMeng Zuo

In various real-world image enhancement applications, the degradations are always non-uniform or non-homogeneous and diverse, which challenges most deep networks with fixed parameters during the inference phase.

Image Dehazing Low-Light Image Enhancement +1

Two-Stream Networks for Object Segmentation in Videos

no code implementations8 Aug 2022 Hannan Lu, Zhi Tian, Lirong Yang, Haibing Ren, WangMeng Zuo

The compact instance stream effectively improves the segmentation accuracy of the unseen pixels, while fusing two streams with the adaptive routing map leads to an overall performance boost.

Object Retrieval +5

W2N:Switching From Weak Supervision to Noisy Supervision for Object Detection

1 code implementation25 Jul 2022 Zitong Huang, Yiping Bao, Bowen Dong, Erjin Zhou, WangMeng Zuo

Generally, with given pseudo ground-truths generated from the well-trained WSOD network, we propose a two-module iterative training algorithm to refine pseudo labels and supervise better object detector progressively.

Object object-detection +2

A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restoration

1 code implementation21 Jul 2022 Ming Liu, Yuxiang Wei, Xiaohe Wu, WangMeng Zuo, Lei Zhang

Generative adversarial networks (GANs) have drawn enormous attention due to the simple yet effective training mechanism and superior image generation quality.

Image Generation Image Restoration

Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks

1 code implementation18 Jul 2022 Yabo Zhang, Mingshuai Yao, Yuxiang Wei, Zhilong Ji, Jinfeng Bai, WangMeng Zuo

In this paper, we present a novel one-shot generative domain adaption method, i. e., DiFa, for diverse generation and faithful adaptation.

Domain Adaptation

Symmetry-Aware Transformer-based Mirror Detection

1 code implementation13 Jul 2022 Tianyu Huang, Bowen Dong, Jiaying Lin, Xiaohui Liu, Rynson W. H. Lau, WangMeng Zuo

Mirror detection aims to identify the mirror regions in the given input image.

Learning Diverse Tone Styles for Image Retouching

1 code implementation12 Jul 2022 Haolin Wang, Jiawei Zhang, Ming Liu, Xiaohe Wu, WangMeng Zuo

In particular, the style encoder predicts the target style representation of an input image, which serves as the conditional information in the RetouchNet for retouching, while the TSFlow maps the style representation vector into a Gaussian distribution in the forward pass.

Image Retouching

An Improved Normed-Deformable Convolution for Crowd Counting

1 code implementation16 Jun 2022 Xin Zhong, Zhaoyi Yan, Jing Qin, WangMeng Zuo, Weigang Lu

However, the heads are not uniformly covered by the sampling points in the deformable convolution, resulting in loss of head information.

Crowd Counting

Robust Deep Ensemble Method for Real-world Image Denoising

1 code implementation8 Jun 2022 Pengju Liu, Hongzhi Zhang, Jinghui Wang, Yuzhi Wang, Dongwei Ren, WangMeng Zuo

In particular, we take well-trained CBDNet, NBNet, HINet, Uformer and GMSNet into denoiser pool, and a U-Net is adopted to predict pixel-wise weighting maps to fuse these denoisers.

Deblurring Image Deblurring +4

Image Super-resolution with An Enhanced Group Convolutional Neural Network

1 code implementation29 May 2022 Chunwei Tian, Yixuan Yuan, Shichao Zhang, Chia-Wen Lin, WangMeng Zuo, David Zhang

In this paper, we present an enhanced super-resolution group CNN (ESRGCNN) with a shallow architecture by fully fusing deep and wide channel features to extract more accurate low-frequency information in terms of correlations of different channels in single image super-resolution (SISR).

Image Super-Resolution

Squeeze Training for Adversarial Robustness

1 code implementation23 May 2022 Qizhang Li, Yiwen Guo, WangMeng Zuo, Hao Chen

The vulnerability of deep neural networks (DNNs) to adversarial examples has attracted great attention in the machine learning community.

Adversarial Robustness

Learning Dual-Pixel Alignment for Defocus Deblurring

1 code implementation26 Apr 2022 Yu Li, Yaling Yi, Dongwei Ren, Qince Li, WangMeng Zuo

Generally, DPANet is an encoder-decoder with skip-connections, where two branches with shared parameters in the encoder are employed to extract and align deep features from left and right views, and one decoder is adopted to fuse aligned features for predicting the sharp image.

Deblurring

Incorporating Semi-Supervised and Positive-Unlabeled Learning for Boosting Full Reference Image Quality Assessment

no code implementations CVPR 2022 Yue Cao, Zhaolin Wan, Dongwei Ren, Zifei Yan, WangMeng Zuo

Particularly, by treating all labeled data as positive samples, PU learning is leveraged to identify negative samples (i. e., outliers) from unlabeled data.

Image Quality Assessment

Unidirectional Video Denoising by Mimicking Backward Recurrent Modules with Look-ahead Forward Ones

1 code implementation12 Apr 2022 Junyi Li, Xiaohe Wu, Zhenxing Niu, WangMeng Zuo

However, BiRNN is intrinsically offline because it uses backward recurrent modules to propagate from the last to current frames, which causes high latency and large memory consumption.

Denoising Video Denoising +1

Localization Distillation for Object Detection

1 code implementation12 Apr 2022 Zhaohui Zheng, Rongguang Ye, Qibin Hou, Dongwei Ren, Ping Wang, WangMeng Zuo, Ming-Ming Cheng

Combining these two new components, for the first time, we show that logit mimicking can outperform feature imitation and the absence of localization distillation is a critical reason for why logit mimicking underperforms for years.

Knowledge Distillation Object +2

Semantic-shape Adaptive Feature Modulation for Semantic Image Synthesis

1 code implementation CVPR 2022 Zhengyao Lv, Xiaoming Li, Zhenxing Niu, Bing Cao, WangMeng Zuo

Obviously, a fine-grained part-level semantic layout will benefit object details generation, and it can be roughly inferred from an object's shape.

Image Generation Object

An Intermediate-level Attack Framework on The Basis of Linear Regression

1 code implementation21 Mar 2022 Yiwen Guo, Qizhang Li, WangMeng Zuo, Hao Chen

This paper substantially extends our work published at ECCV, in which an intermediate-level attack was proposed to improve the transferability of some baseline adversarial examples.

regression

Self-Promoted Supervision for Few-Shot Transformer

1 code implementation14 Mar 2022 Bowen Dong, Pan Zhou, Shuicheng Yan, WangMeng Zuo

The few-shot learning ability of vision transformers (ViTs) is rarely investigated though heavily desired.

Data Augmentation Few-Shot Learning +1

On Steering Multi-Annotations per Sample for Multi-Task Learning

no code implementations6 Mar 2022 Yuanze Li, Yiwen Guo, Qizhang Li, Hongzhi Zhang, WangMeng Zuo

Despite the remarkable progress, the challenge of optimally learning different tasks simultaneously remains to be explored.

Instance Segmentation Multi-Task Learning +2

Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations

1 code implementation2 Mar 2022 Zhilu Zhang, Ruohao Wang, Hongzhi Zhang, Yunjin Chen, WangMeng Zuo

For this purpose, we take the telephoto image instead of an additional high-resolution image as the supervision information and select a center patch from it as the reference to super-resolve the corresponding short-focus image patch.

Reference-based Super-Resolution Self-Supervised Learning

NÜWA-LIP: Language Guided Image Inpainting with Defect-free VQGAN

no code implementations10 Feb 2022 Minheng Ni, Chenfei Wu, Haoyang Huang, Daxin Jiang, WangMeng Zuo, Nan Duan

Language guided image inpainting aims to fill in the defective regions of an image under the guidance of text while keeping non-defective regions unchanged.

Image Inpainting

Invertible Network for Unpaired Low-light Image Enhancement

no code implementations24 Dec 2021 Jize Zhang, Haolin Wang, Xiaohe Wu, WangMeng Zuo

Existing unpaired low-light image enhancement approaches prefer to employ the two-way GAN framework, in which two CNN generators are deployed for enhancement and degradation separately.

Low-Light Image Enhancement

Infrared Small-Dim Target Detection with Transformer under Complex Backgrounds

no code implementations29 Sep 2021 Fangcen Liu, Chenqiang Gao, Fang Chen, Deyu Meng, WangMeng Zuo, Xinbo Gao

We adopt the self-attention mechanism of the transformer to learn the interaction information of image features in a larger range.

Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision

1 code implementation ICCV 2021 Zhilu Zhang, Haolin Wang, Ming Liu, Ruohao Wang, Jiawei Zhang, WangMeng Zuo

To diminish the effect of color inconsistency in image alignment, we introduce to use a global color mapping (GCM) module to generate an initial sRGB image given the input raw image, which can keep the spatial location of the pixels unchanged, and the target sRGB image is utilized to guide GCM for converting the color towards it.

Optical Flow Estimation

Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting

1 code implementation ICCV 2021 Binghui Chen, Zhaoyi Yan, Ke Li, Pengyu Li, Biao Wang, WangMeng Zuo, Lei Zhang

In crowd counting, due to the problem of laborious labelling, it is perceived intractability of collecting a new large-scale dataset which has plentiful images with large diversity in density, scene, etc.

Crowd Counting

Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation

1 code implementation ICCV 2021 Yuxiang Wei, Yupeng Shi, Xiao Liu, Zhilong Ji, Yuan Gao, Zhongqin Wu, WangMeng Zuo

It simply encourages the variation of output caused by perturbations on different latent dimensions to be orthogonal, and the Jacobian with respect to the input is calculated to represent this variation.

Disentanglement Image Generation

Local Patch Network with Global Attention for Infrared Small Target Detection

1 code implementation13 Aug 2021 Fang Chen, Chenqiang Gao, Fangcen Liu, Yue Zhao, Yuxi Zhou, Deyu Meng, WangMeng Zuo

A local patch network (LPNet) with global attention is proposed in this paper to detect small targets by jointly considering the global and local properties of infrared small target images.

Semantic Segmentation

Boosting Weakly Supervised Object Detection via Learning Bounding Box Adjusters

1 code implementation ICCV 2021 Bowen Dong, Zitong Huang, Yuelin Guo, Qilong Wang, Zhenxing Niu, WangMeng Zuo

In this paper, we defend the problem setting for improving localization performance by leveraging the bounding box regression knowledge from a well-annotated auxiliary dataset.

Object object-detection +3

Crowd Counting via Perspective-Guided Fractional-Dilation Convolution

1 code implementation8 Jul 2021 Zhaoyi Yan, Ruimao Zhang, Hongzhi Zhang, Qingfu Zhang, WangMeng Zuo

One of the main issues in this task is how to handle the dramatic scale variations of pedestrians caused by the perspective effect.

Crowd Counting

VirFace: Enhancing Face Recognition via Unlabeled Shallow Data

no code implementations CVPR 2021 Wenyu Li, Tianchu Guo, Pengyu Li, Binghui Chen, Biao Wang, WangMeng Zuo, Lei Zhang

In this paper, we propose a novel face recognition method, named VirFace, to effectively apply the unlabeled shallow data for face recognition.

Face Recognition

Learning Scalable lY=-Constrained Near-Lossless Image Compression via Joint Lossy Image and Residual Compression

no code implementations CVPR 2021 Yuanchao Bai, Xianming Liu, WangMeng Zuo, YaoWei Wang, Xiangyang Ji

To achieve scalable compression with the error bound larger than zero, we derive the probability model of the quantized residual by quantizing the learned probability model of the original residual, instead of training multiple networks.

Image Compression

Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps

1 code implementation25 Apr 2021 Dongsheng Wang, Chaohao Xie, Shaohui Liu, Zhenxing Niu, WangMeng Zuo

In this paper, we present an edge-guided learnable bidirectional attention map (Edge-LBAM) for improving image inpainting of irregular holes with several distinct merits.

Image Inpainting valid

Learning Semantic Person Image Generation by Region-Adaptive Normalization

1 code implementation CVPR 2021 Zhengyao Lv, Xiaoming Li, Xin Li, Fu Li, Tianwei Lin, Dongliang He, WangMeng Zuo

In the first stage, we predict the target semantic parsing maps to eliminate the difficulties of pose transfer and further benefit the latter translation of per-region appearance style.

Pose Transfer Semantic Parsing +1

Learning Scalable $\ell_\infty$-constrained Near-lossless Image Compression via Joint Lossy Image and Residual Compression

no code implementations31 Mar 2021 Yuanchao Bai, Xianming Liu, WangMeng Zuo, YaoWei Wang, Xiangyang Ji

To achieve scalable compression with the error bound larger than zero, we derive the probability model of the quantized residual by quantizing the learned probability model of the original residual, instead of training multiple networks.

Image Compression

Deepfake Forensics via An Adversarial Game

1 code implementation25 Mar 2021 Zhi Wang, Yiwen Guo, WangMeng Zuo

In this paper, we advocate adversarial training for improving the generalization ability to both unseen facial forgeries and unseen image/video qualities.

Classification DeepFake Detection +2

Asymmetric CNN for image super-resolution

1 code implementation25 Mar 2021 Chunwei Tian, Yong Xu, WangMeng Zuo, Chia-Wen Lin, David Zhang

In this paper, we propose an asymmetric CNN (ACNet) comprising an asymmetric block (AB), a memory enhancement block (MEB) and a high-frequency feature enhancement block (HFFEB) for image super-resolution.

Image Super-Resolution

Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser

1 code implementation18 Mar 2021 Yue Cao, Xiaohe Wu, Shuran Qi, Xiao Liu, Zhongqin Wu, WangMeng Zuo

To begin with, the pre-trained denoiser is used to generate the pseudo clean images for the test images.

Denoising

Localization Distillation for Dense Object Detection

2 code implementations CVPR 2022 Zhaohui Zheng, Rongguang Ye, Ping Wang, Dongwei Ren, WangMeng Zuo, Qibin Hou, Ming-Ming Cheng

Previous KD methods for object detection mostly focus on imitating deep features within the imitation regions instead of mimicking classification logit due to its inefficiency in distilling localization information and trivial improvement.

Dense Object Detection Knowledge Distillation +2

Self Sparse Generative Adversarial Networks

no code implementations26 Jan 2021 Wenliang Qian, Yang Xu, WangMeng Zuo, Hui Li

In this work, we propose a Self Sparse Generative Adversarial Network (Self-Sparse GAN) that reduces the parameter space and alleviates the zero gradient problem.

Generative Adversarial Network Image Generation

Hybrid Trilinear and Bilinear Programming for Aligning Partially Overlapping Point Sets

no code implementations19 Jan 2021 Wei Lian, WangMeng Zuo

The resulting lower bound problem has the merit that it can be efficiently solved via linear assignment and low dimensional convex quadratic programming.

Bringing Events Into Video Deblurring With Non-Consecutively Blurry Frames

1 code implementation ICCV 2021 Wei Shang, Dongwei Ren, Dongqing Zou, Jimmy S. Ren, Ping Luo, WangMeng Zuo

EFM can also be easily incorporated into existing deblurring networks, making event-driven deblurring task benefit from state-of-the-art deblurring methods.

Deblurring

Two-Stage Single Image Reflection Removal with Reflection-Aware Guidance

1 code implementation2 Dec 2020 Yu Li, Ming Liu, Yaling Yi, Qince Li, Dongwei Ren, WangMeng Zuo

To be specific, the reflection layer is firstly estimated due to that it generally is much simpler and is relatively easier to estimate.

Reflection Removal Vocal Bursts Valence Prediction

Progressive Training of Multi-level Wavelet Residual Networks for Image Denoising

2 code implementations23 Oct 2020 Yali Peng, Yue Cao, Shigang Liu, Jian Yang, WangMeng Zuo

To cope with this issue, this paper presents a multi-level wavelet residual network (MWRN) architecture as well as a progressive training (PTMWRN) scheme to improve image denoising performance.

Image Denoising

Learning Spatio-Appearance Memory Network for High-Performance Visual Tracking

1 code implementation21 Sep 2020 Fei Xie, Wankou Yang, Bo Liu, Kaihua Zhang, Wanli Xue, WangMeng Zuo

Existing visual object tracking usually learns a bounding-box based template to match the targets across frames, which cannot accurately learn a pixel-wise representation, thereby being limited in handling severe appearance variations.

Segmentation Semantic Segmentation +5

Unpaired Learning of Deep Image Denoising

2 code implementations ECCV 2020 Xiaohe Wu, Ming Liu, Yue Cao, Dongwei Ren, WangMeng Zuo

As for knowledge distillation, we first apply the learned noise models to clean images to synthesize a paired set of training images, and use the real noisy images and the corresponding denoising results in the first stage to form another paired set.

Image Denoising Knowledge Distillation +1

Plug-and-Play Image Restoration with Deep Denoiser Prior

4 code implementations31 Aug 2020 Kai Zhang, Yawei Li, WangMeng Zuo, Lei Zhang, Luc van Gool, Radu Timofte

Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems.

Deblurring Demosaicking +1

Learning Flow-based Feature Warping for Face Frontalization with Illumination Inconsistent Supervision

1 code implementation ECCV 2020 Yuxiang Wei, Ming Liu, Haolin Wang, Ruifeng Zhu, Guosheng Hu, WangMeng Zuo

Despite recent advances in deep learning-based face frontalization methods, photo-realistic and illumination preserving frontal face synthesis is still challenging due to large pose and illumination discrepancy during training.

Face Generation

Component Divide-and-Conquer for Real-World Image Super-Resolution

1 code implementation ECCV 2020 Pengxu Wei, Ziwei Xie, Hannan Lu, Zongyuan Zhan, Qixiang Ye, WangMeng Zuo, Liang Lin

Learning an SR model with conventional pixel-wise loss usually is easily dominated by flat regions and edges, and fails to infer realistic details of complex textures.

Image Super-Resolution

Blind Face Restoration via Deep Multi-scale Component Dictionaries

1 code implementation ECCV 2020 Xiaoming Li, Chaofeng Chen, Shangchen Zhou, Xianhui Lin, WangMeng Zuo, Lei Zhang

Next, with the degraded input, we match and select the most similar component features from their corresponding dictionaries and transfer the high-quality details to the input via the proposed dictionary feature transfer (DFT) block.

Blind Face Restoration Video Super-Resolution

Lightweight image super-resolution with enhanced CNN

1 code implementation8 Jul 2020 Chunwei Tian, Ruibin Zhuge, Zhihao Wu, Yong Xu, WangMeng Zuo, Chen Chen, Chia-Wen Lin

Finally, the IRB uses coarse high-frequency features from the RB to learn more accurate SR features and construct a SR image.

Image Super-Resolution

Designing and Training of A Dual CNN for Image Denoising

1 code implementation8 Jul 2020 Chunwei Tian, Yong Xu, WangMeng Zuo, Bo Du, Chia-Wen Lin, David Zhang

The enhancement block gathers and fuses the global and local features to provide complementary information for the latter network.

Image Denoising

Aligning Partially Overlapping Point Sets: an Inner Approximation Algorithm

no code implementations5 Jul 2020 Wei Lian, WangMeng Zuo, Lei Zhang

Our method is also $\epsilon-$globally optimal and thus is guaranteed to be robust.

Cross-Scale Internal Graph Neural Network for Image Super-Resolution

1 code implementation NeurIPS 2020 Shangchen Zhou, Jiawei Zhang, WangMeng Zuo, Chen Change Loy

Specifically, we dynamically construct a cross-scale graph by searching k-nearest neighboring patches in the downsampled LR image for each query patch in the LR image.

Image Restoration Image Super-Resolution

Flexible Image Denoising with Multi-layer Conditional Feature Modulation

1 code implementation24 Jun 2020 Jiazhi Du, Xin Qiao, Zifei Yan, Hongzhi Zhang, WangMeng Zuo

For flexible non-blind image denoising, existing deep networks usually take both noisy image and noise level map as the input to handle various noise levels with a single model.

Image Denoising

Dark and Bright Channel Prior Embedded Network for Dynamic Scene Deblurring

1 code implementation21 May 2020 Jianrui Cai, WangMeng Zuo, and Lei Zhang

In this work, we propose a Dark and Bright Channel Priors embedded Network (DBCPeNet) to plug the channel priors into a neural network for effective dynamic scene deblurring.

Ranked #31 on Image Deblurring on GoPro (using extra training data)

Deblurring Image Deblurring

Learning Context-Based Non-local Entropy Modeling for Image Compression

no code implementations10 May 2020 Mu Li, Kai Zhang, WangMeng Zuo, Radu Timofte, David Zhang

To address this issue, we propose a non-local operation for context modeling by employing the global similarity within the context.

Image Compression

Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation

6 code implementations7 May 2020 Zhaohui Zheng, Ping Wang, Dongwei Ren, Wei Liu, Rongguang Ye, QinGhua Hu, WangMeng Zuo

In this paper, we propose Complete-IoU (CIoU) loss and Cluster-NMS for enhancing geometric factors in both bounding box regression and Non-Maximum Suppression (NMS), leading to notable gains of average precision (AP) and average recall (AR), without the sacrifice of inference efficiency.

Clustering Instance Segmentation +6

Deep Adaptive Inference Networks for Single Image Super-Resolution

1 code implementation8 Apr 2020 Ming Liu, Zhilu Zhang, Liya Hou, WangMeng Zuo, Lei Zhang

Nonetheless, content and resource adaptive model is more preferred, and it is encouraging to apply simpler and efficient networks to the easier regions with less details and the scenarios with restricted efficiency constraints.

Image Super-Resolution

What Deep CNNs Benefit from Global Covariance Pooling: An Optimization Perspective

1 code implementation CVPR 2020 Qilong Wang, Li Zhang, Banggu Wu, Dongwei Ren, Peihua Li, WangMeng Zuo, QinGhua Hu

Recent works have demonstrated that global covariance pooling (GCP) has the ability to improve performance of deep convolutional neural networks (CNNs) on visual classification task.

Instance Segmentation object-detection +2

Towards Photo-Realistic Virtual Try-On by Adaptively Generating$\leftrightarrow$Preserving Image Content

3 code implementations12 Mar 2020 Han Yang, Ruimao Zhang, Xiaobao Guo, Wei Liu, WangMeng Zuo, Ping Luo

First, a semantic layout generation module utilizes semantic segmentation of the reference image to progressively predict the desired semantic layout after try-on.

Ranked #4 on Virtual Try-on on VITON (IS metric)

Semantic Segmentation Virtual Try-on

Deep Fusion Feature Representation Learning with Hard Mining Center-Triplet Loss for Person Re-identification

1 code implementation IEEE Transactions on Multimedia 2020 Cairong Zhao, Xinbi Lv, Zhang Zhang, WangMeng Zuo, Jun Wu, Duoqian Miao

The extraction of robust feature representations from pedestrian images through CNNs with a single deterministic pooling operation is problematic as the features in real pedestrian images are complex and diverse.

Person Re-Identification Representation Learning

Deep Learning on Image Denoising: An overview

no code implementations31 Dec 2019 Chunwei Tian, Lunke Fei, Wenxian Zheng, Yong Xu, WangMeng Zuo, Chia-Wen Lin

However, there are substantial differences in the various types of deep learning methods dealing with image denoising.

Image Denoising

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

12 code implementations CVPR 2020 Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, WangMeng Zuo, QinGhua Hu

By dissecting the channel attention module in SENet, we empirically show avoiding dimensionality reduction is important for learning channel attention, and appropriate cross-channel interaction can preserve performance while significantly decreasing model complexity.

Dimensionality Reduction Image Classification +4

Perspective-Guided Convolution Networks for Crowd Counting

1 code implementation ICCV 2019 Zhaoyi Yan, Yuchen Yuan, WangMeng Zuo, Xiao Tan, Yezhen Wang, Shilei Wen, Errui Ding

In this paper, we propose a novel perspective-guided convolution (PGC) for convolutional neural network (CNN) based crowd counting (i. e. PGCNet), which aims to overcome the dramatic intra-scene scale variations of people due to the perspective effect.

Crowd Counting

Image denoising using deep CNN with batch renormalization

2 code implementations Neural Networks 2019 Chunwei Tian, Yong Xu, WangMeng Zuo

In this paper, we report the design of a novel network called a batch-renormalization denoising network (BRDNet).

Image Denoising

Image Inpainting with Learnable Bidirectional Attention Maps

1 code implementation ICCV 2019 Chaohao Xie, Shaohui Liu, Chao Li, Ming-Ming Cheng, WangMeng Zuo, Xiao Liu, Shilei Wen, Errui Ding

Most convolutional network (CNN)-based inpainting methods adopt standard convolution to indistinguishably treat valid pixels and holes, making them limited in handling irregular holes and more likely to generate inpainting results with color discrepancy and blurriness.

Image Inpainting valid

Deep Concept-wise Temporal Convolutional Networks for Action Localization

2 code implementations26 Aug 2019 Xin Li, Tianwei Lin, Xiao Liu, Chuang Gan, WangMeng Zuo, Chao Li, Xiang Long, Dongliang He, Fu Li, Shilei Wen

In this paper, we empirically find that stacking more conventional temporal convolution layers actually deteriorates action classification performance, possibly ascribing to that all channels of 1D feature map, which generally are highly abstract and can be regarded as latent concepts, are excessively recombined in temporal convolution.

Action Classification Action Localization

Neural Blind Deconvolution Using Deep Priors

1 code implementation CVPR 2020 Dongwei Ren, Kai Zhang, Qilong Wang, QinGhua Hu, WangMeng Zuo

To connect MAP and deep models, we in this paper present two generative networks for respectively modeling the deep priors of clean image and blur kernel, and propose an unconstrained neural optimization solution to blind deconvolution.

Deblurring Self-Supervised Learning

Multi-level Wavelet Convolutional Neural Networks

3 code implementations6 Jul 2019 Pengju Liu, Hongzhi Zhang, Wei Lian, WangMeng Zuo

Specifically, MWCNN for image restoration is based on U-Net architecture, and inverse wavelet transform (IWT) is deployed to reconstruct the high resolution (HR) feature maps.

Computational Efficiency Image Denoising +2

Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compression

2 code implementations24 Jun 2019 Mu Li, Kede Ma, Jane You, David Zhang, WangMeng Zuo

For the former, we directly apply a CCN to the binarized representation of an image to compute the Bernoulli distribution of each code for entropy estimation.

Image Compression

Data Augmentation for Object Detection via Progressive and Selective Instance-Switching

1 code implementation2 Jun 2019 Hao Wang, Qilong Wang, Fan Yang, Weiqi Zhang, WangMeng Zuo

For guiding our IS to obtain better object performance, we explore issues of instance imbalance and class importance in datasets, which frequently occur and bring adverse effect on detection performance.

Data Augmentation Instance Segmentation +2

Remove Cosine Window from Correlation Filter-based Visual Trackers: When and How

1 code implementation16 May 2019 Feng Li, Xiaohe Wu, WangMeng Zuo, David Zhang, Lei Zhang

Therefore, we in this paper investigate the feasibility to remove cosine window from CF trackers with spatial regularization.

Spatio-Temporal Filter Adaptive Network for Video Deblurring

1 code implementation ICCV 2019 Shangchen Zhou, Jiawei Zhang, Jinshan Pan, Haozhe Xie, WangMeng Zuo, Jimmy Ren

To overcome the limitation of separate optical flow estimation, we propose a Spatio-Temporal Filter Adaptive Network (STFAN) for the alignment and deblurring in a unified framework.

Ranked #3 on Deblurring on DVD (using extra training data)

Deblurring Image Deblurring +1

DAVANet: Stereo Deblurring with View Aggregation

1 code implementation CVPR 2019 Shangchen Zhou, Jiawei Zhang, WangMeng Zuo, Haozhe Xie, Jinshan Pan, Jimmy Ren

Nowadays stereo cameras are more commonly adopted in emerging devices such as dual-lens smartphones and unmanned aerial vehicles.

Deblurring Image Deblurring

Blind Super-Resolution With Iterative Kernel Correction

3 code implementations CVPR 2019 Jinjin Gu, Hannan Lu, WangMeng Zuo, Chao Dong

In this paper, we propose an Iterative Kernel Correction (IKC) method for blur kernel estimation in blind SR problem, where the blur kernels are unknown.

Blind Super-Resolution Image Super-Resolution

Learning Content-Weighted Deep Image Compression

1 code implementation1 Apr 2019 Mu Li, WangMeng Zuo, Shuhang Gu, Jane You, David Zhang

Learning-based lossy image compression usually involves the joint optimization of rate-distortion performance.

Image Compression

Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernels

1 code implementation CVPR 2019 Kai Zhang, WangMeng Zuo, Lei Zhang

In this paper, we propose a principled formulation and framework by extending bicubic degradation based deep SISR with the help of plug-and-play framework to handle LR images with arbitrary blur kernels.

Deblurring Image Restoration +1

Manifold Criterion Guided Transfer Learning via Intermediate Domain Generation

1 code implementation25 Mar 2019 Lei Zhang, Shan-Shan Wang, Guang-Bin Huang, WangMeng Zuo, Jian Yang, David Zhang

The merits of the proposed MCTL are four-fold: 1) the concept of manifold criterion (MC) is first proposed as a measure validating the distribution matching across domains, and domain adaptation is achieved if the MC is satisfied; 2) the proposed MC can well guide the generation of the intermediate domain sharing similar distribution with the target domain, by minimizing the local domain discrepancy; 3) a global generative discrepancy metric (GGDM) is presented, such that both the global and local discrepancy can be effectively and positively reduced; 4) a simplified version of MCTL called MCTL-S is presented under a perfect domain generation assumption for more generic learning scenario.

Transfer Learning Unsupervised Domain Adaptation

Extreme Channel Prior Embedded Network for Dynamic Scene Deblurring

no code implementations2 Mar 2019 Jianrui Cai, WangMeng Zuo, Lei Zhang

In this work, we propose an Extreme Channel Prior embedded Network (ECPeNet) to plug the extreme channel priors (i. e., priors on dark and bright channels) into a network architecture for effective dynamic scene deblurring.

Deblurring Image Deblurring

Progressive Image Deraining Networks: A Better and Simpler Baseline

4 code implementations CVPR 2019 Dongwei Ren, WangMeng Zuo, QinGhua Hu, Pengfei Zhu, Deyu Meng

To handle this issue, this paper provides a better and simpler baseline deraining network by considering network architecture, input and output, and loss functions.

Image Super-Resolution Single Image Deraining +1

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net

no code implementations CVPR 2019 Qi Xie, Minghao Zhou, Qian Zhao, Deyu Meng, WangMeng Zuo, Zongben Xu

In this paper, we propose a model-based deep learning approach for merging an HrMS and LrHS images to generate a high-resolution hyperspectral (HrHS) image.

Learning Symmetry Consistent Deep CNNs for Face Completion

1 code implementation19 Dec 2018 Xiaoming Li, Ming Liu, Jieru Zhu, WangMeng Zuo, Meng Wang, Guosheng Hu, Lei Zhang

As for missing pixels on both of half-faces, we present a generative reconstruction subnet together with a perceptual symmetry loss to enforce symmetry consistency of recovered structures.

Face Recognition Facial Inpainting

Deep Non-Blind Deconvolution via Generalized Low-Rank Approximation

no code implementations NeurIPS 2018 Wenqi Ren, Jiawei Zhang, Lin Ma, Jinshan Pan, Xiaochun Cao, WangMeng Zuo, Wei Liu, Ming-Hsuan Yang

In this paper, we present a deep convolutional neural network to capture the inherent properties of image degradation, which can handle different kernels and saturated pixels in a unified framework.

Deblurring

Global Gated Mixture of Second-order Pooling for Improving Deep Convolutional Neural Networks

1 code implementation NeurIPS 2018 Qilong Wang, Zilin Gao, Jiangtao Xie, WangMeng Zuo, Peihua Li

However, both GAP and existing HOP methods assume unimodal distributions, which cannot fully capture statistics of convolutional activations, limiting representation ability of deep CNNs, especially for samples with complex contents.

Model Inconsistent but Correlated Noise: Multi-view Subspace Learning with Regularized Mixture of Gaussians

no code implementations7 Nov 2018 Hongwei Yong, Deyu Meng, Jinxing Li, WangMeng Zuo, Lei Zhang

Different from single view case, MSL should take both common and specific knowledge among different views into consideration.

Weakly-supervised Video Summarization using Variational Encoder-Decoder and Web Prior

no code implementations ECCV 2018 Sijia Cai, WangMeng Zuo, Larry S. Davis, Lei Zhang

Video summarization is a challenging under-constrained problem because the underlying summary of a single video strongly depends on users' subjective understandings.

Saliency Prediction Supervised Video Summarization

Convolutional Neural Networks based Intra Prediction for HEVC

no code implementations17 Aug 2018 Wenxue Cui, Tao Zhang, Shengping Zhang, Feng Jiang, WangMeng Zuo, Debin Zhao

To overcome this problem, in this paper, an intra prediction convolutional neural network (IPCNN) is proposed for intra prediction, which exploits the rich context of the current block and therefore is capable of improving the accuracy of predicting the current block.

Unsupervised/Semi-supervised Deep Learning for Low-dose CT Enhancement

no code implementations8 Aug 2018 Mingrui Geng, Yun Deng, Qian Zhao, Qi Xie, Dong Zeng, WangMeng Zuo, Deyu Meng

To address this issue, we propose an unsupervised DL method for LdCT enhancement that incorporates unlabeled LdCT sinograms directly into the network training.

Computational Efficiency

Joint Representation and Truncated Inference Learning for Correlation Filter based Tracking

no code implementations ECCV 2018 Yingjie Yao, Xiaohe Wu, Lei Zhang, Shiguang Shan, WangMeng Zuo

In existing off-line deep learning models for CF trackers, the model adaptation usually is either abandoned or has closed-form solution to make it feasible to learn deep representation in an end-to-end manner.

Scaled Simplex Representation for Subspace Clustering

3 code implementations26 Jul 2018 Jun Xu, Mengyang Yu, Ling Shao, WangMeng Zuo, Deyu Meng, Lei Zhang, David Zhang

However, the negative entries in the coefficient matrix are forced to be positive when constructing the affinity matrix via exponentiation, absolute symmetrization, or squaring operations.

Clustering

Identity Preserving Face Completion for Large Ocular Region Occlusion

no code implementations23 Jul 2018 Yajie Zhao, Weikai Chen, Jun Xing, Xiaoming Li, Zach Bessinger, Fuchang Liu, WangMeng Zuo, Ruigang Yang

Different from the state-of-the-art face inpainting methods that have no control over the synthesized content and can only handle frontal face pose, our approach can faithfully recover the missing content under various head poses while preserving the identity.

Facial Inpainting

Toward Convolutional Blind Denoising of Real Photographs

3 code implementations CVPR 2019 Shi Guo, Zifei Yan, Kai Zhang, WangMeng Zuo, Lei Zhang

While deep convolutional neural networks (CNNs) have achieved impressive success in image denoising with additive white Gaussian noise (AWGN), their performance remains limited on real-world noisy photographs.

Image Denoising Noise Estimation

Generative Adversarial Learning Towards Fast Weakly Supervised Detection

no code implementations CVPR 2018 Yunhan Shen, Rongrong Ji, Shengchuan Zhang, WangMeng Zuo, Yan Wang

Without the need of annotating bounding boxes, the existing methods usually follow a two/multi-stage pipeline with an online compulsive stage to extract object proposals, which is an order of magnitude slower than fast fully supervised object detectors such as SSD [31] and YOLO [34].

Object object-detection +1

Multi-level Wavelet-CNN for Image Restoration

5 code implementations18 May 2018 Pengju Liu, Hongzhi Zhang, Kai Zhang, Liang Lin, WangMeng Zuo

With the modified U-Net architecture, wavelet transform is introduced to reduce the size of feature maps in the contracting subnetwork.

Computational Efficiency Image Denoising +2

Learning Warped Guidance for Blind Face Restoration

1 code implementation ECCV 2018 Xiaoming Li, Ming Liu, Yuting Ye, WangMeng Zuo, Liang Lin, Ruigang Yang

For better recovery of fine facial details, we modify the problem setting by taking both the degraded observation and a high-quality guided image of the same identity as input to our guided face restoration network (GFRNet).

Blind Face Restoration

Simultaneous Fidelity and Regularization Learning for Image Restoration

1 code implementation12 Apr 2018 Dongwei Ren, WangMeng Zuo, David Zhang, Lei Zhang, Ming-Hsuan Yang

For blind deconvolution, as estimation error of blur kernel is usually introduced, the subsequent non-blind deconvolution process does not restore the latent image well.

Denoising Image Deconvolution +1

VITAL: VIsual Tracking via Adversarial Learning

no code implementations CVPR 2018 Yibing Song, Chao Ma, Xiaohe Wu, Lijun Gong, Linchao Bao, WangMeng Zuo, Chunhua Shen, Rynson Lau, Ming-Hsuan Yang

To augment positive samples, we use a generative network to randomly generate masks, which are applied to adaptively dropout input features to capture a variety of appearance changes.

General Classification Visual Tracking

Multi-views Fusion CNN for Left Ventricular Volumes Estimation on Cardiac MR Images

1 code implementation9 Apr 2018 Gongning Luo, Suyu Dong, Kuanquan Wang, WangMeng Zuo, Shaodong Cao, Henggui Zhang

Methods: In this paper, we propose a direct volumes prediction method based on the end-to-end deep convolutional neural networks (CNN).

Multi-scale Location-aware Kernel Representation for Object Detection

2 code implementations CVPR 2018 Hao Wang, Qilong Wang, Mingqi Gao, Peihua Li, WangMeng Zuo

Our MLKP can be efficiently computed on a modified multi-scale feature map using a low-dimensional polynomial kernel approximation. Moreover, different from existing orderless global representations based on high-order statistics, our proposed MLKP is location retentive and sensitive so that it can be flexibly adopted to object detection.

General Classification Object +2

Metric Learning with Dynamically Generated Pairwise Constraints for Ear Recognition

no code implementations26 Mar 2018 Ibrahim Omara, Hongzhi Zhang, Faqiang Wang, WangMeng Zuo

Ear recognition task is known as predicting whether two ear images belong to the same person or not.

Metric Learning

Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking

1 code implementation CVPR 2018 Feng Li, Cheng Tian, WangMeng Zuo, Lei Zhang, Ming-Hsuan Yang

Compared with SRDCF, STRCF with hand-crafted features provides a 5 times speedup and achieves a gain of 5. 4% and 3. 6% AUC score on OTB-2015 and Temple-Color, respectively.

Visual Object Tracking Visual Tracking

Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift

1 code implementation CVPR 2018 Ruijia Xu, Ziliang Chen, WangMeng Zuo, Junjie Yan, Liang Lin

Motivated by the theoretical results in \cite{mansour2009domain}, the target distribution can be represented as the weighted combination of source distributions, and, the multi-source unsupervised domain adaptation via DCTN is then performed as two alternating steps: i) It deploys multi-way adversarial learning to minimize the discrepancy between the target and each of the multiple source domains, which also obtains the source-specific perplexity scores to denote the possibilities that a target sample belongs to different source domains.

Multi-Source Unsupervised Domain Adaptation Unsupervised Domain Adaptation

Shift-Net: Image Inpainting via Deep Feature Rearrangement

2 code implementations ECCV 2018 Zhaoyi Yan, Xiaoming Li, Mu Li, WangMeng Zuo, Shiguang Shan

To this end, the encoder feature of the known region is shifted to serve as an estimation of the missing parts.

Image Inpainting

Enlarging Context with Low Cost: Efficient Arithmetic Coding with Trimmed Convolution

no code implementations15 Jan 2018 Mu Li, Shuhang Gu, David Zhang, WangMeng Zuo

One key issue of arithmetic encoding method is to predict the probability of the current coding symbol from its context, i. e., the preceding encoded symbols, which usually can be executed by building a look-up table (LUT).

Computational Efficiency Image Compression

Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networks

2 code implementations20 Dec 2017 Tianshui Chen, Liang Lin, WangMeng Zuo, Xiaonan Luo, Lei Zhang

In this work, aiming at a general and comprehensive way for neural network acceleration, we develop a Wavelet-like Auto-Encoder (WAE) that decomposes the original input image into two low-resolution channels (sub-images) and incorporate the WAE into the classification neural networks for joint training.

Classification General Classification +1

AttGAN: Facial Attribute Editing by Only Changing What You Want

10 code implementations29 Nov 2017 Zhenliang He, WangMeng Zuo, Meina Kan, Shiguang Shan, Xilin Chen

Based on the encoder-decoder architecture, facial attribute editing is achieved by decoding the latent representation of the given face conditioned on the desired attributes.

Attribute

FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising

6 code implementations11 Oct 2017 Kai Zhang, WangMeng Zuo, Lei Zhang

Due to the fast inference and good performance, discriminative learning methods have been widely studied in image denoising.

Color Image Denoising Image Denoising

Visual Tracking via Dynamic Graph Learning

no code implementations4 Oct 2017 Chenglong Li, Liang Lin, WangMeng Zuo, Jin Tang, Ming-Hsuan Yang

First, the graph is initialized by assigning binary weights of some image patches to indicate the object and background patches according to the predicted bounding box.

Graph Learning Object +2

Joint Convolutional Analysis and Synthesis Sparse Representation for Single Image Layer Separation

no code implementations ICCV 2017 Shuhang Gu, Deyu Meng, WangMeng Zuo, Lei Zhang

To exploit the complementary representation mechanisms of ASR and SSR, we integrate the two models and propose a joint convolutional analysis and synthesis (JCAS) sparse representation model.

Tone Mapping

Higher-Order Integration of Hierarchical Convolutional Activations for Fine-Grained Visual Categorization

no code implementations ICCV 2017 Sijia Cai, WangMeng Zuo, Lei Zhang

The success of fine-grained visual categorization (FGVC) extremely relies on the modeling of appearance and interactions of various semantic parts.

Fine-Grained Visual Categorization

Hierarchical Scene Parsing by Weakly Supervised Learning with Image Descriptions

no code implementations27 Sep 2017 Ruimao Zhang, Liang Lin, Guangrun Wang, Meng Wang, WangMeng Zuo

Rather than relying on elaborative annotations (e. g., manually labeled semantic maps and relations), we train our deep model in a weakly-supervised learning manner by leveraging the descriptive sentences of the training images.

Descriptive Object +4

Learning Dynamic Guidance for Depth Image Enhancement

no code implementations CVPR 2017 Shuhang Gu, WangMeng Zuo, Shi Guo, Yunjin Chen, Chongyu Chen, Lei Zhang

To address these limitations, we propose a weighted analysis representation model for guided depth image enhancement, which advances the conventional methods in two aspects: (i) task driven learning and (ii) dynamic guidance.

Depth Image Upsampling Image Enhancement +1

Robust Online Matrix Factorization for Dynamic Background Subtraction

no code implementations28 May 2017 Hongwei Yong, Deyu Meng, WangMeng Zuo, Lei Zhang

We propose an effective online background subtraction method, which can be robustly applied to practical videos that have variations in both foreground and background.

Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation

3 code implementations CVPR 2017 Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, WangMeng Zuo

Specifically, we introduce class-specific auxiliary weights into the original MMD for exploiting the class prior probability on source and target domains, whose challenge lies in the fact that the class label in target domain is unavailable.

Unsupervised Domain Adaptation

Learning Deep CNN Denoiser Prior for Image Restoration

2 code implementations CVPR 2017 Kai Zhang, WangMeng Zuo, Shuhang Gu, Lei Zhang

Recent works have revealed that, with the aid of variable splitting techniques, denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other inverse problems (e. g., deblurring).

Color Image Denoising Deblurring +2

Cannot find the paper you are looking for? You can Submit a new open access paper.