Search Results for author: Yulun Zhang

Found 66 papers, 43 papers with code

LatticeNet: Towards Lightweight Image Super-resolution with Lattice Block

1 code implementation ECCV 2020 Xiaotong Luo, Yuan Xie, Yulun Zhang, Yanyun Qu, Cuihua Li, Yun Fu

Drawing lessons from lattice filter bank, we design the lattice block (LB) in which two butterfly structures are applied to combine two RBs.

Image Super-Resolution Single Image Super Resolution

No One Left Behind: Real-World Federated Class-Incremental Learning

1 code implementation2 Feb 2023 Jiahua Dong, Yang Cong, Gan Sun, Yulun Zhang, Bernt Schiele, Dengxin Dai

To tackle the above issues, we propose a novel Local-Global Anti-forgetting (LGA) model to address local and global catastrophic forgetting on old categories, which is a pioneering work to explore a global class-incremental model in the FL feld.

class-incremental learning Federated Learning +1

Spatial-Spectral Transformer for Hyperspectral Image Denoising

2 code implementations25 Nov 2022 Miaoyu Li, Ying Fu, Yulun Zhang

Hyperspectral image (HSI) denoising is a crucial preprocessing procedure for the subsequent HSI applications.

Hyperspectral Image Denoising Image Denoising

Cross Aggregation Transformer for Image Restoration

1 code implementation24 Nov 2022 Chen Zheng, Yulun Zhang, Jinjin Gu, Yongbing Zhang, Linghe Kong, Xin Yuan

The core of our CAT is the Rectangle-Window Self-Attention (Rwin-SA), which utilizes horizontal and vertical rectangle window attention in different heads parallelly to expand the attention area and aggregate the features cross different windows.

Image Restoration Inductive Bias

Super-Resolution by Predicting Offsets: An Ultra-Efficient Super-Resolution Network for Rasterized Images

no code implementations9 Oct 2022 Jinjin Gu, Haoming Cai, Chenyu Dong, Ruofan Zhang, Yulun Zhang, Wenming Yang, Chun Yuan

We finally use a guided fusion operation to integrate the sharp edges generated by the network and flat areas by the interpolation method to get the final SR image.

Quantization Super-Resolution

Accurate Image Restoration with Attention Retractable Transformer

1 code implementation4 Oct 2022 Jiale Zhang, Yulun Zhang, Jinjin Gu, Yongbing Zhang, Linghe Kong, Xin Yuan

This is considered as a dense attention strategy since the interactions of tokens are restrained in dense regions.

Denoising Image Restoration +2

Basic Binary Convolution Unit for Binarized Image Restoration Network

no code implementations2 Oct 2022 Bin Xia, Yulun Zhang, Yitong Wang, Yapeng Tian, Wenming Yang, Radu Timofte, Luc van Gool

In this study, we reconsider components in binary convolution, such as residual connection, BatchNorm, activation function, and structure, for IR tasks.

Binarization Image Restoration +1

S^2-Transformer for Mask-Aware Hyperspectral Image Reconstruction

1 code implementation24 Sep 2022 Jiamian Wang, Kunpeng Li, Yulun Zhang, Xin Yuan, Zhiqiang Tao

By observing this physical encoding procedure, two major challenges stand in the way of a high-fidelity reconstruction.

Image Reconstruction

Practical Real Video Denoising with Realistic Degradation Model

no code implementations25 Aug 2022 JieZhang Cao, Qin Wang, Jingyun Liang, Yulun Zhang, Kai Zhang, Luc van Gool

Although some studies attempt to train deep models on noisy and noise-free video pairs captured by cameras, such models can only work well for specific cameras and do not generalize well for other videos.

Denoising Video Denoising

Reference-based Image Super-Resolution with Deformable Attention Transformer

1 code implementation25 Jul 2022 JieZhang Cao, Jingyun Liang, Kai Zhang, Yawei Li, Yulun Zhang, Wenguan Wang, Luc van Gool

Reference-based image super-resolution (RefSR) aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images.

Image Super-Resolution

Towards Interpretable Video Super-Resolution via Alternating Optimization

1 code implementation21 Jul 2022 JieZhang Cao, Jingyun Liang, Kai Zhang, Wenguan Wang, Qin Wang, Yulun Zhang, Hao Tang, Luc van Gool

These issues can be alleviated by a cascade of three separate sub-tasks, including video deblurring, frame interpolation, and super-resolution, which, however, would fail to capture the spatial and temporal correlations among video sequences.

Deblurring Space-time Video Super-resolution +2

Structured Sparsity Learning for Efficient Video Super-Resolution

no code implementations15 Jun 2022 Bin Xia, Jingwen He, Yulun Zhang, Yitong Wang, Yapeng Tian, Wenming Yang, Luc van Gool

In SSL, we design pruning schemes for several key components in VSR models, including residual blocks, recurrent networks, and upsampling networks.

Video Super-Resolution

Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

1 code implementation20 May 2022 Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Henghui Ding, Yulun Zhang, Radu Timofte, Luc van Gool

In coded aperture snapshot spectral compressive imaging (CASSI) systems, hyperspectral image (HSI) reconstruction methods are employed to recover the spatial-spectral signal from a compressed measurement.

Compressive Sensing Image Reconstruction +1

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction

1 code implementation17 Apr 2022 Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Radu Timofte, Luc van Gool

Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI).

Spectral Reconstruction Spectral Super-Resolution

Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training

2 code implementations NeurIPS 2021 Yuanhao Cai, Xiaowan Hu, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Donglai Wei

Additionally, for better noise fitting, we present an efficient architecture Simple Multi-scale Network (SMNet) as the generator.

Ranked #2 on Image Denoising on SIDD (using extra training data)

Image Denoising Image Generation

Cross-Modality High-Frequency Transformer for MR Image Super-Resolution

no code implementations29 Mar 2022 Chaowei Fang, Dingwen Zhang, Liang Wang, Yulun Zhang, Lechao Cheng, Junwei Han

Improving the resolution of magnetic resonance (MR) image data is critical to computer-aided diagnosis and brain function analysis.

Image Super-Resolution

Practical Blind Denoising via Swin-Conv-UNet and Data Synthesis

1 code implementation24 Mar 2022 Kai Zhang, Yawei Li, Jingyun Liang, JieZhang Cao, Yulun Zhang, Hao Tang, Radu Timofte, Luc van Gool

While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved.

Image Denoising Image-to-Image Translation

Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution

1 code implementation16 Mar 2022 Bin Sun, Yulun Zhang, Songyao Jiang, Yun Fu

In this paper, we propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing an efficient and effective downsampling module into the SR task.

Image Super-Resolution

Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction

1 code implementation9 Mar 2022 Yuanhao Cai, Jing Lin, Xiaowan Hu, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, Luc van Gool

Many algorithms have been developed to solve the inverse problem of coded aperture snapshot spectral imaging (CASSI), i. e., recovering the 3D hyperspectral images (HSIs) from a 2D compressive measurement.

Compressive Sensing Image Reconstruction

HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging

2 code implementations CVPR 2022 Xiaowan Hu, Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, Luc van Gool

On the one hand, the proposed HR spatial-spectral attention module with its efficient feature fusion provides continuous and fine pixel-level features.

Compressive Sensing Image Reconstruction +1

Texture-Based Error Analysis for Image Super-Resolution

no code implementations CVPR 2022 Salma Abdel Magid, Zudi Lin, Donglai Wei, Yulun Zhang, Jinjin Gu, Hanspeter Pfister

Our key contribution is to leverage a texture classifier, which enables us to assign patches with semantic labels, to identify the source of SR errors both globally and locally.

Image Super-Resolution SSIM

Modeling Mask Uncertainty in Hyperspectral Image Reconstruction

1 code implementation31 Dec 2021 Jiamian Wang, Yulun Zhang, Xin Yuan, Ziyi Meng, Zhiqiang Tao

Recently, hyperspectral imaging (HSI) has attracted increasing research attention, especially for the ones based on a coded aperture snapshot spectral imaging (CASSI) system.

Bilevel Optimization Image Reconstruction

Deep Surrogate Assisted MAP-Elites for Automated Hearthstone Deckbuilding

1 code implementation7 Dec 2021 Yulun Zhang, Matthew C. Fontaine, Amy K. Hoover, Stefanos Nikolaidis

In a Hearthstone deckbuilding case study, we show that our approach improves the sample efficiency of MAP-Elites and outperforms a model trained offline with random decks, as well as a linear surrogate model baseline, setting a new state-of-the-art for quality diversity approaches in automated Hearthstone deckbuilding.

Aligned Structured Sparsity Learning for Efficient Image Super-Resolution

1 code implementation NeurIPS 2021 Yulun Zhang, Huan Wang, Can Qin, Yun Fu

To address the above issues, we propose aligned structured sparsity learning (ASSL), which introduces a weight normalization layer and applies $L_2$ regularization to the scale parameters for sparsity.

Image Super-Resolution Knowledge Distillation +3

Slow Learning and Fast Inference: Efficient Graph Similarity Computation via Knowledge Distillation

1 code implementation NeurIPS 2021 Can Qin, Handong Zhao, Lichen Wang, Huan Wang, Yulun Zhang, Yun Fu

For slow learning of graph similarity, this paper proposes a novel early-fusion approach by designing a co-attention-based feature fusion network on multilevel GNN features.

Anomaly Detection Graph Similarity +3

A Simple Approach to Continual Learning by Transferring Skill Parameters

no code implementations19 Oct 2021 K. R. Zentner, Ryan Julian, Ujjwal Puri, Yulun Zhang, Gaurav S. Sukhatme

We take a fresh look at this problem, by considering a setting in which the robot is limited to storing that knowledge and experience only in the form of learned skill policies.

Continual Learning

A NEW BACKBONE FOR HYPERSPECTRAL IMAGE RECONSTRUCTION

no code implementations29 Sep 2021 Jiamian Wang, Yulun Zhang, Xin Yuan, Yun Fu, Zhiqiang Tao

As the inverse process of snapshot compressive imaging, the hyperspectral image (HSI) reconstruction takes the 2D measurement as input and posteriorly retrieves the captured 3D spatial-spectral signal.

Image Reconstruction

MemREIN: Rein the Domain Shift for Cross-Domain Few-Shot Learning

no code implementations29 Sep 2021 Yi Xu, Lichen Wang, Yizhou Wang, Can Qin, Yulun Zhang, Yun Fu

In this paper, we propose a novel framework, MemREIN, which considers Memorized, Restitution, and Instance Normalization for cross-domain few-shot learning.

Contrastive Learning cross-domain few-shot learning

A Simple and Efficient Reconstruction Backbone for Snapshot Compressive Imaging

1 code implementation17 Aug 2021 Jiamian Wang, Yulun Zhang, Xin Yuan, Yun Fu, Zhiqiang Tao

The emerging technology of snapshot compressive imaging (SCI) enables capturing high dimensional (HD) data in an efficient way.

Compressive Sensing Image Reconstruction +3

Towards Exploiting Geometry and Time for Fast Off-Distribution Adaptation in Multi-Task Robot Learning

no code implementations24 Jun 2021 K. R. Zentner, Ryan Julian, Ujjwal Puri, Yulun Zhang, Gaurav Sukhatme

We explore possible methods for multi-task transfer learning which seek to exploit the shared physical structure of robotics tasks.

Transfer Learning

Rethinking Adam: A Twofold Exponential Moving Average Approach

no code implementations22 Jun 2021 Yizhou Wang, Yue Kang, Can Qin, Huan Wang, Yi Xu, Yulun Zhang, Yun Fu

The intuition is that gradient with momentum contains more accurate directional information and therefore its second moment estimation is a more favorable option for learning rate scaling than that of the raw gradient.

On the Importance of Environments in Human-Robot Coordination

2 code implementations21 Jun 2021 Matthew C. Fontaine, Ya-Chuan Hsu, Yulun Zhang, Bryon Tjanaka, Stefanos Nikolaidis

When studying robots collaborating with humans, much of the focus has been on robot policies that coordinate fluently with human teammates in collaborative tasks.

Pseudo 3D Auto-Correlation Network for Real Image Denoising

no code implementations CVPR 2021 Xiaowan Hu, Ruijun Ma, Zhihong Liu, Yuanhao Cai, Xiaole Zhao, Yulun Zhang, Haoqian Wang

The extraction of auto-correlation in images has shown great potential in deep learning networks, such as the self-attention mechanism in the channel domain and the self-similarity mechanism in the spatial domain.

Image Denoising

MR Image Super-Resolution With Squeeze and Excitation Reasoning Attention Network

no code implementations CVPR 2021 Yulun Zhang, Kai Li, Kunpeng Li, Yun Fu

They also fail to sense the entire space of the input, which is critical for high-quality MR image SR. To address those problems, we propose squeeze and excitation reasoning attention networks (SERAN) for accurate MR image SR. We propose to squeeze attention from global spatial information of the input and obtain global descriptors.

Image Super-Resolution

ECACL: A Holistic Framework for Semi-Supervised Domain Adaptation

1 code implementation ICCV 2021 Kai Li, Chang Liu, Handong Zhao, Yulun Zhang, Yun Fu

This paper studies Semi-Supervised Domain Adaptation (SSDA), a practical yet under-investigated research topic that aims to learn a model of good performance using unlabeled samples and a few labeled samples in the target domain, with the help of labeled samples from a source domain.

Data Augmentation Domain Adaptation

Recent Advances on Neural Network Pruning at Initialization

2 code implementations11 Mar 2021 Huan Wang, Can Qin, Yue Bai, Yulun Zhang, Yun Fu

Neural network pruning typically removes connections or neurons from a pretrained converged model; while a new pruning paradigm, pruning at initialization (PaI), attempts to prune a randomly initialized network.

Network Pruning

GAN Inversion: A Survey

1 code implementation14 Jan 2021 Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, Ming-Hsuan Yang

GAN inversion aims to invert a given image back into the latent space of a pretrained GAN model, for the image to be faithfully reconstructed from the inverted code by the generator.

Image Manipulation Image Restoration

Dynamic High-Pass Filtering and Multi-Spectral Attention for Image Super-Resolution

no code implementations ICCV 2021 Salma Abdel Magid, Yulun Zhang, Donglai Wei, Won-Dong Jang, Zudi Lin, Yun Fu, Hanspeter Pfister

Specifically, we propose a dynamic high-pass filtering (HPF) module that locally applies adaptive filter weights for each spatial location and channel group to preserve high-frequency signals.

Image Super-Resolution

Context Reasoning Attention Network for Image Super-Resolution

no code implementations ICCV 2021 Yulun Zhang, Donglai Wei, Can Qin, Huan Wang, Hanspeter Pfister, Yun Fu

However, the basic convolutional layer in CNNs is designed to extract local patterns, lacking the ability to model global context.

Image Super-Resolution

Neural Sparse Representation for Image Restoration

1 code implementation NeurIPS 2020 Yuchen Fan, Jiahui Yu, Yiqun Mei, Yulun Zhang, Yun Fu, Ding Liu, Thomas S. Huang

Inspired by the robustness and efficiency of sparse representation in sparse coding based image restoration models, we investigate the sparsity of neurons in deep networks.

Image Compression Image Denoising +2

Pyramid Attention Networks for Image Restoration

2 code implementations28 Apr 2020 Yiqun Mei, Yuchen Fan, Yulun Zhang, Jiahui Yu, Yuqian Zhou, Ding Liu, Yun Fu, Thomas S. Huang, Humphrey Shi

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales.

Demosaicking Image Denoising +1

Adversarial Feature Hallucination Networks for Few-Shot Learning

no code implementations CVPR 2020 Kai Li, Yulun Zhang, Kunpeng Li, Yun Fu

The recent flourish of deep learning in various tasks is largely accredited to the rich and accessible labeled data.

Data Augmentation Few-Shot Learning

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

3 code implementations CVPR 2020 Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P. Allebach, Chenliang Xu

Rather than synthesizing missing LR video frames as VFI networks do, we firstly temporally interpolate LR frame features in missing LR video frames capturing local temporal contexts by the proposed feature temporal interpolation network.

Space-time Video Super-resolution Video Frame Interpolation +1

Texture Hallucination for Large-Factor Painting Super-Resolution

no code implementations ECCV 2020 Yulun Zhang, Zhifei Zhang, Stephen DiVerdi, Zhaowen Wang, Jose Echevarria, Yun Fu

We aim to super-resolve digital paintings, synthesizing realistic details from high-resolution reference painting materials for very large scaling factors (e. g., 8X, 16X).

Image Reconstruction Image Super-Resolution +1

Joint Super-Resolution and Alignment of Tiny Faces

1 code implementation19 Nov 2019 Yu Yin, Joseph P. Robinson, Yulun Zhang, Yun Fu

As for SR, the proposed method recovers sharper edges and more details from LR face images than other state-of-the-art methods, which we demonstrate qualitatively and quantitatively.

Super-Resolution

Visual Semantic Reasoning for Image-Text Matching

2 code implementations ICCV 2019 Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, Yun Fu

It outperforms the current best method by 6. 8% relatively for image retrieval and 4. 8% relatively for caption retrieval on MS-COCO (Recall@1 using 1K test set).

Cross-Modal Retrieval Image Retrieval +2

FC$^2$N: Fully Channel-Concatenated Network for Single Image Super-Resolution

1 code implementation7 Jul 2019 Xiaole Zhao, Ying Liao, Tian He, Yulun Zhang, Yadong Wu, Tao Zhang

Most current image super-resolution (SR) methods based on convolutional neural networks (CNNs) use residual learning in network structural design, which favors to effective back propagation and hence improves SR performance by increasing model scale.

Image Super-Resolution Single Image Super Resolution

Multimodal Style Transfer via Graph Cuts

2 code implementations ICCV 2019 Yulun Zhang, Chen Fang, Yilin Wang, Zhaowen Wang, Zhe Lin, Yun Fu, Jimei Yang

An assumption widely used in recent neural style transfer methods is that image styles can be described by global statics of deep features like Gram or covariance matrices.

Style Transfer

Residual Non-local Attention Networks for Image Restoration

2 code implementations ICLR 2019 Yulun Zhang, Kunpeng Li, Kai Li, Bineng Zhong, Yun Fu

To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts.

Demosaicking Image Denoising +1

TDAN: Temporally Deformable Alignment Network for Video Super-Resolution

2 code implementations7 Dec 2018 Yapeng Tian, Yulun Zhang, Yun Fu, Chenliang Xu

Video super-resolution (VSR) aims to restore a photo-realistic high-resolution (HR) video frame from both its corresponding low-resolution (LR) frame (reference frame) and multiple neighboring frames (supporting frames).

Optical Flow Estimation Video Super-Resolution

Channel Splitting Network for Single MR Image Super-Resolution

no code implementations15 Oct 2018 Xiaole Zhao, Yulun Zhang, Tao Zhang, Xueming Zou

The proposed CSN model divides the hierarchical features into two branches, i. e., residual branch and dense branch, with different information transmissions.

Image Super-Resolution Single Image Super Resolution

Support Neighbor Loss for Person Re-Identification

1 code implementation18 Aug 2018 Kai Li, Zhengming Ding, Kunpeng Li, Yulun Zhang, Yun Fu

To ensure scalability and separability, a softmax-like function is formulated to push apart the positive and negative support sets.

Person Re-Identification

Residual Dense Network for Image Super-Resolution

13 code implementations CVPR 2018 Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, Yun Fu

In this paper, we propose a novel residual dense network (RDN) to address this problem in image SR. We fully exploit the hierarchical features from all the convolutional layers.

Color Image Denoising Image Super-Resolution

Cannot find the paper you are looking for? You can Submit a new open access paper.