Search Results for author: Chen Change Loy

Found 151 papers, 92 papers with code

Investigating Tradeoffs in Real-World Video Super-Resolution

1 code implementation24 Nov 2021 Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy

The diversity and complexity of degradations in real-world video super-resolution (VSR) pose non-trivial challenges in inference and training.

Video Super-Resolution

Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data

2 code implementations NeurIPS 2021 Liming Jiang, Bo Dai, Wayne Wu, Chen Change Loy

Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images.

Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements

no code implementations1 Nov 2021 Yu Rong, Jingbo Wang, Ziwei Liu, Chen Change Loy

In this paper, we make the first attempt to reconstruct 3D interacting hands from monocular single RGB images.

3D Reconstruction

A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis

1 code implementation NeurIPS 2021 Xingang Pan, Xudong Xu, Chen Change Loy, Christian Theobalt, Bo Dai

Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images.

3D-Aware Image Synthesis 3D Shape Reconstruction +1

STransGAN: An Empirical Study on Transformer in GANs

no code implementations25 Oct 2021 Rui Xu, Xiangyu Xu, Kai Chen, Bolei Zhou, Chen Change Loy

In this paper, we conduct a comprehensive empirical study to investigate the intrinsic properties of Transformer in GAN for high-fidelity image synthesis.

Image Generation

Self-Supervised Representation Learning: Introduction, Advances and Challenges

no code implementations18 Oct 2021 Linus Ericsson, Henry Gouk, Chen Change Loy, Timothy M. Hospedales

Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation bottleneck that is one of the main barriers to practical deployment of deep learning today.

Representation Learning

Playing for 3D Human Recovery

no code implementations14 Oct 2021 Zhongang Cai, Mingyuan Zhang, Jiawei Ren, Chen Wei, Daxuan Ren, Jiatong Li, Zhengyu Lin, Haiyu Zhao, Shuai Yi, Lei Yang, Chen Change Loy, Ziwei Liu

Image- and video-based 3D human recovery (i. e. pose and shape estimation) have achieved substantial progress.

Motion Capture

Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning

no code implementations9 Oct 2021 Yihao Liu, Hengyuan Zhao, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Yu Qiao, Chao Dong

We address this problem from a new perspective, by jointly considering colorization and temporal consistency in a unified framework.

Colorization

ReconfigISP: Reconfigurable Camera Image Processing Pipeline

no code implementations ICCV 2021 Ke Yu, Zexian Li, Yue Peng, Chen Change Loy, Jinwei Gu

Image Signal Processor (ISP) is a crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand.

Image Restoration Neural Architecture Search +1

Talk-to-Edit: Fine-Grained Facial Editing via Dialog

1 code implementation ICCV 2021 Yuming Jiang, Ziqi Huang, Xingang Pan, Chen Change Loy, Ziwei Liu

In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.

Fine-Grained Facial Editing

3D Human Texture Estimation from a Single Image with Transformers

1 code implementation ICCV 2021 Xiangyu Xu, Chen Change Loy

We propose a Transformer-based framework for 3D human texture estimation from a single image.

Learning to Prompt for Vision-Language Models

1 code implementation2 Sep 2021 Kaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei Liu

It shifts from the tradition of using images and discrete labels for learning a fixed set of weights, seen as visual concepts, to aligning images and raw text for two separate encoders.

Representation Learning

K-Net: Towards Unified Image Segmentation

1 code implementation NeurIPS 2021 Wenwei Zhang, Jiangmiao Pang, Kai Chen, Chen Change Loy

The framework, named K-Net, segments both instances and semantic categories consistently by a group of learnable kernels, where each kernel is responsible for generating a mask for either a potential instance or a stuff class.

Instance Segmentation Panoptic Segmentation

Unsupervised Object-Level Representation Learning from Scene Images

no code implementations NeurIPS 2021 Jiahao Xie, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy

Extensive experiments on COCO show that ORL significantly improves the performance of self-supervised learning on scene images, even surpassing supervised ImageNet pre-training on several downstream tasks.

Self-Supervised Learning Semantic correspondence +1

Pareidolia Face Reenactment

no code implementations CVPR 2021 Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, Ran He

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.

Face Reenactment Texture Synthesis

Robust Reference-based Super-Resolution via C2-Matching

1 code implementation CVPR 2021 Yuming Jiang, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Ziwei Liu

However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e. g. scale and rotation) and the resolution gap (e. g. HR and LR).

Super-Resolution

Semi-Supervised Domain Generalization with Stochastic StyleMatch

2 code implementations1 Jun 2021 Kaiyang Zhou, Chen Change Loy, Ziwei Liu

Our proposed approach, StyleMatch, is inspired by FixMatch, a state-of-the-art semi-supervised learning method based on pseudo-labeling, with several new ingredients tailored to solve SSDG.

Domain Generalization

Unsupervised 3D Shape Completion through GAN Inversion

no code implementations CVPR 2021 Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy

In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time.

GAN inversion

BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

1 code implementation27 Apr 2021 Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy

We show that by empowering the recurrent framework with the enhanced propagation and alignment, one can exploit spatiotemporal information across misaligned video frames more effectively.

Video Enhancement Video Restoration +1

Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation

1 code implementation CVPR 2021 Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, Ziwei Liu

While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.

Talking Face Generation

Low-Light Image and Video Enhancement Using Deep Learning: A Survey

2 code implementations21 Apr 2021 Chongyi Li, Chunle Guo, Linghao Han, Jun Jiang, Ming-Ming Cheng, Jinwei Gu, Chen Change Loy

Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination.

Face Detection Low-Light Image Enhancement +1

Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network

1 code implementation CVPR 2021 Ruicheng Feng, Chongyi Li, Huaijin Chen, Shuai Li, Chen Change Loy, Jinwei Gu

Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath.

Image Restoration

Audio-Driven Emotional Video Portraits

1 code implementation CVPR 2021 Xinya Ji, Hang Zhou, Kaisiyuan Wang, Wayne Wu, Chen Change Loy, Xun Cao, Feng Xu

In this work, we present Emotional Video Portraits (EVP), a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios.

Face Generation

Everything's Talkin': Pareidolia Face Reenactment

1 code implementation7 Apr 2021 Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, Ran He

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.

Face Reenactment Texture Synthesis

Deep Animation Video Interpolation in the Wild

1 code implementation CVPR 2021 Li SiYao, Shiyu Zhao, Weijiang Yu, Wenxiu Sun, Dimitris N. Metaxas, Chen Change Loy, Ziwei Liu

In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming.

Optical Flow Estimation Video Frame Interpolation

Domain Generalization in Vision: A Survey

1 code implementation3 Mar 2021 Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, Chen Change Loy

In particular, intensive research in this topic has led to a broad spectrum of methodologies, e. g., those based on domain alignment, meta-learning, data augmentation, or ensemble learning, just to name a few; and has covered various vision applications such as object recognition, segmentation, action recognition, and person re-identification.

Action Recognition Data Augmentation +6

Network Pruning via Resource Reallocation

no code implementations2 Mar 2021 Yuenan Hou, Zheng Ma, Chunxiao Liu, Zhe Wang, Chen Change Loy

Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network.

Network Pruning

Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation

1 code implementation1 Mar 2021 Chongyi Li, Chunle Guo, Chen Change Loy

This paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network.

Face Detection Image Enhancement

FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation

1 code implementation ICCV 2021 Yuhang Zang, Chen Huang, Chen Change Loy

We propose a simple yet effective method, Feature Augmentation and Sampling Adaptation (FASA), that addresses the data scarcity issue by augmenting the feature space especially for rare classes.

Instance Segmentation Semantic Segmentation +1

Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memory

no code implementations29 Dec 2020 Yu Rong, Ziwei Liu, Chen Change Loy

The reason is that most of the current models perform regression based on a single human prototype, which is similar to common poses while far from the rare poses.

3D Human Reconstruction

Exploring Data Augmentation for Multi-Modality 3D Object Detection

2 code implementations23 Dec 2020 Wenwei Zhang, Zhe Wang, Chen Change Loy

Due to the fact that multi-modality data augmentation must maintain consistency between point cloud and images, recent methods in this field typically use relatively insufficient data augmentation.

3D Object Detection Autonomous Driving +1

Focal Frequency Loss for Image Reconstruction and Synthesis

1 code implementation ICCV 2021 Liming Jiang, Bo Dai, Wayne Wu, Chen Change Loy

In this study, we show that narrowing gaps in the frequency domain can ameliorate image reconstruction and synthesis quality further.

 Ranked #1 on Image-to-Image Translation on Cityscapes Labels-to-Photo (Per-pixel Accuracy metric)

Image Reconstruction Image-to-Image Translation

Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup

1 code implementation17 Dec 2020 Guodong Xu, Ziwei Liu, Chen Change Loy

Our goal is to achieve a performance comparable to conventional knowledge distillation with a lower computation cost during training.

Knowledge Distillation Model Compression +1

Positional Encoding as Spatial Inductive Bias in GANs

no code implementations CVPR 2021 Rui Xu, Xintao Wang, Kai Chen, Bolei Zhou, Chen Change Loy

In this work, taking SinGAN and StyleGAN2 as examples, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.

Image Manipulation Translation

CARAFE++: Unified Content-Aware ReAssembly of FEatures

no code implementations7 Dec 2020 Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, Dahua Lin

Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.

Image Inpainting Instance Segmentation +2

BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond

4 code implementations CVPR 2021 Kelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong, Chen Change Loy

Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension.

Video Super-Resolution

GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution

no code implementations CVPR 2021 Kelvin C. K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy

We show that pre-trained Generative Adversarial Networks (GANs), e. g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR).

GAN inversion Image Super-Resolution

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

1 code implementation ICLR 2021 Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo

Through our investigation, we found that such a pre-trained GAN indeed contains rich 3D knowledge and thus can be used to recover 3D shape from a single 2D image in an unsupervised manner.

3D Shape Reconstruction

Flexible Piecewise Curves Estimation for Photo Enhancement

no code implementations26 Oct 2020 Chongyi Li, Chunle Guo, Qiming Ai, Shangchen Zhou, Chen Change Loy

This paper presents a new method, called FlexiCurve, for photo enhancement.

Texture Memory-Augmented Deep Patch-Based Image Inpainting

1 code implementation28 Sep 2020 Rui Xu, Minghao Guo, Jiaqi Wang, Xiaoxiao Li, Bolei Zhou, Chen Change Loy

By bringing together the best of both paradigms, we propose a new deep inpainting framework where texture generation is guided by a texture memory of patch samples extracted from unmasked regions.

Image Inpainting Texture Synthesis

Understanding Deformable Alignment in Video Super-Resolution

no code implementations15 Sep 2020 Kelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong, Chen Change Loy

Aside from the contributions to deformable alignment, our formulation inspires a more flexible approach to introduce offset diversity to flow-based alignment, improving its performance.

Optical Flow Estimation Video Super-Resolution

Delving into Inter-Image Invariance for Unsupervised Visual Representations

1 code implementation26 Aug 2020 Jiahao Xie, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy

In this work, we present a rigorous and comprehensive study on inter-image invariance learning from three main constituting components: pseudo-label maintenance, sampling strategy, and decision boundary design.

Contrastive Learning Representation Learning

MessyTable: Instance Association in Multiple Camera Views

no code implementations ECCV 2020 Zhongang Cai, Junzhe Zhang, Daxuan Ren, Cunjun Yu, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Chen Change Loy

We present an interesting and challenging dataset that features a large number of scenes with messy tables captured from multiple camera views.

Cross-Scale Internal Graph Neural Network for Image Super-Resolution

1 code implementation NeurIPS 2020 Shangchen Zhou, Jiawei Zhang, WangMeng Zuo, Chen Change Loy

Specifically, we dynamically construct a cross-scale graph by searching k-nearest neighboring patches in the downsampled LR image for each query patch in the LR image.

Image Restoration Image Super-Resolution

Knowledge Distillation Meets Self-Supervision

2 code implementations ECCV 2020 Guodong Xu, Ziwei Liu, Xiaoxiao Li, Chen Change Loy

Knowledge distillation, which involves extracting the "dark knowledge" from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning.

Contrastive Learning Knowledge Distillation +2

Inter-Region Affinity Distillation for Road Marking Segmentation

1 code implementation CVPR 2020 Yuenan Hou, Zheng Ma, Chunxiao Liu, Tak-Wai Hui, Chen Change Loy

We study the problem of distilling knowledge from a large deep teacher network to a much smaller student network for the task of road marking segmentation.

Knowledge Distillation Lane Detection +1

Feature Pyramid Grids

1 code implementation7 Apr 2020 Kai Chen, Yuhang Cao, Chen Change Loy, Dahua Lin, Christoph Feichtenhofer

Feature pyramid networks have been widely adopted in the object detection literature to improve feature representations for better handling of variations in scale.

Neural Architecture Search Object Detection +1

Self-Supervised Scene De-occlusion

no code implementations CVPR 2020 Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, Chen Change Loy

This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.

Image Manipulation Scene Understanding

Learning to Cluster Faces via Confidence and Connectivity Estimation

2 code implementations CVPR 2020 Lei Yang, Dapeng Chen, Xiaohang Zhan, Rui Zhao, Chen Change Loy, Dahua Lin

With the vertex confidence and edge connectivity, we can naturally organize more relevant vertices on the affinity graph and group them into clusters.

Connectivity Estimation Face Clustering

TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting

no code implementations CVPR 2020 Zhuoqian Yang, Wentao Zhu, Wayne Wu, Chen Qian, Qiang Zhou, Bolei Zhou, Chen Change Loy

We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person.

motion retargeting

1st Place Solutions for OpenImage2019 -- Object Detection and Instance Segmentation

2 code implementations17 Mar 2020 Yu Liu, Guanglu Song, Yuhang Zang, Yan Gao, Enze Xie, Junjie Yan, Chen Change Loy, Xiaogang Wang

Given such good instance bounding box, we further design a simple instance-level semantic segmentation pipeline and achieve the 1st place on the segmentation challenge.

General Classification Instance Segmentation +3

Residual Knowledge Distillation

no code implementations21 Feb 2020 Mengya Gao, Yujun Shen, Quanquan Li, Chen Change Loy

Knowledge distillation (KD) is one of the most potent ways for model compression.

Knowledge Distillation Model Compression

Real or Not Real, that is the Question

2 code implementations ICLR 2020 Yuanbo Xiangli, Yubin Deng, Bo Dai, Chen Change Loy, Dahua Lin

While generative adversarial networks (GAN) have been widely adopted in various topics, in this paper we generalize the standard GAN to a new perspective by treating realness as a random variable that can be estimated from multiple angles.

Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement

5 code implementations CVPR 2020 Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, Runmin Cong

The paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network.

Face Detection Low-Light Image Enhancement

Everybody's Talkin': Let Me Talk as You Want

no code implementations15 Jan 2020 Linsen Song, Wayne Wu, Chen Qian, Ran He, Chen Change Loy

The audio-translated expression parameters are then used to synthesize a photo-realistic human subject in each video frame, with the movement of the mouth regions precisely mapped to the source audio.

3D Face Reconstruction

EcoNAS: Finding Proxies for Economical Neural Architecture Search

no code implementations CVPR 2020 Dongzhan Zhou, Xinchi Zhou, Wenwei Zhang, Chen Change Loy, Shuai Yi, Xuesen Zhang, Wanli Ouyang

While many methods have been proposed to improve the efficiency of NAS, the search progress is still laborious because training and evaluating plausible architectures over large search space is time-consuming.

Neural Architecture Search

Side-Aware Boundary Localization for More Precise Object Detection

1 code implementation ECCV 2020 Jiaqi Wang, Wenwei Zhang, Yuhang Cao, Kai Chen, Jiangmiao Pang, Tao Gong, Jianping Shi, Chen Change Loy, Dahua Lin

To tackle the difficulty of precise localization in the presence of displacements with large variance, we further propose a two-step localization scheme, which first predicts a range of movement through bucket prediction and then pinpoints the precise position within the predicted bucket.

Object Detection

Learning to Synthesize Fashion Textures

no code implementations18 Nov 2019 Wu Shi, Tak-Wai Hui, Ziwei Liu, Dahua Lin, Chen Change Loy

Another important observation is that fashion textures are multi-modal.

Robust Multi-Modality Multi-Object Tracking

1 code implementation ICCV 2019 Wenwei Zhang, Hui Zhou, Shuyang Sun, Zhe Wang, Jianping Shi, Chen Change Loy

Multi-sensor perception is crucial to ensure the reliability and accuracy in autonomous driving system, while multi-object tracking (MOT) improves that by tracing sequential movement of dynamic objects.

Autonomous Driving Multi-Object Tracking +1

Delving Deep Into Hybrid Annotations for 3D Human Recovery in the Wild

1 code implementation ICCV 2019 Yu Rong, Ziwei Liu, Cheng Li, Kaidi Cao, Chen Change Loy

Specifically, we focus on the challenging task of in-the-wild 3D human recovery from single images when paired 3D annotations are not fully available.

One-shot Face Reenactment

1 code implementation5 Aug 2019 Yunxuan Zhang, Siwei Zhang, Yue He, Cheng Li, Chen Change Loy, Ziwei Liu

However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable.

Face Reconstruction Face Reenactment

Learning Lightweight Lane Detection CNNs by Self Attention Distillation

2 code implementations ICCV 2019 Yuenan Hou, Zheng Ma, Chunxiao Liu, Chen Change Loy

Training deep models for lane detection is challenging due to the very subtle and sparse supervisory signals inherent in lane annotations.

Knowledge Distillation Lane Detection +1

Disentangling Content and Style via Unsupervised Geometry Distillation

1 code implementation ICLR Workshop DeepGenStruct 2019 Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy

It is challenging to disentangle an object into two orthogonal spaces of content and style since each can influence the visual observation differently and unpredictably.

Deep Flow-Guided Video Inpainting

2 code implementations CVPR 2019 Rui Xu, Xiaoxiao Li, Bolei Zhou, Chen Change Loy

Then the synthesized flow field is used to guide the propagation of pixels to fill up the missing regions in the video.

One-shot visual object segmentation Optical Flow Estimation +2

EDVR: Video Restoration with Enhanced Deformable Convolutional Networks

8 code implementations7 May 2019 Xintao Wang, Kelvin C. K. Chan, Ke Yu, Chao Dong, Chen Change Loy

In this work, we propose a novel Video Restoration framework with Enhanced Deformable networks, termed EDVR, to address these challenges.

 Ranked #1 on Deblurring on REDS

Deblurring Video Restoration +1

CARAFE: Content-Aware ReAssembly of FEatures

2 code implementations ICCV 2019 Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, Dahua Lin

CARAFE introduces little computational overhead and can be readily integrated into modern network architectures.

Instance Segmentation Object Detection +1

Path-Restore: Learning Network Path Selection for Image Restoration

no code implementations23 Apr 2019 Ke Yu, Xintao Wang, Chao Dong, Xiaoou Tang, Chen Change Loy

To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route for each image region.

Denoising Image Restoration

TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation

no code implementations CVPR 2019 Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy

Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks.

Translation Unsupervised Image-To-Image Translation

Prime Sample Attention in Object Detection

1 code implementation CVPR 2020 Yuhang Cao, Kai Chen, Chen Change Loy, Dahua Lin

Our experiments demonstrate that it is often more effective to focus on prime samples than hard samples when training a detector.

Object Detection

Learning to Cluster Faces on an Affinity Graph

2 code implementations CVPR 2019 Lei Yang, Xiaohang Zhan, Dapeng Chen, Junjie Yan, Chen Change Loy, Dahua Lin

Face recognition sees remarkable progress in recent years, and its performance has reached a very high level.

Face Recognition

Dense Intrinsic Appearance Flow for Human Pose Transfer

1 code implementation CVPR 2019 Yining Li, Chen Huang, Chen Change Loy

Unlike existing methods, we propose to estimate dense and intrinsic 3D appearance flow to better guide the transfer of pixels between poses.

Pose Transfer

Self-Supervised Learning via Conditional Motion Propagation

no code implementations CVPR 2019 Xiaohang Zhan, Xingang Pan, Ziwei Liu, Dahua Lin, Chen Change Loy

Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.

Human Parsing Instance Segmentation +2

A Lightweight Optical Flow CNN -- Revisiting Data Fidelity and Regularization

1 code implementation15 Mar 2019 Tak-Wai Hui, Xiaoou Tang, Chen Change Loy

Over four decades, the majority addresses the problem of optical flow estimation using variational methods.

Optical Flow Estimation

Unsupervised Bi-directional Flow-based Video Generation from one Snapshot

no code implementations3 Mar 2019 Lu Sheng, Junting Pan, Jiaming Guo, Jing Shao, Xiaogang Wang, Chen Change Loy

Imagining multiple consecutive frames given one single snapshot is challenging, since it is difficult to simultaneously predict diverse motions from a single image and faithfully generate novel frames without visual distortions.

Video Generation

Hybrid Task Cascade for Instance Segmentation

3 code implementations CVPR 2019 Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, Chen Change Loy, Dahua Lin

In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation.

Instance Segmentation Object Detection +1

Region Proposal by Guided Anchoring

2 code implementations CVPR 2019 Jiaqi Wang, Kai Chen, Shuo Yang, Chen Change Loy, Dahua Lin

State-of-the-art detectors mostly rely on a dense anchoring scheme, where anchors are sampled uniformly over the spatial domain with a predefined set of scales and aspect ratios.

Object Detection Region Proposal

An Embarrassingly Simple Approach for Knowledge Distillation

1 code implementation5 Dec 2018 Mengya Gao, Yujun Shen, Quanquan Li, Junjie Yan, Liang Wan, Dahua Lin, Chen Change Loy, Xiaoou Tang

Knowledge Distillation (KD) aims at improving the performance of a low-capacity student model by inheriting knowledge from a high-capacity teacher model.

Face Recognition Knowledge Distillation +2

Instance-level Facial Attributes Transfer with Geometry-Aware Flow

no code implementations30 Nov 2018 Weidong Yin, Ziwei Liu, Chen Change Loy

Geometry-aware flow is able to warp the source face attribute into the target face context and generate a warp-and-blend result.

Deep Network Interpolation for Continuous Imagery Effect Transition

2 code implementations CVPR 2019 Xintao Wang, Ke Yu, Chao Dong, Xiaoou Tang, Chen Change Loy

Deep convolutional neural network has demonstrated its capability of learning a deterministic mapping for the desired imagery effect.

Image Restoration Image-to-Image Translation +2

Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks

2 code implementations7 Nov 2018 Yuenan Hou, Zheng Ma, Chunxiao Liu, Chen Change Loy

In this paper, we considerably improve the accuracy and robustness of predictions through heterogeneous auxiliary networks feature mimicking, a new and effective training method that provides us with much richer contextual signals apart from steering direction.

Multi-Task Learning Optical Flow Estimation +2

Unsupervised Disentangling Structure and Appearance

no code implementations27 Sep 2018 Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy

It is challenging to disentangle an object into two orthogonal spaces of structure and appearance since each can influence the visual observation in a different and unpredictable way.

Improving On-policy Learning with Statistical Reward Accumulation

no code implementations7 Sep 2018 Yubin Deng, Ke Yu, Dahua Lin, Xiaoou Tang, Chen Change Loy

Most methods in deep-RL achieve good results via the maximization of the reward signal provided by the environment, typically in the form of discounted cumulative returns.

Atari Games

Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition

1 code implementation ECCV 2018 Xiaohang Zhan, Ziwei Liu, Junjie Yan, Dahua Lin, Chen Change Loy

Face recognition has witnessed great progress in recent years, mainly attributed to the high-capacity model designed and the abundant labeled data collected.

Face Recognition

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

28 code implementations1 Sep 2018 Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Chen Change Loy, Yu Qiao, Xiaoou Tang

To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN).

Face Hallucination Image Super-Resolution +1

PSANet: Point-wise Spatial Attention Network for Scene Parsing

3 code implementations ECCV 2018 Hengshuang Zhao, Yi Zhang, Shu Liu, Jianping Shi, Chen Change Loy, Dahua Lin, Jiaya Jia

We notice information flow in convolutional neural networks is restricted inside local neighborhood regions due to the physical design of convolutional filters, which limits the overall understanding of complex scenes.

Scene Parsing Semantic Segmentation

The Devil of Face Recognition is in the Noise

2 code implementations ECCV 2018 Fei Wang, Liren Chen, Cheng Li, Shiyao Huang, Yanjie Chen, Chen Qian, Chen Change Loy

2) With the original datasets and cleaned subsets, we profile and analyze label noise properties of MegaFace and MS-Celeb-1M.

Face Recognition

Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition

no code implementations ECCV 2018 Guojun Yin, Lu Sheng, Bin Liu, Nenghai Yu, Xiaogang Wang, Jing Shao, Chen Change Loy

We show that by encouraging deep message propagation and interactions between local object features and global predicate features, one can achieve compelling performance in recognizing complex relationships without using any linguistic priors.

Non-Local Recurrent Network for Image Restoration

1 code implementation NeurIPS 2018 Ding Liu, Bihan Wen, Yuchen Fan, Chen Change Loy, Thomas S. Huang

The main contributions of this work are: (1) Unlike existing methods that measure self-similarity in an isolated manner, the proposed non-local module can be flexibly integrated into existing deep networks for end-to-end training to capture deep feature correlation between each location and its neighborhood.

Image Denoising Image Restoration +1

Deep Imbalanced Learning for Face Recognition and Attribute Prediction

1 code implementation1 Jun 2018 Chen Huang, Yining Li, Chen Change Loy, Xiaoou Tang

Data for face analysis often exhibit highly-skewed class distribution, i. e., most data belong to a few majority classes, while the minority classes only contain a scarce amount of instances.

Face Recognition Representation Learning

LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation

4 code implementations CVPR 2018 Tak-Wai Hui, Xiaoou Tang, Chen Change Loy

FlowNet2, the state-of-the-art convolutional neural network (CNN) for optical flow estimation, requires over 160M parameters to achieve accurate flow estimation.

Optical Flow Estimation

Optimizing Video Object Detection via a Scale-Time Lattice

1 code implementation CVPR 2018 Kai Chen, Jiaqi Wang, Shuo Yang, Xingcheng Zhang, Yuanjun Xiong, Chen Change Loy, Dahua Lin

High-performance object detection relies on expensive convolutional networks to compute features, often leading to significant challenges in applications, e. g. those that require detecting objects from video streams in real time.

Video Object Detection

Pose-Robust Face Recognition via Deep Residual Equivariant Mapping

1 code implementation CVPR 2018 Kaidi Cao, Yu Rong, Cheng Li, Xiaoou Tang, Chen Change Loy

However, many contemporary face recognition models still perform relatively poor in processing profile faces compared to frontal faces.

Face Identification Face Recognition +2

Mix-and-Match Tuning for Self-Supervised Semantic Segmentation

no code implementations2 Dec 2017 Xiaohang Zhan, Ziwei Liu, Ping Luo, Xiaoou Tang, Chen Change Loy

The key of this new form of learning is to design a proxy task (e. g. image colorization), from which a discriminative loss can be formulated on unlabeled data.

Colorization Fine-tuning +1

Be Your Own Prada: Fashion Synthesis with Structural Coherence

no code implementations ICCV 2017 Shizhan Zhu, Sanja Fidler, Raquel Urtasun, Dahua Lin, Chen Change Loy

In the second stage, a generative model with a newly proposed compositional mapping layer is used to render the final image with precise regions and textures conditioned on this map.

Fashion Synthesis Semantic Segmentation

Quantifying Facial Age by Posterior of Age Comparisons

1 code implementation31 Aug 2017 Yunxuan Zhang, Li Liu, Cheng Li, Chen Change Loy

We introduce a novel approach for annotating large quantity of in-the-wild facial images with high-quality posterior age distribution as labels.

Ranked #4 on Age Estimation on MORPH Album2 (using extra training data)

Age And Gender Classification Age Estimation

Video Object Segmentation with Re-identification

3 code implementations1 Aug 2017 Xiaoxiao Li, Yuankai Qi, Zhe Wang, Kai Chen, Ziwei Liu, Jianping Shi, Ping Luo, Xiaoou Tang, Chen Change Loy

Specifically, our Video Object Segmentation with Re-identification (VS-ReID) model includes a mask propagation module and a ReID module.

Semantic Segmentation Video Object Segmentation +2

Discover and Learn New Objects from Documentaries

1 code implementation CVPR 2017 Kai Chen, Hang Song, Chen Change Loy, Dahua Lin

Despite the remarkable progress in recent years, detecting objects in a new context remains a challenging task.

Aesthetic-Driven Image Enhancement by Adversarial Learning

1 code implementation17 Jul 2017 Yubin Deng, Chen Change Loy, Xiaoou Tang

We introduce EnhanceGAN, an adversarial learning based model that performs automatic image enhancement.

 Ranked #1 on Image Cropping on AVA (using extra training data)

Image Cropping Image Enhancement

Merge or Not? Learning to Group Faces via Imitation Learning

1 code implementation13 Jul 2017 Yue He, Kaidi Cao, Cheng Li, Chen Change Loy

Given a large number of unlabeled face images, face grouping aims at clustering the images into individual identities present in the data.

Imitation Learning

Face Detection through Scale-Friendly Deep Convolutional Networks

no code implementations9 Jun 2017 Shuo Yang, Yuanjun Xiong, Chen Change Loy, Xiaoou Tang

Specifically, our method achieves 76. 4 average precision on the challenging WIDER FACE dataset and 96% recall rate on the FDDB dataset with 7 frames per second (fps) for 900 * 1300 input image.

Face Detection

Robust and Fast Decoding of High-Capacity Color QR Codes for Mobile Applications

1 code implementation21 Apr 2017 Zhibo Yang, Huanle Xu, Jianyuan Deng, Chen Change Loy, Wing Cheong Lau

Particularly, we further discover a new type of chromatic distortion in high-density color QR codes, cross-module color interference, caused by the high density which also makes the geometric distortion correction more challenging.

Faceness-Net: Face Detection through Deep Facial Part Responses

no code implementations29 Jan 2017 Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang

We propose a deep convolutional neural network (CNN) for face detection leveraging on facial attributes based supervision.

Face Detection

PolyNet: A Pursuit of Structural Diversity in Very Deep Networks

3 code implementations CVPR 2017 Xingcheng Zhang, Zhizhong Li, Chen Change Loy, Dahua Lin

A number of studies have shown that increasing the depth or width of convolutional networks is a rewarding approach to improve the performance of image recognition.

Image Classification

Local Similarity-Aware Deep Feature Embedding

no code implementations NeurIPS 2016 Chen Huang, Chen Change Loy, Xiaoou Tang

Existing deep embedding methods in vision tasks are capable of learning a compact Euclidean space from images, where Euclidean distances correspond to a similarity metric.

Image Retrieval Transfer Learning +1

Image Aesthetic Assessment: An Experimental Survey

1 code implementation4 Oct 2016 Yubin Deng, Chen Change Loy, Xiaoou Tang

This survey aims at reviewing recent computer vision techniques used in the assessment of image aesthetic quality.

From Facial Expression Recognition to Interpersonal Relation Prediction

no code implementations21 Sep 2016 Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang

Unlike existing models that typically learn from facial expression labels alone, we devise an effective multitask network that is capable of learning from rich auxiliary attributes such as gender, age, and head pose, beyond just facial expression data.

Facial Expression Recognition

Deep Convolution Networks for Compression Artifacts Reduction

2 code implementations9 Aug 2016 Ke Yu, Chao Dong, Chen Change Loy, Xiaoou Tang

Lossy compression introduces complex compression artifacts, particularly blocking artifacts, ringing effects and blurring.

Transfer Learning

Accelerating the Super-Resolution Convolutional Neural Network

9 code implementations1 Aug 2016 Chao Dong, Chen Change Loy, Xiaoou Tang

As a successful deep model applied in image super-resolution (SR), the Super-Resolution Convolutional Neural Network (SRCNN) has demonstrated superior performance to the previous hand-crafted models either in speed and restoration quality.

Image Super-Resolution

Deep Cascaded Bi-Network for Face Hallucination

no code implementations18 Jul 2016 Shizhan Zhu, Sifei Liu, Chen Change Loy, Xiaoou Tang

We present a novel framework for hallucinating faces of unconstrained poses and with very low resolution (face size as small as 5pxIOD).

Face Hallucination

Learning Deep Representation for Imbalanced Classification

no code implementations CVPR 2016 Chen Huang, Yining Li, Chen Change Loy, Xiaoou Tang

We further demonstrate that more discriminative deep representation can be learned by enforcing a deep network to maintain both inter-cluster and inter-class margins.

Classification General Classification +2

Discriminative Sparse Neighbor Approximation for Imbalanced Learning

no code implementations3 Feb 2016 Chen Huang, Chen Change Loy, Xiaoou Tang

These methods further deteriorate on small, imbalanced data that has a large degree of class overlap.

Towards Arbitrary-View Face Alignment by Recommendation Trees

no code implementations20 Nov 2015 Shizhan Zhu, Cheng Li, Chen Change Loy, Xiaoou Tang

The unified framework seamlessly handles different viewpoints and landmark protocols, and it is trained by optimising directly on landmark locations, thus yielding superior results on arbitrary-view face alignment.

Face Alignment Head Pose Estimation +1

An Empirical Study of Recent Face Alignment Methods

no code implementations16 Nov 2015 Heng Yang, Xuhui Jia, Chen Change Loy, Peter Robinson

In this paper, we carry out a rigorous evaluation of these methods by making the following contributions: 1) we proposes a new evaluation metric for face alignment on a set of images, i. e., area under error distribution curve within a threshold, AUC$_\alpha$, given the fact that the traditional evaluation measure (mean error) is very sensitive to big alignment error.

Face Alignment Face Detection

From Facial Parts Responses to Face Detection: A Deep Learning Approach

2 code implementations ICCV 2015 Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang

In this paper, we propose a novel deep convolutional network (DCN) that achieves outstanding performance on FDDB, PASCAL Face, and AFW.

Face Detection

Learning Social Relation Traits from Face Images

no code implementations ICCV 2015 Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang

Social relation defines the association, e. g, warm, friendliness, and dominance, between two or more people.

Semantic Image Segmentation via Deep Parsing Network

no code implementations ICCV 2015 Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, Xiaoou Tang

This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts.

Semantic Segmentation

Reading Scene Text in Deep Convolutional Sequences

1 code implementation14 Jun 2015 Pan He, Weilin Huang, Yu Qiao, Chen Change Loy, Xiaoou Tang

We develop a Deep-Text Recurrent Network (DTRN) that regards scene text reading as a sequence labelling problem.

Scene Text Scene Text Recognition

Learning from Multiple Sources for Video Summarisation

no code implementations13 Jan 2015 Xiatian Zhu, Chen Change Loy, Shaogang Gong

Many visual surveillance tasks, e. g. video summarisation, is conventionally accomplished through analysing imagerybased features.

Video Understanding

Learning to Recognize Pedestrian Attribute

no code implementations5 Jan 2015 Yubin Deng, Ping Luo, Chen Change Loy, Xiaoou Tang

Learning to recognize pedestrian attributes at far distance is a challenging problem in visual surveillance since face and body close-shots are hardly available; instead, only far-view image frames of pedestrian are given.

Image Super-Resolution Using Deep Convolutional Networks

52 code implementations31 Dec 2014 Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang

We further show that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network.

Image Super-Resolution Video Super-Resolution

Crowd Saliency Detection via Global Similarity Structure

no code implementations14 Oct 2014 Mei Kuan Lim, Ven Jyn Kok, Chen Change Loy, Chee Seng Chan

This paper proposes a novel framework to identify and localize salient regions in a crowd scene, by transforming low-level features extracted from crowd motion field into a global similarity structure.

Saliency Detection

Transferring Landmark Annotations for Cross-Dataset Face Alignment

no code implementations2 Sep 2014 Shizhan Zhu, Cheng Li, Chen Change Loy, Xiaoou Tang

We show extensive results on combining various popular databases (LFW, AFLW, LFPW, HELEN) for improved cross-dataset and unseen data alignment.

Face Alignment Object Recognition

Constructing Robust Affinity Graphs for Spectral Clustering

no code implementations CVPR 2014 Xiatian Zhu, Chen Change Loy, Shaogang Gong

Spectral clustering requires robust and meaningful affinity graphs as input in order to form clusters with desired structures that can well support human intuition.

Cumulative Attribute Space for Age and Crowd Density Estimation

no code implementations CVPR 2013 Ke Chen, Shaogang Gong, Tao Xiang, Chen Change Loy

A number of computer vision problems such as human age estimation, crowd density estimation and body/face pose (view angle) estimation can be formulated as a regression problem by learning a mapping function between a high dimensional vector-formed feature input and a scalarvalued output.

Age Estimation Crowd Counting +1

Cannot find the paper you are looking for? You can Submit a new open access paper.