Search Results for author: Ran He

Found 133 papers, 43 papers with code

Information-Theoretic Measures for Objective Evaluation of Classifications

no code implementations10 Jul 2011 Bao-Gang Hu, Ran He, Xiaotong Yuan

This work presents a systematic study of objective evaluations of abstaining classifications using Information-Theoretic Measures (ITMs).

Cross-Modal Learning via Pairwise Constraints

no code implementations28 Nov 2014 Ran He, Man Zhang, Liang Wang, Ye Ji, Qiyue Yin

For unsupervised learning, we propose a cross-modal subspace clustering method to learn a common structure for different modalities.

Clustering Retrieval

Learning Structured Ordinal Measures for Video based Face Recognition

no code implementations9 Jul 2015 Ran He, Tieniu Tan, Larry Davis, Zhenan Sun

This paper presents a structured ordinal measure method for video-based face recognition that simultaneously learns ordinal filters and structured ordinal features.

Face Recognition

A Light CNN for Deep Face Representation with Noisy Labels

16 code implementations9 Nov 2015 Xiang Wu, Ran He, Zhenan Sun, Tieniu Tan

This paper presents a Light CNN framework to learn a compact embedding on the large-scale face data with massive noisy labels.

Face Identification Face Recognition +2

Locally Imposing Function for Generalized Constraint Neural Networks - A Study on Equality Constraints

no code implementations18 Apr 2016 Linlin Cao, Ran He, Bao-Gang Hu

A new method called locally imposing function (LIF) is proposed to provide a local correction to the GCNN prediction function, which therefore falls within Locally Imposing Scheme (LIS).

Deep Aesthetic Quality Assessment with Semantic Information

no code implementations18 Apr 2016 Yueying Kao, Ran He, Kaiqi Huang

Human beings often assess the aesthetic quality of an image coupled with the identification of the image's semantic content.

Aesthetics Quality Assessment

Self-Paced Learning: an Implicit Regularization Perspective

no code implementations1 Jun 2016 Yanbo Fan, Ran He, Jian Liang, Bao-Gang Hu

In this paper, we focus on the minimizer function, and study a group of new regularizer, named self-paced implicit regularizer that is deduced from robust loss function.

DeMeshNet: Blind Face Inpainting for Deep MeshFace Verification

no code implementations16 Nov 2016 Shu Zhang, Ran He, Tieniu Tan

The occlusions incurred by random meshes severely degenerate the performance of face verification systems, which raises the MeshFace verification problem between MeshFace and daily photos.

Face Alignment Face Verification +1

Coupled Deep Learning for Heterogeneous Face Recognition

no code implementations8 Apr 2017 Xiang Wu, Lingxiao Song, Ran He, Tieniu Tan

CDL seeks a shared feature space in which the heterogeneous face matching problem can be approximately treated as a homogeneous face matching problem.

Face Recognition Heterogeneous Face Recognition

Attention-Set based Metric Learning for Video Face Recognition

no code implementations12 Apr 2017 Yibo Hu, Xiang Wu, Ran He

In this paper, we propose a novel Attention-Set based Metric Learning (ASML) method to measure the statistical characteristics of image sets.

Face Recognition Metric Learning

Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis

3 code implementations ICCV 2017 Rui Huang, Shu Zhang, Tianyu Li, Ran He

This paper proposes a Two-Pathway Generative Adversarial Network (TP-GAN) for photorealistic frontal view synthesis by simultaneously perceiving global structures and local details.

Face Recognition Generative Adversarial Network

Robust Localized Multi-view Subspace Clustering

no code implementations22 May 2017 Yanbo Fan, Jian Liang, Ran He, Bao-Gang Hu, Siwei Lyu

In multi-view clustering, different views may have different confidence levels when learning a consensus representation.

Clustering Multi-view Subspace Clustering

Deep Supervised Discrete Hashing

no code implementations NeurIPS 2017 Qi Li, Zhenan Sun, Ran He, Tieniu Tan

Benefit from recent advances in deep learning, deep hashing methods have achieved promising results for image retrieval.

Deep Hashing General Classification +1

Recent Progress of Face Image Synthesis

no code implementations15 Jun 2017 Zhihe Lu, Zhihang Li, Jie Cao, Ran He, Zhenan Sun

Face synthesis has been a fascinating yet challenging problem in computer vision and machine learning.

Face Generation Face Recognition

Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognition

no code implementations8 Aug 2017 Ran He, Xiang Wu, Zhenan Sun, Tieniu Tan

To avoid the over-fitting problem on small-scale heterogeneous face data, a correlation prior is introduced on the fully-connected layers of WCNN network to reduce parameter space.

Face Recognition Heterogeneous Face Recognition

Joint Adaptive Neighbours and Metric Learning for Multi-view Subspace Clustering

no code implementations12 Sep 2017 Nan Xu, Yanqing Guo, Jiujun Wang, Xiangyang Luo, Ran He

In this method, we use the subspace representations of different views to adaptively learn a consensus similarity matrix, uncovering the subspace structure and avoiding noisy nature of original data.

Clustering Metric Learning +2

Anti-Makeup: Learning A Bi-Level Adversarial Network for Makeup-Invariant Face Verification

no code implementations12 Sep 2017 Yi Li, Lingxiao Song, Xiang Wu, Ran He, Tieniu Tan

This paper proposes a learning from generation approach for makeup-invariant face verification by introducing a bi-level adversarial network (BLAN).

Face Verification

Adversarial Discriminative Heterogeneous Face Recognition

no code implementations12 Sep 2017 Lingxiao Song, Man Zhang, Xiang Wu, Ran He

This framework integrates cross-spectral face hallucination and discriminative feature learning into an end-to-end adversarial network.

Face Hallucination Face Recognition +2

Adversarial Occlusion-aware Face Detection

1 code implementation15 Sep 2017 Yujia Chen, Lingxiao Song, Ran He

This paper introduces an Adversarial Occlusion-aware Face Detector (AOFD) by simultaneously detecting occluded faces and segmenting occluded areas.

Occluded Face Detection

Geometry Guided Adversarial Facial Expression Synthesis

no code implementations10 Dec 2017 Lingxiao Song, Zhihe Lu, Ran He, Zhenan Sun, Tieniu Tan

An expression invariant face recognition experiment is also performed to further show the advantages of our proposed method.

Face Recognition Face Transfer +2

Learning Disentangling and Fusing Networks for Face Completion Under Structured Occlusions

no code implementations13 Dec 2017 Zhihang Li, Yibo Hu, Ran He

We treat the face completion and corruption as disentangling and fusing processes of clean faces and occlusions, and propose a jointly disentangling and fusing Generative Adversarial Network (DF-GAN).

Facial Inpainting Generative Adversarial Network

Global and Local Consistent Age Generative Adversarial Networks

no code implementations25 Jan 2018 Pei-Pei Li, Yibo Hu, Qi Li, Ran He, Zhenan Sun

To utilize both global and local facial information, we propose a Global and Local Consistent Age Generative Adversarial Network (GLCA-GAN).

Attribute Generative Adversarial Network +2

Load Balanced GANs for Multi-view Face Image Synthesis

no code implementations21 Feb 2018 Jie Cao, Yibo Hu, Bing Yu, Ran He, Zhenan Sun

Multi-view face synthesis from a single image is an ill-posed problem and often suffers from serious appearance distortion.

Face Generation

Pose-Guided Photorealistic Face Rotation

no code implementations CVPR 2018 Yibo Hu, Xiang Wu, Bing Yu, Ran He, Zhenan Sun

Face rotation provides an effective and cheap way for data augmentation and representation learning of face recognition.

Data Augmentation Face Recognition +2

Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization

no code implementations NeurIPS 2018 Jie Cao, Yibo Hu, Hongwen Zhang, Ran He, Zhenan Sun

We decompose the prerequisite of warping into dense correspondence field estimation and facial texture map recovering, which are both well addressed by deep networks.

Dictionary Learning Face Recognition +2

IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis

3 code implementations NeurIPS 2018 Huaibo Huang, Zhihang Li, Ran He, Zhenan Sun, Tieniu Tan

On the other hand, the inference model is encouraged to classify between the generated and real samples while the generator tries to fool it as GANs.

Image Generation

Disentangled Variational Representation for Heterogeneous Face Recognition

no code implementations6 Sep 2018 Xiang Wu, Huaibo Huang, Vishal M. Patel, Ran He, Zhenan Sun

Visible (VIS) to near infrared (NIR) face matching is a challenging problem due to the significant domain discrepancy between the domains and a lack of sufficient data for training cross-modal matching algorithms.

Face Recognition Heterogeneous Face Recognition

Geometry-Aware Face Completion and Editing

no code implementations9 Sep 2018 Linsen Song, Jie Cao, Linxiao Song, Yibo Hu, Ran He

Extensive experimental results qualitatively and quantitatively demonstrate that our network is able to generate visually pleasing face completion results and edit face attributes as well.

Facial Inpainting

Global and Local Consistent Wavelet-domain Age Synthesis

no code implementations20 Sep 2018 Pei-Pei Li, Yibo Hu, Ran He, Zhenan Sun

%Moreover, to achieve accurate age generation under the premise of preserving the identity information, age estimation network and face verification network are employed.

Age Estimation Face Verification +3

A Coupled Evolutionary Network for Age Estimation

no code implementations20 Sep 2018 Pei-Pei Li, Yibo Hu, Ran He, Zhenan Sun

Inspired by the biological evolutionary mechanism, we propose a Coupled Evolutionary Network (CEN) with two concurrent evolutionary processes: evolutionary label distribution learning and evolutionary slack regression.

Age Estimation MORPH +1

Arbitrary Talking Face Generation via Attentional Audio-Visual Coherence Learning

no code implementations17 Dec 2018 Hao Zhu, Huaibo Huang, Yi Li, Aihua Zheng, Ran He

Talking face generation aims to synthesize a face video with precise lip synchronization as well as a smooth transition of facial motion over the entire video via the given speech clip and facial image.

Talking Face Generation

A Survey of Deep Facial Attribute Analysis

no code implementations26 Dec 2018 Xin Zheng, Yanqing Guo, Huaibo Huang, Yi Li, Ran He

Deep learning based facial attribute analysis consists of two basic sub-issues: facial attribute estimation (FAE), which recognizes whether facial attributes are present in given images, and facial attribute manipulation (FAM), which synthesizes or removes desired facial attributes.

Attribute

Joint Iris Segmentation and Localization Using Deep Multi-task Learning Framework

1 code implementation31 Jan 2019 Caiyong Wang, Yuhao Zhu, Yunfan Liu, Ran He, Zhenan Sun

In this paper, we propose a deep multi-task learning framework, named as IrisParseNet, to exploit the inherent correlations between pupil, iris and sclera to boost up the performance of iris segmentation and localization in a unified model.

Iris Segmentation Multi-Task Learning +1

Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition

no code implementations10 Feb 2019 Ran He, Jie Cao, Lingxiao Song, Zhenan Sun, Tieniu Tan

This paper models high resolution heterogeneous face synthesis as a complementary combination of two components, a texture inpainting component and pose correction component.

Face Generation Face Recognition +3

Dual Variational Generation for Low-Shot Heterogeneous Face Recognition

1 code implementation25 Mar 2019 Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, Ran He

Then, in order to ensure the identity consistency of the generated paired heterogeneous images, we impose a distribution alignment in the latent space and a pairwise identity preserving in the image space.

Face Recognition Heterogeneous Face Recognition

High Fidelity Face Manipulation with Extreme Poses and Expressions

no code implementations28 Mar 2019 Chaoyou Fu, Yibo Hu, Xiang Wu, Guoli Wang, Qian Zhang, Ran He

Furthermore, due to the lack of high-resolution face manipulation databases to verify the effectiveness of our method, we collect a new high-quality Multi-View Face (MVF-HQ) database.

Face Generation Face Recognition +1

UVA: A Universal Variational Framework for Continuous Age Analysis

no code implementations30 Mar 2019 Pei-Pei Li, Huaibo Huang, Yibo Hu, Xiang Wu, Ran He, Zhenan Sun

UVA is the first attempt to achieve facial age analysis tasks, including age translation, age generation and age estimation, in a universal framework.

Age Estimation MORPH +1

M2FPA: A Multi-Yaw Multi-Pitch High-Quality Database and Benchmark for Facial Pose Analysis

no code implementations30 Mar 2019 Pei-Pei Li, Xiang Wu, Yibo Hu, Ran He, Zhenan Sun

In this paper, a new large-scale Multi-yaw Multi-pitch high-quality database is proposed for Facial Pose Analysis (M2FPA), including face frontalization, face rotation, facial pose estimation and pose-invariant face recognition.

Attribute Face Generation +3

PyramidBox++: High Performance Detector for Finding Tiny Face

4 code implementations31 Mar 2019 Zhihang Li, Xu Tang, Junyu Han, Jingtuo Liu, Ran He

With the rapid development of deep convolutional neural network, face detection has made great progress in recent years.

Data Augmentation Face Detection +1

Biphasic Learning of GANs for High-Resolution Image-to-Image Translation

no code implementations14 Apr 2019 Jie Cao, Huaibo Huang, Yi Li, Jingtuo Liu, Ran He, Zhenan Sun

In this work, we present a novel training framework for GANs, namely biphasic learning, to achieve image-to-image translation in multiple visual domains at $1024^2$ resolution.

Image-to-Image Translation Mutual Information Estimation +2

Attributes Guided Feature Learning for Vehicle Re-identification

no code implementations22 May 2019 Hongchao Li, Xianmin Lin, Aihua Zheng, Chenglong Li, Bin Luo, Ran He, Amir Hussain

In particular, our network is end-to-end trained and contains three subnetworks of deep features embedded by the corresponding attributes (i. e., camera view, vehicle type and vehicle color).

Generative Adversarial Network Vehicle Re-Identification

Theme-Aware Aesthetic Distribution Prediction With Full-Resolution Photographs

no code implementations4 Aug 2019 Gengyun Jia, Pei-Pei Li, Ran He

RoM pooling pools image features and discards extra padded features to eliminate the side effects of padding.

Make a Face: Towards Arbitrary High Fidelity Face Manipulation

no code implementations ICCV 2019 Shengju Qian, Kwan-Yee Lin, Wayne Wu, Yangxiaokang Liu, Quan Wang, Fumin Shen, Chen Qian, Ran He

Recent studies have shown remarkable success in face manipulation task with the advance of GANs and VAEs paradigms, but the outputs are sometimes limited to low-resolution and lack of diversity.

Clustering Disentanglement +1

Cross-Spectral Face Hallucination via Disentangling Independent Factors

no code implementations CVPR 2020 Boyan Duan, Chaoyou Fu, Yi Li, Xingguang Song, Ran He

The cross-sensor gap is one of the challenges that have aroused much research interests in Heterogeneous Face Recognition (HFR).

Face Alignment Face Hallucination +3

Augmented Data Science: Towards Industrialization and Democratization of Data Science

no code implementations12 Sep 2019 Huseyin Uzunalioglu, Jin Cao, Chitra Phadke, Gerald Lehmann, Ahmet Akyamac, Ran He, Jeongran Lee, Maria Able

Conversion of raw data into insights and knowledge requires substantial amounts of effort from data scientists.

PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer

1 code implementation CVPR 2020 Wentao Jiang, Si Liu, Chen Gao, Jie Cao, Ran He, Jiashi Feng, Shuicheng Yan

In this paper, we address the makeup transfer task, which aims to transfer the makeup from a reference image to a source image.

Dual Variational Generation for Low Shot Heterogeneous Face Recognition

no code implementations NeurIPS 2019 Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, Ran He

Specifically, we first introduce a dual variational autoencoder to represent a joint distribution of paired heterogeneous images.

Face Recognition Heterogeneous Face Recognition

LAMP-HQ: A Large-Scale Multi-Pose High-Quality Database and Benchmark for NIR-VIS Face Recognition

no code implementations17 Dec 2019 Aijing Yu, Haoxue Wu, Huaibo Huang, Zhen Lei, Ran He

A spectral conditional attention module is introduced to reduce the domain gap between NIR and VIS data and then improve the performance of NIR-VIS heterogeneous face recognition on various databases including the LAMP-HQ.

Attribute Face Recognition +1

Informative Sample Mining Network for Multi-Domain Image-to-Image Translation

no code implementations ECCV 2020 Jie Cao, Huaibo Huang, Yi Li, Ran He, Zhenan Sun

The performance of multi-domain image-to-image translation has been significantly improved by recent progress in deep generative models.

Image-to-Image Translation Informativeness +1

Deep Audio-Visual Learning: A Survey

no code implementations14 Jan 2020 Hao Zhu, Mandi Luo, Rui Wang, Aihua Zheng, Ran He

Audio-visual learning, aimed at exploiting the relationship between audio and visual modalities, has drawn considerable attention since deep learning started to be used successfully.

audio-visual learning Representation Learning

Everybody's Talkin': Let Me Talk as You Want

no code implementations15 Jan 2020 Linsen Song, Wayne Wu, Chen Qian, Ran He, Chen Change Loy

The audio-translated expression parameters are then used to synthesize a photo-realistic human subject in each video frame, with the movement of the mouth regions precisely mapped to the source audio.

3D Face Reconstruction

Augmented Parallel-Pyramid Net for Attention Guided Pose-Estimation

no code implementations17 Mar 2020 Luanxuan Hou, Jie Cao, Yuan Zhao, Haifeng Shen, Yiping Meng, Ran He, Jieping Ye

At last, we proposed a differentiable auto data augmentation method to further improve estimation accuracy.

Data Augmentation Pose Estimation

Cosmetic-Aware Makeup Cleanser

no code implementations20 Apr 2020 Yi Li, Huaibo Huang, Junchi Yu, Ran He, Tieniu Tan

Face verification aims at determining whether a pair of face images belongs to the same identity.

Face Parsing Face Verification +1

Recapture as You Want

no code implementations2 Jun 2020 Chen Gao, Si Liu, Ran He, Shuicheng Yan, Bo Li

LGR module utilizes body skeleton knowledge to construct a layout graph that connects all relevant part features, where graph reasoning mechanism is used to propagate information among part nodes to mine their relations.

TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search

1 code implementation ECCV 2020 Yibo Hu, Xiang Wu, Ran He

In this paper, we rethink three freedoms of differentiable NAS, i. e. operation-level, depth-level and width-level, and propose a novel method, named Three-Freedom NAS (TF-NAS), to achieve both good classification accuracy and precise latency constraint.

Neural Architecture Search

Deep Momentum Uncertainty Hashing

no code implementations17 Sep 2020 Chaoyou Fu, Guoli Wang, Xiang Wu, Qian Zhang, Ran He

It embodies the uncertainty of the hashing network to the corresponding input image.

Combinatorial Optimization Deep Hashing

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition

1 code implementation20 Sep 2020 Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, Ran He

As a consequence, massive new diverse paired heterogeneous images with the same identity can be generated from noises.

Contrastive Learning Face Recognition +1

Graph Information Bottleneck for Subgraph Recognition

1 code implementation ICLR 2021 Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, Ran He

In this paper, we propose a framework of Graph Information Bottleneck (GIB) for the subgraph recognition problem in deep graph learning.

Denoising Graph Classification +1

Free-Form Image Inpainting via Contrastive Attention Network

no code implementations29 Oct 2020 Xin Ma, Xiaoqiang Zhou, Huaibo Huang, Zhenhua Chai, Xiaolin Wei, Ran He

It is difficult for encoders to capture such powerful representations under this complex situation.

Image Inpainting

AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection

no code implementations NeurIPS 2020 Hao Zhu, Chaoyou Fu, Qianyi Wu, Wayne Wu, Chen Qian, Ran He

However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation.

Lets Play Music: Audio-driven Performance Video Generation

no code implementations5 Nov 2020 Hao Zhu, Yi Li, Feixia Zhu, Aihua Zheng, Ran He

We propose a new task named Audio-driven Per-formance Video Generation (APVG), which aims to synthesizethe video of a person playing a certain instrument guided bya given music audio clip.

Video Generation

Unsupervised Contrastive Photo-to-Caricature Translation based on Auto-distortion

no code implementations10 Nov 2020 Yuhe Ding, Xin Ma, Mandi Luo, Aihua Zheng, Ran He

Considering the intuitive artifacts in the existing methods, we propose a contrastive style loss for style rendering to enforce the similarity between the style of rendered photo and the caricature, and simultaneously enhance its discrepancy to the photos.

Caricature Photo-To-Caricature Translation +1

Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer

2 code implementations14 Dec 2020 Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, Jiashi Feng

Furthermore, we propose a new labeling transfer strategy, which separates the target data into two splits based on the confidence of predictions (labeling information), and then employ semi-supervised learning to improve the accuracy of less-confident predictions in the target domain.

Classification General Classification +3

CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification

1 code implementation ICCV 2021 Chaoyou Fu, Yibo Hu, Xiang Wu, Hailin Shi, Tao Mei, Ran He

Visible-Infrared person re-identification (VI-ReID) aims to match cross-modality pedestrian images, breaking through the limitation of single-modality person ReID in dark environment.

Neural Architecture Search Person Re-Identification

ReMix: Towards Image-to-Image Translation with Limited Data

1 code implementation CVPR 2021 Jie Cao, Luanxuan Hou, Ming-Hsuan Yang, Ran He, Zhenan Sun

We interpolate training samples at the feature level and propose a novel content loss based on the perceptual relations among samples.

Data Augmentation Image-to-Image Translation +1

DINE: Domain Adaptation from Single and Multiple Black-box Predictors

3 code implementations CVPR 2022 Jian Liang, Dapeng Hu, Jiashi Feng, Ran He

To ease the burden of labeling, unsupervised domain adaptation (UDA) aims to transfer knowledge in previous and related labeled datasets (sources) to a new unlabeled dataset (target).

Transductive Learning Unsupervised Domain Adaptation

Everything's Talkin': Pareidolia Face Reenactment

1 code implementation7 Apr 2021 Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, Ran He

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.

Face Reenactment Texture Synthesis

PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal

1 code implementation26 May 2021 Si Liu, Wentao Jiang, Chen Gao, Ran He, Jiashi Feng, Bo Li, Shuicheng Yan

In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively.

Style Transfer

Memory Oriented Transfer Learning for Semi-Supervised Image Deraining

no code implementations CVPR 2021 Huaibo Huang, Aijing Yu, Ran He

To address this issue, we propose a memory-oriented semi-supervised (MOSS) method which enables the network to explore and exploit the properties of rain streaks from both synthetic and real data.

Rain Removal Transfer Learning

Information Bottleneck Disentanglement for Identity Swapping

1 code implementation CVPR 2021 Gege Gao, Huaibo Huang, Chaoyou Fu, Zhaoyang Li, Ran He

In this work, we propose a novel information disentangling and swapping network, called InfoSwap, to extract the most expressive information for identity representation from a pre-trained face recognition model.

Disentanglement Face Recognition +1

Pareidolia Face Reenactment

no code implementations CVPR 2021 Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, Ran He

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.

Face Reenactment Texture Synthesis

FaceInpainter: High Fidelity Face Adaptation to Heterogeneous Domains

no code implementations CVPR 2021 Jia Li, Zhaoyang Li, Jie Cao, Xingguang Song, Ran He

In this work, we propose a novel two-stage framework named FaceInpainter to implement controllable Identity-Guided Face Inpainting (IGFI) under heterogeneous domains.

Attribute Facial Inpainting +1

Universal Face Restoration With Memorized Modulation

no code implementations3 Oct 2021 Jia Li, Huaibo Huang, Xiaofei Jia, Ran He

Blind face restoration (BFR) is a challenging problem because of the uncertainty of the degradation patterns.

Blind Face Restoration

Causal Representation Learning for Context-Aware Face Transfer

no code implementations4 Oct 2021 Gege Gao, Huaibo Huang, Chaoyou Fu, Ran He

Human face synthesis involves transferring knowledge about the identity and identity-dependent face shape (IDFS) of a human face to target face images where the context (e. g., facial expressions, head poses, and other background factors) may change dramatically.

counterfactual Counterfactual Inference +4

Toward Accurate and Reliable Iris Segmentation Using Uncertainty Learning

no code implementations20 Oct 2021 Jianze Wei, Huaibo Huang, Muyi Sun, Yunlong Wang, Min Ren, Ran He, Zhenan Sun

To make further efforts on accurate and reliable iris segmentation, we propose a bilateral self-attention module and design Bilateral Transformer (BiTrans) with hierarchical architecture by exploring spatial and visual relationships.

Iris Recognition Iris Segmentation +1

UMAD: Universal Model Adaptation under Domain and Category Shift

no code implementations16 Dec 2021 Jian Liang, Dapeng Hu, Jiashi Feng, Ran He

To achieve bilateral adaptation in the target domain, we further maximize localized mutual information to align known samples with the source classifier and employ an entropic loss to push unknown samples far away from the source classification boundary, respectively.

Universal Domain Adaptation Unsupervised Domain Adaptation

Towards the Explanation of Graph Neural Networks in Digital Pathology with Information Flows

no code implementations18 Dec 2021 Junchi Yu, Tingyang Xu, Ran He

In this work, we address these key challenges and propose IFEXPLAINER, which generates a necessary and sufficient explanation for GNNs.

Improving Subgraph Recognition with Variational Graph Information Bottleneck

1 code implementation CVPR 2022 Junchi Yu, Jie Cao, Ran He

Subgraph recognition aims at discovering a compressed substructure of a graph that is most informative to the graph property.

Graph Classification

Few-shot Backdoor Defense Using Shapley Estimation

no code implementations CVPR 2022 Jiyang Guan, Zhuozhuo Tu, Ran He, DaCheng Tao

Deep neural networks have achieved impressive performance in a variety of tasks over the last decade, such as autonomous driving, face recognition, and medical diagnosis.

Autonomous Driving backdoor defense +2

Styleverse: Towards Identity Stylization across Heterogeneous Domains

no code implementations2 Mar 2022 Jia Li, Jie Cao, Junxian Duan, Ran He

We propose a new challenging task namely IDentity Stylization (IDS) across heterogeneous domains.

Style Transfer

ProxyMix: Proxy-based Mixup Training with Label Refinery for Source-Free Domain Adaptation

2 code implementations29 May 2022 Yuhe Ding, Lijun Sheng, Jian Liang, Aihua Zheng, Ran He

First of all, to avoid additional parameters and explore the information in the source model, ProxyMix defines the weights of the classifier as the class prototypes and then constructs a class-balanced proxy source domain by the nearest neighbors of the prototypes to bridge the unseen source domain and the target domain.

Object Recognition Source-Free Domain Adaptation +1

Finding Diverse and Predictable Subgraphs for Graph Domain Generalization

no code implementations19 Jun 2022 Junchi Yu, Jian Liang, Ran He

Extensive experiments on both node-level and graph-level benchmarks shows that the proposed DPS achieves impressive performance for various graph domain generalization tasks.

Domain Generalization Out-of-Distribution Generalization

Parallel Augmentation and Dual Enhancement for Occluded Person Re-identification

1 code implementation11 Oct 2022 Zi Wang, Huaibo Huang, Aihua Zheng, Chenglong Li, Ran He

To alleviate these two issues, we propose a simple yet effective method with Parallel Augmentation and Dual Enhancement (PADE), which is robust on both occluded and non-occluded data and does not require any auxiliary clues.

Person Re-Identification

Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks

1 code implementation21 Oct 2022 Jiyang Guan, Jian Liang, Ran He

To reduce the training time, we further develop SAC-m that selects CutMix Augmented samples as model inputs, without the need for training the surrogate models or generating adversarial examples.

Adversarial Defense Transfer Learning

ScoreMix: A Scalable Augmentation Strategy for Training GANs with Limited Data

no code implementations27 Oct 2022 Jie Cao, Mandi Luo, Junchi Yu, Ming-Hsuan Yang, Ran He

Then, we optimize the augmented samples by minimizing the norms of the data scores, i. e., the gradients of the log-density functions.

Data Augmentation Image Generation

Vision Transformer with Super Token Sampling

1 code implementation CVPR 2023 Huaibo Huang, Xiaoqiang Zhou, Jie Cao, Ran He, Tieniu Tan

STA decomposes vanilla global attention into multiplications of a sparse association map and a low-dimensional attention, leading to high efficiency in capturing global dependencies.

Semantic Segmentation Superpixels

MSRA-SR: Image Super-resolution Transformer with Multi-scale Shared Representation Acquisition

no code implementations ICCV 2023 Xiaoqiang Zhou, Huaibo Huang, Ran He, Zilei Wang, Jie Hu, Tieniu Tan

In particular, self-attention with cross-scale matching and convolution filters with different kernel sizes are designed to exploit the multi-scale features in images.

Image Super-Resolution

MAPS: A Noise-Robust Progressive Learning Approach for Source-Free Domain Adaptive Keypoint Detection

1 code implementation9 Feb 2023 Yuhe Ding, Jian Liang, Bo Jiang, Aihua Zheng, Ran He

Existing cross-domain keypoint detection methods always require accessing the source data during adaptation, which may violate the data privacy law and pose serious security concerns.

Data Augmentation Keypoint Detection

Masked Relation Learning for DeepFake Detection

2 code implementations 2023 2023 Ziming Yang, Jian Liang, Yuting Xu, Xiao-Yu Zhang, Ran He

A relation learning module masks partial correlations between regions to reduce redundancy and then propagates the relational information across regions to capture the irregularity from a global view of the graph.

Binary Classification DeepFake Detection +3

MODIFY: Model-driven Face Stylization without Style Images

1 code implementation17 Mar 2023 Yuhe Ding, Jian Liang, Jie Cao, Aihua Zheng, Ran He

Briefly, MODIFY first trains a generative model in the target domain and then translates a source input to the target domain via the provided style model.

Translation

AdaptGuard: Defending Against Universal Attacks for Model Adaptation

1 code implementation ICCV 2023 Lijun Sheng, Jian Liang, Ran He, Zilei Wang, Tieniu Tan

To address this issue, we propose a model preprocessing framework, named AdaptGuard, to improve the security of model adaptation algorithms.

Knowledge Distillation Transfer Learning

Pluralistic Aging Diffusion Autoencoder

no code implementations ICCV 2023 Peipei Li, Rui Wang, Huaibo Huang, Ran He, Zhaofeng He

Face aging is an ill-posed problem because multiple plausible aging patterns may correspond to a given input.

Denoising

AUTO: Adaptive Outlier Optimization for Online Test-Time OOD Detection

1 code implementation22 Mar 2023 Puning Yang, Jian Liang, Jie Cao, Ran He

Out-of-distribution (OOD) detection is a crucial aspect of deploying machine learning models in open-world applications.

Out of Distribution (OOD) Detection

A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts

1 code implementation27 Mar 2023 Jian Liang, Ran He, Tieniu Tan

Test-time adaptation (TTA), an emerging paradigm, has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions.

Source-Free Domain Adaptation Test-time Adaptation

Mind the Label Shift of Augmentation-based Graph OOD Generalization

1 code implementation CVPR 2023 Junchi Yu, Jian Liang, Ran He

Recent works employ different graph editions to generate augmented environments and learn an invariant GNN for generalization.

Rethinking Local Perception in Lightweight Vision Transformer

1 code implementation31 Mar 2023 Qihang Fan, Huaibo Huang, Jiyang Guan, Ran He

The combination of the AttnConv and vanilla attention which uses pooling to reduce FLOPs in CloFormer enables the model to perceive high-frequency and low-frequency information.

Image Classification object-detection +2

Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer

no code implementations31 Mar 2023 Yuang Ai, Xiaoqiang Zhou, Huaibo Huang, Lei Zhang, Ran He

Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image Super-Resolution (SR) by accessing both the source and target data.

Image Super-Resolution Source-Free Domain Adaptation +1

Lightweight Vision Transformer with Bidirectional Interaction

1 code implementation NeurIPS 2023 Qihang Fan, Huaibo Huang, Xiaoqiang Zhou, Ran He

This paper proposes a Fully Adaptive Self-Attention (FASA) mechanism for vision transformer to model the local and global information as well as the bidirectional interaction between them in context-aware ways.

Boosting Fast and High-Quality Speech Synthesis with Linear Diffusion

no code implementations9 Jun 2023 Haogeng Liu, Tao Wang, Jie Cao, Ran He, JianHua Tao

When decreasing the number of sampling steps (i. e., the number of line segments used to fit the path), the ease of fitting straight lines compared to curves allows us to generate higher quality samples from a random noise with fewer iterations.

Denoising Speech Synthesis

Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification

2 code implementations NeurIPS 2023 Rui Wang, Peipei Li, Huaibo Huang, Chunshui Cao, Ran He, Zhaofeng He

Consequently, we propose a cross-modal ordinal pairwise loss to refine the CLIP feature space, where texts and images maintain both semantic alignment and ordering alignment.

Age Estimation Classification +2

Benchmarking Test-Time Adaptation against Distribution Shifts in Image Classification

1 code implementation6 Jul 2023 Yongcan Yu, Lijun Sheng, Ran He, Jian Liang

To implement this benchmark, we have developed a unified framework in PyTorch, which allows for consistent evaluation and comparison of the TTA methods across the different datasets and network architectures.

Benchmarking Image Classification +1

TALL: Thumbnail Layout for Deepfake Video Detection

1 code implementation ICCV 2023 Yuting Xu, Jian Liang, Gengyun Jia, Ziming Yang, Yanhao Zhang, Ran He

This paper introduces a simple yet effective strategy named Thumbnail Layout (TALL), which transforms a video clip into a pre-defined layout to realize the preservation of spatial and temporal dependencies.

Face Swapping

Towards Realistic Unsupervised Fine-tuning with CLIP

no code implementations24 Aug 2023 Jian Liang, Lijun Sheng, Zhengbo Wang, Ran He, Tieniu Tan

The emergence of vision-language models (VLMs), such as CLIP, has spurred a significant research effort towards their application for downstream supervised learning tasks.

Out-of-Distribution Detection

Learning Cross-modality Information Bottleneck Representation for Heterogeneous Person Re-Identification

no code implementations29 Aug 2023 Haichao Shi, Mandi Luo, Xiao-Yu Zhang, Ran He

Visible-Infrared person re-identification (VI-ReID) is an important and challenging task in intelligent video surveillance.

Person Re-Identification

Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis

no code implementations31 Aug 2023 Linsen Song, Wayne Wu, Chaoyou Fu, Chen Change Loy, Ran He

Existing automated dubbing methods are usually designed for Professionally Generated Content (PGC) production, which requires massive training data and training time to learn a person-specific audio-video mapping.

RMT: Retentive Networks Meet Vision Transformers

1 code implementation20 Sep 2023 Qihang Fan, Huaibo Huang, Mingrui Chen, Hongmin Liu, Ran He

To alleviate these issues, we draw inspiration from the recent Retentive Network (RetNet) in the field of NLP, and propose RMT, a strong vision backbone with explicit spatial prior for general purposes.

Instance Segmentation object-detection +2

Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models

1 code implementation6 Oct 2023 Junchi Yu, Ran He, Rex Ying

These analogous problems are related to the input one, with reusable solutions and problem-solving strategies.

Prompt Engineering

Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling

no code implementations8 Oct 2023 Haogeng Liu, Qihang Fan, Tingkai Liu, Linjie Yang, Yunzhe Tao, Huaibo Huang, Ran He, Hongxia Yang

This paper proposes Video-Teller, a video-language foundation model that leverages multi-modal fusion and fine-grained modality alignment to significantly enhance the video-to-text generation task.

Text Generation Video Summarization

Video-CSR: Complex Video Digest Creation for Visual-Language Models

no code implementations8 Oct 2023 Tingkai Liu, Yunzhe Tao, Haogeng Liu, Qihang Fan, Ding Zhou, Huaibo Huang, Ran He, Hongxia Yang

We present a novel task and human annotated dataset for evaluating the ability for visual-language models to generate captions and summaries for real-world video clips, which we call Video-CSR (Captioning, Summarization and Retrieval).

Retrieval Sentence +1

ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models

1 code implementation11 Oct 2023 Yingqing He, Shaoshu Yang, Haoxin Chen, Xiaodong Cun, Menghan Xia, Yong Zhang, Xintao Wang, Ran He, Qifeng Chen, Ying Shan

Our work also suggests that a pre-trained diffusion model trained on low-resolution images can be directly used for high-resolution visual generation without further tuning, which may provide insights for future research on ultra-high-resolution image and video synthesis.

Image Generation

Exploring Straighter Trajectories of Flow Matching with Diffusion Guidance

no code implementations28 Nov 2023 Siyu Xing, Jie Cao, Huaibo Huang, Xiao-Yu Zhang, Ran He

First, we propose a coupling strategy to straighten trajectories, creating couplings between image and noise samples under diffusion model guidance.

Portrait Diffusion: Training-free Face Stylization with Chain-of-Painting

1 code implementation3 Dec 2023 Jin Liu, Huaibo Huang, Chao Jin, Ran He

Face stylization refers to the transformation of a face into a specific portrait style.

Image Reconstruction

Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration

no code implementations5 Dec 2023 Yuang Ai, Huaibo Huang, Xiaoqiang Zhou, Jiexiang Wang, Ran He

Extensive experiments on 16 IR tasks underscore the superiority of MPerceiver in terms of adaptiveness, generalizability and fidelity.

Image Restoration

Not all Minorities are Equal: Empty-Class-Aware Distillation for Heterogeneous Federated Learning

no code implementations4 Jan 2024 Kuangpu Guo, Yuhe Ding, Jian Liang, Ran He, Zilei Wang, Tieniu Tan

Data heterogeneity, characterized by disparities in local data distribution across clients, poses a significant challenge in federated learning.

Federated Learning Knowledge Distillation

Towards Eliminating Hard Label Constraints in Gradient Inversion Attacks

1 code implementation5 Feb 2024 Yanbo Wang, Jian Liang, Ran He

Even for single-image reconstruction, we still lack an analysis-based algorithm to recover augmented soft labels.

Federated Learning Image Reconstruction

Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models

no code implementations6 Feb 2024 Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, Tieniu Tan

This paper proposes a \textbf{C}ollabo\textbf{ra}tive \textbf{F}ine-\textbf{T}uning (\textbf{CraFT}) approach for fine-tuning black-box VLMs to downstream tasks, where one only has access to the input prompts and the output predictions of the model.

A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation

1 code implementation6 Feb 2024 Zhengbo Wang, Jian Liang, Lijun Sheng, Ran He, Zilei Wang, Tieniu Tan

Extensive results on 17 datasets validate that our method surpasses or achieves comparable results with state-of-the-art methods on few-shot classification, imbalanced learning, and out-of-distribution generalization.

Out-of-Distribution Generalization

CSCNET: Class-Specified Cascaded Network for Compositional Zero-Shot Learning

no code implementations9 Mar 2024 Yanyi Zhang, Qi Jia, Xin Fan, Yu Liu, Ran He

Inspired by this, we propose a novel A-O disentangled framework for CZSL, namely Class-specified Cascaded Network (CSCNet).

Attribute Compositional Zero-Shot Learning +2

DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration

no code implementations15 Mar 2024 Nan Gao, Jia Li, Huaibo Huang, Zhi Zeng, Ke Shang, Shuwu Zhang, Ran He

Experimental results demonstrate the superiority of DiffMAC over state-of-the-art methods, with a high degree of generalization in real-world and heterogeneous settings.

Attribute Blind Face Restoration +1

ViTAR: Vision Transformer with Any Resolution

no code implementations27 Mar 2024 Qihang Fan, Quanzeng You, Xiaotian Han, Yongfei Liu, Yunzhe Tao, Huaibo Huang, Ran He, Hongxia Yang

Firstly, we propose a novel module for dynamic resolution adjustment, designed with a single Transformer block, specifically to achieve highly efficient incremental token integration.

Hierarchical Face Aging through Disentangled Latent Characteristics

no code implementations ECCV 2020 Pei-Pei Li, Huaibo Huang, Yibo Hu, Xiang Wu, Ran He, Zhenan Sun

To explore the age effects on facial images, we propose a Disentangled Adversarial Autoencoder (DAAE) to disentangle the facial images into three independent factors: age, identity and extraneous information.

Age Estimation MORPH

Cannot find the paper you are looking for? You can Submit a new open access paper.