FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

19 Oct 2022  ·  Felix Rosberg, Eren Erdal Aksoy, Fernando Alonso-Fernandez, Cristofer Englund ·

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identityn transfer, while having significantly better pose preservation than most of the previous methods.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Face Swapping AFLW2000-3D FaceDancer (Config C) ID retrieval 97.65 # 3
pose 5.82 # 2
exp embedding L2 4.13 # 4
Face Swapping AFLW2000-3D FaceDancer (Config D) ID retrieval 97.10 # 4
pose 5.75 # 1
exp embedding L2 4.15 # 3
Face Swapping AFLW2000-3D FaceDancer (Config A) ID retrieval 98.5 # 1
pose 14.97 # 4
exp embedding L2 7.07 # 1
Face Swapping AFLW2000-3D FaceDancer (Config B) ID retrieval 97.95 # 2
pose 5.86 # 3
exp embedding L2 5.74 # 2
Face Swapping FaceForensics++ FaceDancer (Config D) ID retrieval 98.19 # 5
pose 2.15 # 4
exp embedding L2 5.70 # 3
Face Swapping FaceForensics++ FaceDancer (Config B) ID retrieval 98.54 # 3
pose 2.24 # 6
exp embedding L2 8.52 # 1
Face Swapping FaceForensics++ FaceDancer (Config C) ID retrieval 98.84 # 2
pose 2.04 # 3
exp embedding L2 7.97 # 2

Methods


No methods listed for this paper. Add relevant methods here