Talking Face Generation Edit Task
Computer Vision • Face Generation
About
Talking face generation aims to synthesize a sequence of face images that correspond to given speech semantics.
( Image credit: Talking Face Generation by Adversarially Disentangled Audio-Visual Representation )
Benchmarks
| TREND | DATASET | BEST METHOD | PAPER TITLE | PAPER | CODE | COMPARE |
|---|---|---|---|---|---|---|
|
Subtasks
Datasets
Greatest papers with code
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild
• Rudrabha/Wav2Lip
• 
However, they fail to accurately morph the lip movements of arbitrary identities in dynamic, unconstrained talking face videos, resulting in significant parts of the video being out-of-sync with the new audio.
Ranked #1 on
Unconstrained Lip-synchronization
on LRW
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation
• Hangz-nju-cuhk/Talking-Face-Generation-DAVS
•
Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech.
Capture, Learning, and Synthesis of 3D Speaking Styles
CVPR 2019
• TimoBolkart/voca
• 
To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers.
Hierarchical Cross-Modal Talking Face Generation With Dynamic Pixel-Wise Loss
CVPR 2019
• lelechen63/ATVGnet
•
We devise a cascade GAN approach to generate talking face video, which is robust to different face shapes, view angles, facial characteristics, and noisy audio conditions.
Talking Face Generation by Conditional Recurrent Adversarial Network
• susanqq/Talking_Face_Generation
•
Given an arbitrary face image and an arbitrary speech clip, the proposed work attempts to generating the talking face video with accurate lip synchronization while maintaining smooth transition of both lip and facial movement over the entire video clip.