Talking Head Generation Edit Task

Computer Vision • 3D Absolute Human Pose Estimation

papers with code

benchmarks

datasets

About
Edit

Talking head generation is the task of generating a talking face from a set of images of a person.

( Image credit: Few-Shot Adversarial Learning of Realistic Neural Talking Head Models )

Benchmarks

Add a Result

DATASET	BEST METHOD	PAPER TITLE	COMPARE
VoxCeleb2 - 1-shot learning	Fast Bi-layer Avatars (medium size)	Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars	See all
VoxCeleb1 - 1-shot learning	Few-shot Adversarial Model	Few-Shot Adversarial Learning of Realistic Neural Talking Head Models	See all
VoxCeleb1 - 8-shot learning	Few-shot Adversarial Model	Few-Shot Adversarial Learning of Realistic Neural Talking Head Models	See all
VoxCeleb1 - 32-shot learning	Few-shot Adversarial Model	Few-Shot Adversarial Learning of Realistic Neural Talking Head Models	See all
VoxCeleb2 - 8-shot learning	CainGAN	Pose Manipulation with Identity Preservation	See all
VoxCeleb2 - 32-shot learning	Few-shot Adversarial Model	Few-Shot Adversarial Learning of Realistic Neural Talking Head Models	See all
100 sleep nights of 8 caregivers	Ashok	0-Step Capturability, Motion Decomposition and Global Feedback Control of the 3D Variable Height-Inverted Pendulum	See all

Subtasks

Unconstrained Lip-synchronization

3 benchmarks

3 papers with code

Datasets

VoxCeleb1

VoxCeleb2

Greatest papers with code

A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 Aug 2020 • Rudrabha/Wav2Lip •

However, they fail to accurately morph the lip movements of arbitrary identities in dynamic, unconstrained talking face videos, resulting in significant parts of the video being out-of-sync with the new audio.

Ranked #1 on Unconstrained Lip-synchronization on LRW

UNCONSTRAINED LIP-SYNCHRONIZATION

2,225

Paper
Code

Few-Shot Adversarial Learning of Realistic Neural Talking Head Models

ICCV 2019 • vincent-thevenin/Realistic-Neural-Talking-Head-Models •

In order to create a personalized talking head model, these works require training on a large dataset of images of a single person.

Ranked #1 on Talking Head Generation on VoxCeleb1 - 32-shot learning

META-LEARNING ONE-SHOT LEARNING TALKING HEAD GENERATION

627

Paper
Code

Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars

ECCV 2020 • saic-violet/bilayer-model •

The texture image is generated offline, warped and added to the coarse image to ensure a high effective resolution of synthesized head views.

Ranked #1 on Talking Head Generation on VoxCeleb2 - 1-shot learning

NEURAL RENDERING TALKING HEAD GENERATION

154

Paper
Code

ReenactGAN: Learning to Reenact Faces via Boundary Transfer

ECCV 2018 • wywu/ReenactGAN •

A transformer is subsequently used to adapt the boundary of source face to the boundary of target face.

FACE REENACTMENT TALKING FACE GENERATION TALKING HEAD GENERATION

149

Paper
Code

Talking-head Generation with Rhythmic Head Motion

16 Jul 2020 • lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion •

When people deliver a speech, they naturally move heads, and this rhythmic head motion conveys prosodic information.

TALKING HEAD GENERATION

108

Paper
Code

What comprises a good talking-head video generation?: A Survey and Benchmark

7 May 2020 • lelechen63/talking-head-generation-survey

In this work, we present a carefully-designed benchmark for evaluating talking-head video generation with standardized dataset pre-processing strategies.

TALKING HEAD GENERATION VIDEO GENERATION