Constrained Lip-synchronization

6 papers with code • 0 benchmarks • 0 datasets

This task deals with lip-syncing a video (or) an image to the desired target speech. Approaches in this task work only for a specific (limited set) of identities, languages, speech/voice. See also: Unconstrained lip-synchronization -

Most implemented papers

ObamaNet: Photo-realistic lip-sync from text

ung200/thats-what-obama-said 6 Dec 2017

We present ObamaNet, the first architecture that generates both audio and synchronized photo-realistic lip-sync videos from any new text.

Talking Face Generation by Conditional Recurrent Adversarial Network

susanqq/Talking_Face_Generation 13 Apr 2018

Given an arbitrary face image and an arbitrary speech clip, the proposed work attempts to generating the talking face video with accurate lip synchronization while maintaining smooth transition of both lip and facial movement over the entire video clip.

Dynamic Temporal Alignment of Speech to Lips

tavihalperin/AV-sync 19 Aug 2018

This alignment is based on deep audio-visual features, mapping the lips video and the speech signal to a shared representation.

DeepFakes: a New Threat to Face Recognition? Assessment and Detection

CatoGit/Comparing-the-Performance-of-Deepfake-Detection-Methods-on-Benchmark-Datasets 20 Dec 2018

The best performing method, which is based on visual quality metrics and is often used in presentation attack detection domain, resulted in 8. 97% equal error rate on high quality Deepfakes.

Real-Time Lip Sync for Live 2D Animation

deepalianeja/CharacterLipSync2D 19 Oct 2019

The emergence of commercial tools for real-time performance-based 2D animation has enabled 2D characters to appear on live broadcasts and streaming platforms.

Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localization

abhinavdhall/deepfake 29 May 2020

MDS is computed as an aggregate of dissimilarity scores between audio and visual segments in a video.