Towards Automatic Face-to-Face Translation

ACM Multimedia, 2019 2019 Prajwal K RRudrabha MukhopadhyayJerin PhilipAbhishek JhaVinay NamboodiriC. V. Jawahar

In light of the recent breakthroughs in automatic machine translation systems, we propose a novel approach that we term as "Face-to-Face Translation". As today's digital communication becomes increasingly visual, we argue that there is a need for systems that can automatically translate a video of a person speaking in language A into a target language B with realistic lip synchronization... (read more)

PDF Abstract ACM Multimedia, 2019 2019 PDF ACM Multimedia, 2019 2019 Abstract

Results from the Paper


 Ranked #1 on Talking Face Generation on LRW (using extra training data)

     Get a GitHub badge
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
Talking Face Generation LRW LipGAN LMD 0.60 # 1
SSIM 0.96 # 1

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet