ModEFormer: Modality-Preserving Embedding for Audio-Video Synchronization using Transformers

no code implementations • 21 Mar 2023 • Akash Gupta, Rohun Tripathi, WonDong Jang

Lack of audio-video synchronization is a common problem during television broadcasts and video conferencing, leading to an unsatisfactory viewing experience.

Contrastive Learning Video Synchronization

SIDGAN: High-Resolution Dubbed Video Generation via Shift-Invariant Learning

no code implementations • ICCV 2023 • Urwa Muaz, WonDong Jang, Rohun Tripathi, Santhosh Mani, Wenbin Ouyang, Ravi Teja Gadde, Baris Gecer, Sergio Elizondo, Reza Madad, Naveen Nair

Dubbed video generation aims to accurately synchronize mouth movements of a given facial video with driving audio while preserving identity and scene-specific visual dynamics, such as head pose and lighting.

Image Generation Video Generation +1