Search Results for author: Neha Sahipjohn

Found 3 papers, 0 papers with code

DubWise: Video-Guided Speech Duration Control in Multimodal LLM-based Text-to-Speech for Dubbing

no code implementations13 Jun 2024 Neha Sahipjohn, Ashishkumar Gudmalwar, Nirmesh Shah, Pankaj Wasnik, Rajiv Ratn Shah

To this end, we propose a novel method, DubWise Multi-modal Large Language Model (LLM)-based Text-to-Speech (TTS), which can control the speech duration of synthesized speech in such a way that it aligns well with the speakers lip movements given in the reference video even when the spoken text is different or in a different language.

Language Modeling Language Modelling +3

RobustL2S: Speaker-Specific Lip-to-Speech Synthesis exploiting Self-Supervised Representations

no code implementations3 Jul 2023 Neha Sahipjohn, Neil Shah, Vishal Tambrahalli, Vineet Gandhi

Significant progress has been made in speaker dependent Lip-to-Speech synthesis, which aims to generate speech from silent videos of talking faces.

Speaker-Specific Lip to Speech Synthesis Speech Synthesis

Cannot find the paper you are looking for? You can Submit a new open access paper.