Search Results for author:
Found 25 papers, 10 papers with code
PhraseOut: A Code Mixed Data Augmentation Method for MultilingualNeural Machine Tranlsation
Back-translation aug- ments parallel data by translating monolingual sentences in the target side to source language.
Towards Accurate Lip-to-Speech Synthesis in-the-Wild
In this paper, we introduce a novel approach to address the task of synthesizing speech from silent videos of any in-the-wild speaker solely based on lip movements.
Dubbing for Everyone: Data-Efficient Visual Dubbing using Neural Rendering Priors
Our prior learning and adaptation method $\textbf{generalises to limited data}$ better and is more $\textbf{scalable}$ than existing person-specific models.
FACTS: Facial Animation Creation using the Transfer of Styles
The ability to accurately capture and express emotions is a critical aspect of creating believable characters in video games and other forms of entertainment.
READ Avatars: Realistic Emotion-controllable Audio Driven Avatars
We present READ Avatars, a 3D-based approach for generating 2D avatars that are driven by audio input with direct and granular control over the emotion.
Audio-Visual Face Reenactment
The identity-aware generator takes the source image and the warped motion features as input to generate a high-quality output with fine-grained details.
FaceOff: A Video-to-Video Face Swapping System
To tackle this challenge, we introduce video-to-video (V2V) face-swapping, a novel task of face-swapping that can preserve (1) the identity and expressions of the source (actor) face video and (2) the background and pose of the target (double) video.
Towards MOOCs for Lipreading: Using Synthetic Talking Heads to Train Humans in Lipreading at Scale
Because of the manual pipeline, such platforms are also limited in vocabulary, supported languages, accents, and speakers and have a high usage cost.
Personalized One-Shot Lipreading for an ALS Patient
Apart from evaluating our approach on the ALS patient, we also extend it to people with hearing impairment relying extensively on lip movements to communicate.
Towards Automatic Speech to Sign Language Generation
Since the current datasets are inadequate for generating sign language directly from speech, we collect and release the first Indian sign language dataset comprising speech-level annotations, text transcripts, and the corresponding sign-language videos.
Knowledge Consolidation based Class Incremental Online Learning with Limited Data
In our approach, we learn robust representations that are generalizable across tasks without suffering from the problems of catastrophic forgetting and overfitting to accommodate future classes with limited samples.
Visual Speech Enhancement Without A Real Visual Stream
In this work, we re-think the task of speech enhancement in unconstrained real-world environments.
Ranked #1 on
Speech Denoising
on LRS3+VGGSound
STEER : Simple Temporal Regularization For Neural ODE
Training Neural Ordinary Differential Equations (ODEs) is often computationally expensive.
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild
However, they fail to accurately morph the lip movements of arbitrary identities in dynamic, unconstrained talking face videos, resulting in significant parts of the video being out-of-sync with the new audio.
Ranked #1 on
Unconstrained Lip-synchronization
on LRS3
(using extra training data)
STEER: Simple Temporal Regularization For Neural ODEs
Training Neural Ordinary Differential Equations (ODEs) is often computationally expensive.
Learning Individual Speaking Styles for Accurate Lip to Speech Synthesis
In this work, we explore the task of lip to speech synthesis, i. e., learning to generate natural speech given only the lip movements of a speaker.
Ranked #1 on
Lip Reading
on LRW
CovidAID: COVID-19 Detection Using Chest X-Ray
This may be useful in an inpatient setting where the present systems are struggling to decide whether to keep the patient in the ward along with other patients or isolate them in COVID-19 areas.
Towards Automatic Face-to-Face Translation
As today's digital communication becomes increasingly visual, we argue that there is a need for systems that can automatically translate a video of a person speaking in language A into a target language B with realistic lip synchronization.
Ranked #1 on
Talking Face Generation
on LRW
(using extra training data)
CVIT's submissions to WAT-2019
This paper describes the Neural Machine Translation systems used by IIIT Hyderabad (CVIT-MT) for the translation tasks part of WAT-2019.
CPWC: Contextual Point Wise Convolution for Object Recognition
We propose an alternative design for pointwise convolution, which uses spatial information from the input efficiently.
Multimodal Differential Network for Visual Question Generation
Generating natural questions from an image is a semantic task that requires using visual and language modality to learn multimodal representations.
Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator
One way to ensure this is by adding constraints for true paraphrase embeddings to be close and unrelated paraphrase candidate sentence embeddings to be far.
Multi-Agent Diverse Generative Adversarial Networks
Second, to enforce that different generators capture diverse high probability modes, the discriminator of MAD-GAN is designed such that along with finding the real and fake samples, it is also required to identify the generator that generated the given fake sample.
Message Passing Multi-Agent GANs
As a first step towards this challenge, we introduce a novel framework for image generation: Message Passing Multi-Agent Generative Adversarial Networks (MPM GANs).
Contextual RNN-GANs for Abstract Reasoning Diagram Generation
Understanding, predicting, and generating object motions and transformations is a core problem in artificial intelligence.
