Search Results for author:
Found 42 papers, 15 papers with code
Empathic Machines: Using Intermediate Features as Levers to Emulate Emotions in Text-To-Speech Systems
We present a method to control the emotional prosody of Text to Speech (TTS) systems by using phoneme-level intermediate features (pitch, energy, and duration) as levers.
EditIQ: Automated Cinematic Editing of Static Wide-Angle Videos via Dialogue Interpretation and Saliency Cues
We present EditIQ, a completely automated framework for cinematically editing scenes captured via a stationary, large field-of-view and high-resolution camera.
Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues
This paper introduces ViNet-S, a 36MB model based on the ViNet architecture with a U-Net design, featuring a lightweight decoder that significantly reduces model size and parameters without compromising performance.
MRI2Speech: Speech Synthesis from Articulatory Movements Recorded by Real-time MRI
Previous real-time MRI (rtMRI)-based speech synthesis models depend heavily on noisy ground-truth speech.
Advancing NAM-to-Speech Conversion with Novel Methods and the MultiNAM Dataset
To address this issue, we focus on learning phoneme-level alignments from paired whispers and text and employ a Text-to-Speech (TTS) system to simulate the ground-truth.
TIDE: Training Locally Interpretable Domain Generalization Models Enables Test-time Correction
Our next innovation is TIDE, a novel training scheme with a concept saliency alignment loss that ensures model focus on the right per-concept regions and a local concept contrastive loss that promotes learning domain-invariant concept representations.
IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark
The benchmark also consists of a curated mixture of different mention types and corresponding entities, allowing for a fine-grained analysis of model performance.
Towards Improving NAM-to-Speech Synthesis Intelligibility using Self-Supervised Speech Models
Moreover, we present a methodology for augmenting the existing CSTR NAM TIMIT Plus corpus, setting a benchmark with a Word Error Rate (WER) of 42. 57% to gauge the intelligibility of the synthesized speech.
Major Entity Identification: A Generalizable Alternative to Coreference Resolution
Rather than relying on this additional annotation, we propose an alternative referential task, Major Entity Identification (MEI), where we: (a) assume the target entities to be specified in the input, and (b) limit the task to only the frequent entities.
VELOCITI: Can Video-Language Models Bind Semantic Concepts through Time?
Compositionality is a fundamental aspect of vision-language understanding and is especially required for videos since they contain multiple entities (e. g. persons, actions, and scenes) interacting dynamically over time.
Pseudo-labelling meets Label Smoothing for Noisy Partial Label Learning
Noisy PLL (NPLL) relaxes this constraint by allowing some partial labels to not contain the true label, enhancing the practicality of the problem.
Real Time GAZED: Online Shot Selection and Editing of Virtual Cameras from Wide-Angle Monocular Video Recordings
Eliminating time-consuming post-production processes and delivering high-quality videos in today's fast-paced digital landscape are the key advantages of real-time approaches.
RobustL2S: Speaker-Specific Lip-to-Speech Synthesis exploiting Self-Supervised Representations
Significant progress has been made in speaker dependent Lip-to-Speech synthesis, which aims to generate speech from silent videos of talking faces.
MParrotTTS: Multilingual Multi-speaker Text to Speech Synthesis in Low Resource Setting
We present MParrotTTS, a unified multilingual, multi-speaker text-to-speech (TTS) synthesis model that can produce high-quality speech.
ParrotTTS: Text-to-Speech synthesis by exploiting self-supervised representations
We present ParrotTTS, a modularized text-to-speech synthesis model leveraging disentangled self-supervised speech representations.
Test-Time Amendment with a Coarse Classifier for Fine-Grained Classification
We investigate the problem of reducing mistake severity for fine-grained classification.
Ground then Navigate: Language-guided Navigation in Dynamic Scenes
We investigate the Vision-and-Language Navigation (VLN) problem in the context of autonomous driving in outdoor settings.
Grounding Linguistic Commands to Navigable Regions
We introduce a new dataset, Talk2Car-RegSeg, which extends the existing Talk2car dataset with segmentation masks for the regions described by the linguistic commands.
Emotional Prosody Control for Speech Generation
The proposed TTS system can generate speech from the text in any speaker's style, with fine control of emotion.
Reappraising Domain Generalization in Neural Networks
We find that the presence of multiple domains incentivizes domain agnostic learning and is the primary reason for generalization in Tradition DG.
Bringing Generalization to Deep Multi-View Pedestrian Detection
Multi-view Detection (MVD) is highly effective for occlusion reasoning in a crowded environment.
Ranked #2 on
Multiview Detection
on GMVD
High-Resolution Depth Maps Based on TOF-Stereo Fusion
Most of these systems suffer from the problems of noise in the range-data and resolution mismatch between the range sensor and the color cameras, since the resolution of current range sensors is much less than the resolution of color cameras.
Comprehensive Multi-Modal Interactions for Referring Image Segmentation
We investigate Referring Image Segmentation (RIS), which outputs a segmentation map corresponding to the natural language description.
Ranked #3 on
Referring Expression Segmentation
on ReferIt
No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks
There has been increasing interest in building deep hierarchy-aware classifiers that aim to quantify and reduce the severity of mistakes, and not just reduce the number of errors.
Amending Mistakes Post-hoc in Deep Networks by Leveraging Class Hierarchies
There has been increasing interest in building deep hierarchy-aware classifiers, aiming to quantify and reduce the severity of mistakes and not just count the number of errors.
ViNet: Pushing the limits of Visual Modality for Audio-Visual Saliency Prediction
We also explore a variation of ViNet architecture by augmenting audio features into the decoder.
GAZED- Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings
We present GAZED- eye GAZe-guided EDiting for videos captured by a solitary, static, wide-angle and high-resolution camera.
Cosine meets Softmax: A tough-to-beat baseline for visual grounding
In this paper, we present a simple baseline for visual grounding for autonomous driving which outperforms the state of the art methods, while retaining minimal design choices.
Ranked #6 on
Referring Expression Comprehension
on Talk2Car
The Curious Case of Convex Neural Networks
In this paper, we investigate a constrained formulation of neural networks where the output is a convex function of the input.
Simple Unsupervised Multi-Object Tracking
Multi-object tracking has seen a lot of progress recently, albeit with substantial annotation costs for developing better and larger labeled datasets.
LiDAR guided Small obstacle Segmentation
In this paper, we present a method to reliably detect such obstacles through a multi-modal framework of sparse LiDAR(VLP-16) and Monocular vision.
Tidying Deep Saliency Prediction Architectures
As a result, we propose two novel end-to-end architectures called SimpleNet and MDNSal, which are neater, minimal, more interpretable and achieve state of the art performance on public saliency benchmarks.
CineFilter: Unsupervised Filtering for Real Time Autonomous Camera Systems
Autonomous camera systems are often subjected to an optimization/filtering operation to smoothen and stabilize the rough trajectory estimates.
Exploring 3 R's of Long-term Tracking: Re-detection, Recovery and Reliability
Recent works have proposed several long term tracking benchmarks and highlight the importance of moving towards long-duration tracking to bridge the gap with application requirements.
Nose, eyes and ears: Head pose estimation by locating facial keypoints
Monocular head pose estimation requires learning a model that computes the intrinsic Euler angles for pose (yaw, pitch, roll) from an input image of human face.
Ranked #2 on
Head Pose Estimation
on AFLW
An Iterative Approach for Shadow Removal in Document Images
Uneven illumination and shadows in document images cause a challenge for digitization applications and automated workflows.
MergeNet: A Deep Net Architecture for Small Obstacle Discovery
We present here, a novel network architecture called MergeNet for discovering small obstacles for on-road scenes in the context of autonomous driving.
Learning Unsupervised Visual Grounding Through Semantic Self-Supervision
Localizing natural language phrases in images is a challenging problem that requires joint understanding of both the textual and visual modalities.
Long-Term Visual Object Tracking Benchmark
We propose a new long video dataset (called Track Long and Prosper - TLP) and benchmark for single object tracking.
Automated Top View Registration of Broadcast Football Videos
The proposed method is fully automatic in contrast to the current state of the art which requires manual initialization of point correspondences between the image and the static model.
The Prose Storyboard Language: A Tool for Annotating and Directing Movies
The prose storyboard language is a formal language for describing movies shot by shot, where each shot is described with a unique sentence.
Graphics
Detecting and Naming Actors in Movies Using Generative Appearance Models
We introduce a generative model for learning person and costume specific detectors from labeled examples.
