Search Results for author:
Found 17 papers, 6 papers with code
A Probability-guided Sampler for Neural Implicit Surface Rendering
In this paper, we leverage the implicit surface representation of the foreground scene and model a probability density function in a 3D image projection space to achieve a more targeted sampling of the rays toward regions of interest, resulting in improved rendering.
Programmatic Video Prediction Using Large Language Models
The task of estimating the world model describing the dynamics of a real world process assumes immense importance for anticipating and preparing for future outcomes.
UWAV: Uncertainty-weighted Weakly-supervised Audio-Visual Video Parsing
Audio-Visual Video Parsing (AVVP) entails the challenging task of localizing both uni-modal events (i. e., those occurring exclusively in either the visual or acoustic modality of a video) and multi-modal events (i. e., those occurring in both modalities concurrently).
Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling
Extensions of Neural Radiance Fields (NeRFs) to model dynamic scenes have enabled their near photo-realistic, free-viewpoint rendering.
Ranked #1 on
Novel View Synthesis
on .
Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection
We first pretrain these factor matrices on the RGB modality, for which plenty of training data are assumed to exist and then augment only a few trainable parameters for training on the IR modality to avoid over-fitting, while encouraging them to capture complementary cues from those trained only on the RGB modality.
CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments
Audio-visual navigation of an agent towards locating an audio goal is a challenging task especially when the audio is sporadic or the environment is noisy.
Learning Audio-Visual Dynamics Using Scene Graphs for Audio Source Separation
In this paper, we propose to use this connection between audio and visual dynamics for solving two challenging tasks simultaneously, namely: (i) separating audio sources from a mixture using visual cues, and (ii) predicting the 3D visual motion of a sounding source using its separated audio.
Audio Source Separation
Visually Guided Sound Source Separation
A Hierarchical Variational Neural Uncertainty Model for Stochastic Video Prediction
Predicting the future frames of a video is a challenging task, in part due to the underlying stochastic real-world phenomena.
Visual Scene Graphs for Audio Source Separation
At its core, AVSGS uses a recursive neural network that emits mutually-orthogonal sub-graph embeddings of the visual graph using multi-head attention.
Audio Source Separation
Visually Guided Sound Source Separation
Learning to Generate Videos Using Neural Uncertainty Priors
Predicting the future frames of a video is a challenging task, in part due to the underlying stochastic real-world phenomena.
Sound2Sight: Generating Visual Dynamics from Sound and Context
To tackle this problem, we present Sound2Sight, a deep variational framework, that is trained to learn a per frame stochastic prior conditioned on a joint embedding of audio and past frames.
Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers
Given an input video, its associated audio, and a brief caption, the audio-visual scene aware dialog (AVSD) task requires an agent to indulge in a question-answer dialog with a human about the audio-visual content.
Diverse and Coherent Paragraph Generation from Images
Paragraph generation from images, which has gained popularity recently, is an important task for video summarization, editing, and support of the disabled.
Coreset-Based Neural Network Compression
We propose a novel Convolutional Neural Network (CNN) compression algorithm based on coreset representations of filters.
Deep Neural Networks with Inexact Matching for Person Re-Identification
A novel inexact matching technique then matches pixels in the first representation with those of the second.
An Active Learning Based Approach For Effective Video Annotation And Retrieval
Conventional multimedia annotation/retrieval systems such as Normalized Continuous Relevance Model (NormCRM) [16] require a fully labeled training data for a good performance.
