Search Results for author:
Found 24 papers, 6 papers with code
Multimodal Diffusion Bridge with Attention-Based SAR Fusion for Satellite Image Cloud Removal
In addition, we propose a novel multimodal diffusion bridge architecture with a two-branch backbone for multimodal image restoration, incorporating an efficient backbone and dedicated cross-modality fusion blocks to effectively extract and fuse features from synthetic aperture radar (SAR) and optical images.
Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography
In this paper, we present a modular, interpretable pipeline for pulse signal estimation from video of the face that achieves state-of-the-art results on publicly available datasets. Our imaging photoplethysmography (iPPG) system consists of three modules: face and landmark detection, time-series extraction, and pulse signal/pulse rate estimation.
Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Models
Camera-based monitoring of vital signs, also known as imaging photoplethysmography (iPPG), has seen applications in driver-monitoring, perfusion assessment in surgical settings, affective computing, and more.
TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models
To guide video generation with the additional image input, we propose a "repeat-and-slide" strategy that modulates the reverse denoising process, allowing the frozen diffusion model to synthesize a video frame-by-frame starting from the provided image.
RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation
We leverage Large Language Models (LLM) for zeroshot Semantic Audio Visual Navigation (SAVN).
Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis
To this end, and capitalizing on the powerful fine-grained generative control offered by the recent diffusion-based generative models, we introduce Steered Diffusion, a generalized framework for photorealistic zero-shot conditional image generation using a diffusion model trained for unconditional generation.
H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactions
The world is filled with articulated objects that are difficult to determine how to use from vision alone, e. g., a door might open inwards or outwards.
(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering
Spatio-temporal scene-graph approaches to video-based reasoning tasks, such as video question-answering (QA), typically construct such graphs for every video frame.
Ranked #43 on
Video Question Answering
on NExT-QA
MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation
Recent advances in generative adversarial networks (GANs) have led to remarkable achievements in face image synthesis.
Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning
In previous work, we have proposed the Audio-Visual Scene-Aware Dialog (AVSD) task, collected an AVSD dataset, developed AVSD technologies, and hosted an AVSD challenge track at both the 7th and 8th Dialog System Technology Challenges (DSTC7, DSTC8).
InSeGAN: A Generative Approach to Segmenting Identical Instances in Depth Images
To use our model for instance segmentation, we propose an instance pose encoder that learns to take in a generated depth image and reproduce the pose code vectors for all of the object instances.
LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood
In this paper, we present a novel framework for jointly predicting landmark locations, associated uncertainties of these predicted locations, and landmark visibilities.
Ranked #1 on
Face Alignment
on Menpo
Spatio-Temporal Ranked-Attention Networks for Video Captioning
To this end, we propose a Spatio-Temporal and Temporo-Spatial (STaTS) attention model which, conditioned on the language state, hierarchically combines spatial and temporal attention to videos in two different orders: (i) a spatio-temporal (ST) sub-model, which first attends to regions that have temporal evolution, then temporally pools the features from these regions; and (ii) a temporo-spatial (TS) sub-model, which first decides a single frame to attend to, then applies spatial attention within that frame.
The Eighth Dialog System Technology Challenge
This paper introduces the Eighth Dialog System Technology Challenge.
Dialog System Technology Challenge 7
This paper introduces the Seventh Dialog System Technology Challenges (DSTC), which use shared datasets to explore the problem of building dialog systems.
End-to-End Audio Visual Scene-Aware Dialog using Multimodal Attention-Based Video Features
We introduce a new dataset of dialogs about videos of human behaviors.
Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7
Scene-aware dialog systems will be able to have conversations with users about the objects and events around them.
Class Subset Selection for Transfer Learning using Submodularity
To accomplish this, we use a submodular set function to model the accuracy achievable on a new task when the features have been learned on a given subset of classes of the source dataset.
A Multi-Stream Bi-Directional Recurrent Neural Network for Fine-Grained Action Detection
We present a multi-stream bi-directional recurrent neural network for fine-grained action detection.
Action Recognition In Videos
Fine-Grained Action Detection
+2
Robust Face Alignment Using a Mixture of Invariant Experts
Face alignment is particularly challenging when there are large variations in pose (in-plane and out-of-plane rotations) and facial expression.
An Improved Deep Learning Architecture for Person Re-Identification
Novel elements of our architecture include a layer that computes cross-input neighborhood differences, which capture local relationships among mid-level features that were computed separately from the two input images.
Real-Time 3D Head Pose and Facial Landmark Estimation From Depth Images Using Triangular Surface Patch Features
The matched triangular surface patches in the training set are used to compute estimates of the 3D head pose and facial landmark positions in the input depth map.
