Search Results for author:
Found 12 papers, 3 papers with code
DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction
Audio-visual saliency prediction can draw support from diverse modality complements, but further performance enhancement is still challenged by customized architectures as well as task-specific loss functions.
UniST: Towards Unifying Saliency Transformer for Video Saliency Prediction and Detection
While many approaches have crafted task-specific training paradigms for either video saliency prediction or video salient object detection tasks, few attention has been devoted to devising a generalized saliency modeling framework that seamlessly bridges both these distinct tasks.
Induction Network: Audio-Visual Modality Gap-Bridging for Self-Supervised Sound Source Localization
By decoupling the gradients of visual and audio modalities, the discriminative visual representations of sound sources can be learned with the designed Induction Vector in a bootstrap manner, which also enables the audio modality to be aligned with the visual modality consistently.
FTFDNet: Learning to Detect Talking Face Video Manipulation with Tri-Modality Interaction
DeepFake based digital facial forgery is threatening public media security, especially when lip manipulation has been used in talking face generation, and the difficulty of fake video detection is further improved.
CASP-Net: Rethinking Video Saliency Prediction from an Audio-VisualConsistency Perceptual Perspective
Incorporating the audio stream enables Video Saliency Prediction (VSP) to imitate the selective attention mechanism of human brain.
CASP-Net: Rethinking Video Saliency Prediction From an Audio-Visual Consistency Perceptual Perspective
Incorporating the audio stream enables Video Saliency Prediction (VSP) to imitate the selective attention mechanism of human brain.
An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detection
DeepFake based digital facial forgery is threatening the public media security, especially when lip manipulation has been used in talking face generation, the difficulty of fake video detection is further improved.
Attention-Based Lip Audio-Visual Synthesis for Talking Face Generation in the Wild
Rather than focusing on the unimportant regions of the face image, the proposed AttnWav2Lip model is able to pay more attention on the lip region reconstruction.
Audio-visual speech separation based on joint feature representation with cross-modal attention
Multi-modal based speech separation has exhibited a specific advantage on isolating the target character in multi-talker noisy environments.
Look\&Listen: Multi-Modal Correlation Learning for Active Speaker Detection and Speech Enhancement
Active speaker detection and speech enhancement have become two increasingly attractive topics in audio-visual scenario understanding.
Unsupervised Cross-Modal Distillation for Thermal Infrared Tracking
To solve this problem, we propose to distill representations of the TIR modality from the RGB modality with Cross-Modal Distillation (CMD) on a large amount of unlabeled paired RGB-TIR data.
Push for Quantization: Deep Fisher Hashing
In this paper we push for quantization: We optimize maximum class separability in the binary space.
