Search Results for author:
Found 11 papers, 5 papers with code
OmniAudio: Generating Spatial Audio from 360-Degree Video
To generate spatial audio from 360-degree video, we propose a novel framework OmniAudio, which leverages self-supervised pre-training using both spatial audio data (in FOA format) and large-scale non-spatial data.
Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation
However, when it comes to stereo audio generation, the soundscapes often have a complex scene of multiple objects and directions.
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model
By enhancing the semantic ability of the codec, X-Codec significantly reduces WER in speech synthesis tasks and extends these benefits to non-speech applications, including music and sound generation.
Unveiling and Mitigating Bias in Audio Visual Segmentation
For audio priming bias, to enhance audio sensitivity to different intensities and semantics, a perception module specifically for audio perceives the latent semantic information and incorporates information into a limited set of queries, namely active queries.
Stepping Stones: A Progressive Training Strategy for Audio-Visual Semantic Segmentation
Audio-Visual Segmentation (AVS) aims to achieve pixel-level localization of sound sources in videos, while Audio-Visual Semantic Segmentation (AVSS), as an extension of AVS, further pursues semantic understanding of audio-visual scenes.
Ref-AVS: Refer and Segment Objects in Audio-Visual Scenes
In this work, we introduce a novel task called Reference Audio-Visual Segmentation (Ref-AVS), which seeks to segment objects within the visual domain based on expressions containing multimodal cues.
Can Textual Semantics Mitigate Sounding Object Segmentation Preference?
The Audio-Visual Segmentation (AVS) task aims to segment sounding objects in the visual space using audio cues.
FlashSpeech: Efficient Zero-Shot Speech Synthesis
The generation processes of FlashSpeech can be achieved efficiently with one or two sampling steps while maintaining high audio quality and high similarity to the audio prompt for zero-shot speech generation.
FusionINN: Decomposable Image Fusion for Brain Tumor Monitoring
Image fusion typically employs non-invertible neural networks to merge multiple source images into a single fused image.
More than Vanilla Fusion: a Simple, Decoupling-free, Attention Module for Multimodal Fusion Based on Signal Theory
The vanilla fusion methods still dominate a large percentage of mainstream audio-visual tasks.
Learning Audio-Visual embedding for Person Verification in the Wild
It has already been observed that audio-visual embedding is more robust than uni-modality embedding for person verification.
