Search Results for author:
Found 19 papers, 12 papers with code
TF-Locoformer: Transformer with Local Modeling by Convolution for Speech Separation and Enhancement
This work presents TF-Locoformer, a Transformer-based model with LOcal-modeling by COnvolution.
Enhanced Reverberation as Supervision for Unsupervised Speech Separation
Separation performance is also boosted by adding a novel loss term where separated signals mapped back to their own input mixture are used as pseudo-targets for the signals separated from other channels and mapped to the same channel.
NIIRF: Neural IIR Filter Field for HRTF Upsampling and Personalization
Existing NF-based methods focused on estimating the magnitude of the HRTF from a given sound source direction, and the magnitude is converted to a finite impulse response (FIR) filter.
NeuroHeed+: Improving Neuro-steered Speaker Extraction with Joint Auditory Attention Detection
Neuro-steered speaker extraction aims to extract the listener's brain-attended speech signal from a multi-talker speech signal, in which the attention is derived from the cortical activity.
Scenario-Aware Audio-Visual TF-GridNet for Target Speech Extraction
Target speech extraction aims to extract, based on a given conditioning cue, a target speech signal that is corrupted by interfering sources, such as noise or competing speakers.
Generation or Replication: Auscultating Audio Latent Diffusion Models
The introduction of audio latent diffusion models possessing the ability to generate realistic sound clips on demand from a text description has the potential to revolutionize how we work with audio.
LocSelect: Target Speaker Localization with an Auditory Selective Hearing Mechanism
The prevailing noise-resistant and reverberation-resistant localization algorithms primarily emphasize separating and providing directional output for each speaker in multi-speaker scenarios, without association with the identity of speakers.
NeuroHeed: Neuro-Steered Speaker Extraction using EEG Signals
We propose both an offline and an online NeuroHeed, with the latter designed for real-time inference.
Target Active Speaker Detection with Audio-visual Cues
To benefit from both facial cue and reference speech, we propose the Target Speaker TalkNet (TS-TalkNet), which leverages a pre-enrolled speaker embedding to complement the audio-visual synchronization cue in detecting whether the target speaker is speaking.
Late Audio-Visual Fusion for In-The-Wild Speaker Diarization
Speaker diarization is well studied for constrained audios but little explored for challenging in-the-wild videos, which have more speakers, shorter utterances, and inconsistent on-screen speakers.
ImagineNET: Target Speaker Extraction with Intermittent Visual Cue through Embedding Inpainting
In this paper, we study the audio-visual speaker extraction algorithms with intermittent visual cue.
VCSE: Time-Domain Visual-Contextual Speaker Extraction Network
In the first stage, we pre-extract a target speech with visual cues and estimate the underlying phonetic sequence.
A Hybrid Continuity Loss to Reduce Over-Suppression for Time-domain Target Speaker Extraction
We propose a hybrid continuity loss function for time-domain speaker extraction algorithms to settle the over-suppression problem.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speaker Extraction with Co-Speech Gestures Cue
Speaker extraction seeks to extract the clean speech of a target speaker from a multi-talker mixture speech.
USEV: Universal Speaker Extraction with Visual Cue
The speaker extraction algorithm requires an auxiliary reference, such as a video recording or a pre-recorded speech, to form top-down auditory attention on the target speaker.
Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers.
Active Speaker Detection
Audio-Visual Active Speaker Detection
Selective Listening by Synchronizing Speech with Lips
A speaker extraction algorithm seeks to extract the speech of a target speaker from a multi-talker speech mixture when given a cue that represents the target speaker, such as a pre-enrolled speech utterance, or an accompanying video track.
NUS-HLT Report for ActivityNet Challenge 2021 AVA (Speaker)
Active speaker detection (ASD) seeks to detect who is speaking in a visual scene of one or more speakers.
Active Speaker Detection
Audio-Visual Active Speaker Detection
Muse: Multi-modal target speaker extraction with visual cues
Speaker extraction algorithm relies on the speech sample from the target speaker as the reference point to focus its attention.
