Search Results for author:
Found 33 papers, 7 papers with code
Probing Self-supervised Learning Models with Target Speech Extraction
TSE uniquely requires both speaker identification and speech separation, distinguishing it from other tasks in the Speech processing Universal PERformance Benchmark (SUPERB) evaluation.
Target Speech Extraction with Pre-trained Self-supervised Learning Models
We then extend a powerful TSE architecture by incorporating two SSL-based modules: an Adaptive Input Enhancer (AIE) and a speaker encoder.
Array Geometry-Robust Attention-Based Neural Beamformer for Moving Speakers
Recently, a mask-based beamformer with attention-based spatial covariance matrix aggregator (ASA) was proposed, which was demonstrated to track moving sources accurately.
Neural network-based virtual microphone estimation with virtual microphone and beamformer-level multi-task loss
However, this training objective may not be optimal for a specific array processing back-end, such as beamforming.
How does end-to-end speech recognition training impact speech enhancement artifacts?
Jointly training a speech enhancement (SE) front-end and an automatic speech recognition (ASR) back-end has been investigated as a way to mitigate the influence of \emph{processing distortion} generated by single-channel SE on ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data
Neural transducer (RNNT)-based target-speaker speech recognition (TS-RNNT) directly transcribes a target speaker's voice from a multi-talker mixture.
Downstream Task Agnostic Speech Enhancement with Self-Supervised Representation Loss
In this work, we propose a new SE training criterion that minimizes the distance between clean and enhanced signals in the feature representation of the SSL model to alleviate the mismatch.
Neural Target Speech Extraction: An Overview
Humans can listen to a target speaker even in challenging acoustic conditions that have noise, reverberation, and interfering speakers.
Streaming Target-Speaker ASR with Neural Transducer
We confirm in experiments that our TS-ASR achieves comparable recognition performance with conventional cascade systems in the offline setting, while reducing computation costs and realizing streaming TS-ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Analysis of impact of emotions on target speech extraction and speech separation
One of the factors causing such degradation may be intrinsic speaker variability, such as emotions, occurring commonly in realistic speech.
ConceptBeam: Concept Driven Target Speech Extraction
We use it to bridge modality-dependent information, i. e., the speech segments in the mixture, and the specified, modality-independent concept.
Strategies to Improve Robustness of Target Speech Extraction to Enrollment Variations
Experimental validation reveals the effectiveness of both worst-enrollment target training and SI-loss training to improve robustness against enrollment variations, by increasing speaker discriminability.
Mask-based Neural Beamforming for Moving Speakers with Self-Attention-based Tracking
We thus introduce a learning-based framework that computes optimal attention weights for beamforming.
Listen only to me! How well can target speech extraction handle false alarms?
Target speech extraction (TSE) extracts the speech of a target speaker in a mixture given auxiliary clues characterizing the speaker, such as an enrollment utterance.
SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learning
We can achieve this with a neural network that extracts the target SEs by conditioning it on clues representing the target SE classes.
How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR
The artifact component is defined as the SE error signal that cannot be represented as a linear combination of speech and noise sources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Learning to Enhance or Not: Neural Network-Based Switching of Enhanced and Observed Signals for Overlapping Speech Recognition
To mitigate the degradation, we introduced a rule-based method to switch the ASR input between the enhanced and observed signals, which showed promising results.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model
We modify the acoustic model to predict joint state posteriors for all speakers, enabling the network to express uncertainty about the attribution of parts of the speech signal to the speakers.
Few-shot learning of new sound classes for target sound extraction
Target sound extraction consists of extracting the sound of a target acoustic event (AE) class from a mixture of AE sounds.
PILOT: Introducing Transformers for Probabilistic Sound Event Localization
Sound event localization aims at estimating the positions of sound sources in the environment with respect to an acoustic receiver (e. g. a microphone array).
Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition
', we analyze ASR performance on observed and enhanced speech at various noise and interference conditions, and show that speech enhancement degrades ASR under some conditions even for overlapping speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Exploiting Attention-based Sequence-to-Sequence Architectures for Sound Event Localization
Herein, attentions allow for capturing temporal dependencies in the audio signal by focusing on specific frames that are relevant for estimating the activity and direction-of-arrival of sound events at the current time-step.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
End-to-End Dereverberation, Beamforming, and Speech Recognition with Improved Numerical Stability and Advanced Frontend
Recently, the end-to-end approach has been successfully applied to multi-speaker speech separation and recognition in both single-channel and multichannel conditions.
Data Fusion for Audiovisual Speaker Localization: Extending Dynamic Stream Weights to the Spatial Domain
Estimating the positions of multiple speakers can be helpful for tasks like automatic speech recognition or speaker diarization.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multimodal Attention Fusion for Target Speaker Extraction
Recently an audio-visual target speaker extraction has been proposed that extracts target speech by using complementary audio and visual clues.
Speaker activity driven neural speech extraction
Target speech extraction, which extracts the speech of a target speaker in a mixture given auxiliary speaker clues, has recently received increased interest.
Neural Network-based Virtual Microphone Estimator
Developing microphone array technologies for a small number of microphones is important due to the constraints of many devices.
Listen to What You Want: Neural Network-based Universal Sound Selector
In this paper, we propose instead a universal sound selection neural network that enables to directly select AE sounds from a mixture given user-specified target AE classes.
Improving noise robust automatic speech recognition with single-channel time-domain enhancement network
With the advent of deep learning, research on noise-robust automatic speech recognition (ASR) has progressed rapidly.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam
First, we propose a time-domain implementation of SpeakerBeam similar to that proposed for a time-domain audio separation network (TasNet), which has achieved state-of-the-art performance for speech separation.
ESPnet: End-to-End Speech Processing Toolkit
This paper introduces a new open source platform for end-to-end speech processing named ESPnet.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multichannel End-to-end Speech Recognition
The field of speech recognition is in the midst of a paradigm shift: end-to-end neural networks are challenging the dominance of hidden Markov models as a core technology.
Automatic Node Selection for Deep Neural Networks using Group Lasso Regularization
We examine the effect of the Group Lasso (gLasso) regularizer in selecting the salient nodes of Deep Neural Network (DNN) hidden layers by applying a DNN-HMM hybrid speech recognizer to TED Talks speech data.
