Search Results for author:
Found 44 papers, 14 papers with code
Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data
Neural transducer (RNNT)-based target-speaker speech recognition (TS-RNNT) directly transcribes a target speaker's voice from a multi-talker mixture.
Downstream Task Agnostic Speech Enhancement with Self-Supervised Representation Loss
In this work, we propose a new SE training criterion that minimizes the distance between clean and enhanced signals in the feature representation of the SSL model to alleviate the mismatch.
Multi-Stream Extension of Variational Bayesian HMM Clustering (MS-VBx) for Combined End-to-End and Vector Clustering-based Diarization
Combining end-to-end neural speaker diarization (EEND) with vector clustering (VC), known as EEND-VC, has gained interest for leveraging the strengths of both methods.
Leveraging Large Text Corpora for End-to-End Speech Summarization
The first technique is to utilize a text-to-speech (TTS) system to generate synthesized speech, which is used for E2E SSum training with the text summary.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Neural Target Speech Extraction: An Overview
Humans can listen to a target speaker even in challenging acoustic conditions that have noise, reverberation, and interfering speakers.
On Word Error Rate Definitions and their Efficient Computation for Multi-Speaker Speech Recognition Systems
We present a general framework to compute the word error rate (WER) of ASR systems that process recordings containing multiple speakers at their input and that produce multiple output word sequences (MIMO).
Streaming Target-Speaker ASR with Neural Transducer
We confirm in experiments that our TS-ASR achieves comparable recognition performance with conventional cascade systems in the offline setting, while reducing computation costs and realizing streaming TS-ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Analysis of impact of emotions on target speech extraction and speech separation
One of the factors causing such degradation may be intrinsic speaker variability, such as emotions, occurring commonly in realistic speech.
Utterance-by-utterance overlap-aware neural diarization with Graph-PIT
In this paper, we argue that such an approach involving the segmentation has several issues; for example, it inevitably faces a dilemma that larger segment sizes increase both the context available for enhancing the performance and the number of speakers for the local EEND module to handle.
ConceptBeam: Concept Driven Target Speech Extraction
We use it to bridge modality-dependent information, i. e., the speech segments in the mixture, and the specified, modality-independent concept.
Strategies to Improve Robustness of Target Speech Extraction to Enrollment Variations
Experimental validation reveals the effectiveness of both worst-enrollment target training and SI-loss training to improve robustness against enrollment variations, by increasing speaker discriminability.
Mask-based Neural Beamforming for Moving Speakers with Self-Attention-based Tracking
We thus introduce a learning-based framework that computes optimal attention weights for beamforming.
Listen only to me! How well can target speech extraction handle false alarms?
Target speech extraction (TSE) extracts the speech of a target speaker in a mixture given auxiliary clues characterizing the speaker, such as an enrollment utterance.
SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learning
We can achieve this with a neural network that extracts the target SEs by conditioning it on clues representing the target SE classes.
Tight integration of neural- and clustering-based diarization through deep unfolding of infinite Gaussian mixture model
Speaker diarization has been investigated extensively as an important central task for meeting analysis.
How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR
The artifact component is defined as the SE error signal that cannot be represented as a linear combination of speech and noise sources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Learning to Enhance or Not: Neural Network-Based Switching of Enhanced and Observed Signals for Overlapping Speech Recognition
To mitigate the degradation, we introduced a rule-based method to switch the ASR input between the enhanced and observed signals, which showed promising results.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Attention-based Multi-hypothesis Fusion for Speech Summarization
We propose a cascade speech summarization model that is robust to ASR errors and that exploits multiple hypotheses generated by ASR to attenuate the effect of ASR errors on the summary.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model
We modify the acoustic model to predict joint state posteriors for all speakers, enabling the network to express uncertainty about the attribution of parts of the speech signal to the speakers.
SA-SDR: A novel loss function for separation of meeting style data
Many state-of-the-art neural network-based source separation systems use the averaged Signal-to-Distortion Ratio (SDR) as a training objective function.
Speeding Up Permutation Invariant Training for Source Separation
The Hungarian algorithm can be used for uPIT and we introduce various algorithms for the Graph-PIT assignment problem to reduce the complexity to be polynomial in the number of utterances.
Graph-PIT: Generalized permutation invariant training for continuous separation of arbitrary numbers of speakers
When processing meeting-like data in a segment-wise manner, i. e., by separating overlapping segments independently and stitching adjacent segments to continuous output streams, this constraint has to be fulfilled for any segment.
Few-shot learning of new sound classes for target sound extraction
Target sound extraction consists of extracting the sound of a target acoustic event (AE) class from a mixture of AE sounds.
PILOT: Introducing Transformers for Probabilistic Sound Event Localization
Sound event localization aims at estimating the positions of sound sources in the environment with respect to an acoustic receiver (e. g. a microphone array).
Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition
', we analyze ASR performance on observed and enhanced speech at various noise and interference conditions, and show that speech enhancement degrades ASR under some conditions even for overlapping speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech
This paper is to (1) report recent advances we made to this framework, including newly introduced robust constrained clustering algorithms, and (2) experimentally show that the method can now significantly outperform competitive diarization methods such as Encoder-Decoder Attractor (EDA)-EEND, on CALLHOME data which comprises real conversational speech data including overlapped speech and an arbitrary number of speakers.
Exploiting Attention-based Sequence-to-Sequence Architectures for Sound Event Localization
Herein, attentions allow for capturing temporal dependencies in the audio signal by focusing on specific frames that are relevant for estimating the activity and direction-of-arrival of sound events at the current time-step.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Dual-Path Modeling for Long Recording Speech Separation in Meetings
A transformer-based dual-path system is proposed, which integrates transform layers for global modeling.
End-to-End Dereverberation, Beamforming, and Speech Recognition with Improved Numerical Stability and Advanced Frontend
Recently, the end-to-end approach has been successfully applied to multi-speaker speech separation and recognition in both single-channel and multichannel conditions.
Data Fusion for Audiovisual Speaker Localization: Extending Dynamic Stream Weights to the Spatial Domain
Estimating the positions of multiple speakers can be helpful for tasks like automatic speech recognition or speaker diarization.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multimodal Attention Fusion for Target Speaker Extraction
Recently an audio-visual target speaker extraction has been proposed that extracts target speech by using complementary audio and visual clues.
Speaker activity driven neural speech extraction
Target speech extraction, which extracts the speech of a target speaker in a mixture given auxiliary speaker clues, has recently received increased interest.
Neural Network-based Virtual Microphone Estimator
Developing microphone array technologies for a small number of microphones is important due to the constraints of many devices.
Continuous Speech Separation Using Speaker Inventory for Long Multi-talker Recording
Leveraging additional speaker information to facilitate speech separation has received increasing attention in recent years.
Integration of variational autoencoder and spatial clustering for adaptive multi-channel neural speech separation
In this paper, we propose a method combining variational autoencoder model of speech with a spatial clustering approach for multi-channel speech separation.
Audio and Speech Processing
Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds
In this paper, we propose a simple but effective hybrid diarization framework that works with overlapped speech and for long recordings containing an arbitrary number of speakers.
Listen to What You Want: Neural Network-based Universal Sound Selector
In this paper, we propose instead a universal sound selection neural network that enables to directly select AE sounds from a mixture given user-specified target AE classes.
Multi-talker ASR for an unknown number of sources: Joint training of source counting, separation and ASR
Most approaches to multi-talker overlapped speech separation and recognition assume that the number of simultaneously active speakers is given, but in realistic situations, it is typically unknown.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Tackling real noisy reverberant meetings with all-neural source separation, counting, and diarization system
Automatic meeting analysis is an essential fundamental technology required to let, e. g. smart devices follow and respond to our conversations.
Improving noise robust automatic speech recognition with single-channel time-domain enhancement network
With the advent of deep learning, research on noise-robust automatic speech recognition (ASR) has progressed rapidly.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention
This paper investigates a self-adaptation method for speech enhancement using auxiliary speaker-aware features; we extract a speaker representation used for adaptation directly from the test utterance.
Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam
First, we propose a time-domain implementation of SpeakerBeam similar to that proposed for a time-domain audio separation network (TasNet), which has achieved state-of-the-art performance for speech separation.
End-to-end training of time domain audio separation and recognition
The rising interest in single-channel multi-speaker speech separation sparked development of End-to-End (E2E) approaches to multi-speaker speech recognition.
All-neural online source separation, counting, and diarization for meeting analysis
While significant progress has been made on individual tasks, this paper presents for the first time an all-neural approach to simultaneous speaker counting, diarization and source separation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
