Search Results for author:
Found 69 papers, 29 papers with code
Aligning Speech to Languages to Enhance Code-switching Speech Recognition
Performance evaluation using large language models reveals the advantage of the linguistic hint by achieving 14. 1% and 5. 5% relative improvement on test sets of the ASRU and SEAME datasets, respectively.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Model
Recently, Denoising Diffusion Probabilistic Models (DDPMs) have attained leading performances across a diverse range of generative tasks.
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Leveraging the rich linguistic knowledge and strong reasoning abilities of LLMs, our new paradigm can integrate the rich information in N-best candidates to generate a higher-quality translation result.
ICMC-ASR: The ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition Challenge
To promote speech processing and recognition research in driving scenarios, we build on the success of the Intelligent Cockpit Speech Recognition Challenge (ICSRC) held at ISCSLP 2022 and launch the ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition (ICMC-ASR) Challenge.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Adapting OpenAI's Whisper for Speech Recognition on Code-Switch Mandarin-English SEAME and ASRU2019 Datasets
The Mixed Error Rate results show that the amount of adaptation data may be as low as $1\sim10$ hours to achieve saturation in performance gain (SEAME) while the ASRU task continued to show performance with more adaptation data ($>$100 hours).
Generative error correction for code-switching speech recognition using large language models
In this work, we propose to leverage large language models (LLMs) and lists of hypotheses generated by an ASR to address the CS problem.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models
We make our results publicly accessible for reproducible pipelines with released pre-trained models, thus providing a new evaluation paradigm for ASR error correction with LLMs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
SPGM: Prioritizing Local Features for enhanced speech separation performance
Dual-path is a popular architecture for speech separation models (e. g. Sepformer) which splits long sequences into overlapping chunks for its intra- and inter-blocks that separately model intra-chunk local features and inter-chunk global relationships.
Ranked #5 on
Speech Separation
on WSJ0-2mix
Codec Data Augmentation for Time-domain Heart Sound Classification
In this work, we propose a simple time domain approach, to the heart sound classification problem with a base classification error rate of 0. 8 and show that augmentation of the data through codec simulation can improve the classification error rate to 0. 2.
Noise-aware Speech Enhancement using Diffusion Probabilistic Model
Specifically, we design a noise classification (NC) model to produce acoustic embedding as a noise conditioner for guiding the reverse denoising process.
Hearing Lips in Noise: Universal Viseme-Phoneme Mapping and Transfer for Robust Audio-Visual Speech Recognition
In this work, we investigate the noise-invariant visual modality to strengthen robustness of AVSR, which can adapt to any testing noises while without dependence on noisy training data, a. k. a., unsupervised noise adaptation.
MIR-GAN: Refining Frame-Level Modality-Invariant Representations with Adversarial Network for Audio-Visual Speech Recognition
In this paper, we aim to learn the shared representations across modalities to bridge their gap.
A Neural State-Space Model Approach to Efficient Speech Separation
In this work, we introduce S4M, a new efficient speech separation framework based on neural state-space models (SSM).
ACA-Net: Towards Lightweight Speaker Verification using Asymmetric Cross Attention
In this paper, we propose ACA-Net, a lightweight, global context-aware speaker embedding extractor for Speaker Verification (SV) that improves upon existing work by using Asymmetric Cross Attention (ACA) to replace temporal pooling.
UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning
Multimodal learning aims to imitate human beings to acquire complementary information from multiple modalities for various downstream tasks.
Cross-Modal Global Interaction and Local Alignment for Audio-Visual Speech Recognition
However, most existing AVSR approaches simply fuse the audio and visual features by concatenation, without explicit interactions to capture the deep correlations between them, which results in sub-optimal multimodal representations for downstream speech recognition task.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+3
Leveraging Audio-Tagging Assisted Sound Event Detection using Weakified Strong Labels and Frequency Dynamic Convolutions
Stage-1 of our proposed framework focuses on audio-tagging (AT), which assists the sound event detection (SED) system in Stage-2.
Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR
Second, during finetuning we propose a Transformer-based code predictor to accurately predict clean codes by modeling the global dependency of input noisy representations, which enables discovery and restoration of high-quality clean representations without distortions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Unsupervised Noise adaptation using Data Simulation
Deep neural network based speech enhancement approaches aim to learn a noisy-to-clean transformation using a supervised learning paradigm.
Metric-oriented Speech Enhancement using Diffusion Probabilistic Model
Deep neural network based speech enhancement technique focuses on learning a noisy-to-clean transformation supervised by paired training data.
Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation
To alleviate this problem, we propose a novel network to unify speech enhancement and separation with gradient modulation to improve noise-robustness.
Gradient Remedy for Multi-Task Learning in End-to-End Noise-Robust Speech Recognition
In this paper, we propose a simple yet effective approach called gradient remedy (GR) to solve interference between task gradients in noise-robust speech recognition, from perspectives of both angle and magnitude.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Probabilistic Back-ends for Online Speaker Recognition and Clustering
This paper focuses on multi-enrollment speaker recognition which naturally occurs in the task of online speaker clustering, and studies the properties of different scoring back-ends in this scenario.
Improving Spoken Language Identification with Map-Mix
The pre-trained multi-lingual XLSR model generalizes well for language identification after fine-tuning on unseen languages.
Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning
Audio-visual speech recognition (AVSR) has gained remarkable success for ameliorating the noise-robustness of speech recognition.
Speech-text based multi-modal training with bidirectional attention for improved speech recognition
To let the state-of-the-art end-to-end ASR model enjoy data efficiency, as well as much more unpaired text data by multi-modal training, one needs to address two problems: 1) the synchronicity of feature sampling rates between speech and language (aka text data); 2) the homogeneity of the learned representations from two encoders.
Amino Acid Classification in 2D NMR Spectra via Acoustic Signal Embeddings
Nuclear Magnetic Resonance (NMR) is used in structural biology to experimentally determine the structure of proteins, which is used in many areas of biology and is an important part of drug development.
Continual Learning For On-Device Environmental Sound Classification
Experimental results on the DCASE 2019 Task 1 and ESC-50 dataset show that our proposed method outperforms baseline continual learning methods on classification accuracy and computational efficiency, indicating our method can efficiently and incrementally learn new classes without the catastrophic forgetting problem for on-device environmental sound classification.
Internal Language Model Estimation based Language Model Fusion for Cross-Domain Code-Switching Speech Recognition
More importantly, we train an end-to-end (E2E) speech recognition model by means of merging two monolingual data sets and observe the efficacy of the proposed ILME-based LM fusion for CSSR.
Intermediate-layer output Regularization for Attention-based Speech Recognition with Shared Decoder
Intermediate layer output (ILO) regularization by means of multitask training on encoder side has been shown to be an effective approach to yielding improved results on a wide range of end-to-end ASR frameworks.
Language-Based Audio Retrieval with Converging Tied Layers and Contrastive Loss
In this paper, we tackle the new Language-Based Audio Retrieval task proposed in DCASE 2022.
Self-critical Sequence Training for Automatic Speech Recognition
Although automatic speech recognition (ASR) task has gained remarkable success by sequence-to-sequence models, there are two main mismatches between its training and testing that might lead to performance degradation: 1) The typically used cross-entropy criterion aims to maximize log-likelihood of the training data, while the performance is evaluated by word error rate (WER), not log-likelihood; 2) The teacher-forcing method leads to the dependence on ground truth during training, which means that model has never been exposed to its own prediction before testing.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting
Catastrophic forgetting is a thorny challenge when updating keyword spotting (KWS) models after deployment.
Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information
In this paper, we propose an end-to-end speech emotion recognition system using multi-level acoustic information with a newly designed co-attention module.
Noise-robust Speech Recognition with 10 Minutes Unparalleled In-domain Data
Noise-robust speech recognition systems require large amounts of training data including noisy speech data and corresponding transcripts to achieve state-of-the-art performances in face of various practical environments.
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning
Automated Audio captioning (AAC) is a cross-modal task that generates natural language to describe the content of input audio.
Dual-Path Style Learning for End-to-End Noise-Robust Speech Recognition
Then, we propose style learning to map the fused feature close to clean feature, in order to learn latent speech information from the latter, i. e., clean "speech style".
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
L-SpEx: Localized Target Speaker Extraction
Speaker extraction aims to extract the target speaker's voice from a multi-talker speech mixture given an auxiliary reference utterance.
A Unified Speaker Adaptation Approach for ASR
For model adaptation, we use a novel gradual pruning method to adapt to target speakers without changing the model architecture, which to the best of our knowledge, has never been explored in ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition
Speech enhancement (SE) aims to suppress the additive noise from a noisy speech signal to improve the speech's perceptual quality and intelligibility.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Minimum word error training for non-autoregressive Transformer-based code-switching ASR
Non-autoregressive end-to-end ASR framework might be potentially appropriate for code-switching recognition task thanks to its inherent property that present output token being independent of historical ones.
Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization
In this paper, we examine the use of Transfer Learning using Pretrained Audio Neural Networks (PANNs), and propose an architecture that is able to better leverage the acoustic features provided by PANNs for the Automated Audio Captioning Task.
Multitask-Based Joint Learning Approach To Robust ASR For Radio Communication Speech
One of the advantage of the proposed method is that the entire system can be trained from scratch.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
E2E-based Multi-task Learning Approach to Joint Speech and Accent Recognition
In this paper, we propose a single multi-task learning framework to perform End-to-End (E2E) speech recognition (ASR) and accent recognition (AR) simultaneously.
End-to-End Speaker Height and age estimation using Attention Mechanism with LSTM-RNN
In this work, we propose a novel approach of using attention mechanism to build an end-to-end architecture for height and age estimation.
An Embarrassingly Simple Model for Dialogue Relation Extraction
Dialogue relation extraction (RE) is to predict the relation type of two entities mentioned in a dialogue.
Ranked #9 on
Dialog Relation Extraction
on DialogRE
GDPNet: Refining Latent Multi-View Graph for Relation Extraction
Recent advances on RE task are from BERT-based sequence modeling and graph-based modeling of relationships among the tokens in the sequence.
Ranked #4 on
Dialog Relation Extraction
on DialogRE
(F1c (v1) metric)
Multi-stage Speaker Extraction with Utterance and Frame-Level Reference Signals
Speaker extraction requires a sample speech from the target speaker as the reference.
Multilingual Approach to Joint Speech and Accent Recognition with DNN-HMM Framework
Experimental results on an 8-accent English speech recognition show both methods can yield WERs close to the conventional ASR systems that completely ignore the accent, as well as desired AR accuracy.
Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences
Domain adaptation or transfer learning using pre-trained language models such as BERT has proven to be an effective approach for many natural language processing tasks.
Leveraging Text Data Using Hybrid Transformer-LSTM Based End-to-End ASR in Transfer Learning
To this end, we extend our prior work [1], and propose a hybrid Transformer-LSTM based architecture.
Approaches to Improving Recognition of Underrepresented Named Entities in Hybrid ASR Systems
In this paper, we present a series of complementary approaches to improve the recognition of underrepresented named entities (NE) in hybrid ASR systems without compromising overall word error rate performance.
SpEx+: A Complete Time Domain Speaker Extraction Network
To eliminate such mismatch, we propose a complete time-domain speaker extraction solution, that is called SpEx+.
Ranked #1 on
Speech Extraction
on WSJ0-2mix-extr
Speech Extraction
Audio and Speech Processing
Sound
Time-domain speaker extraction network
The inaccuracy of phase estimation is inherent to the frequency domain processing, that affects the quality of signal reconstruction.
Audio and Speech Processing Sound
SpEx: Multi-Scale Time Domain Speaker Extraction Network
Inspired by Conv-TasNet, we propose a time-domain speaker extraction network (SpEx) that converts the mixture speech into multi-scale embedding coefficients instead of decomposing the speech signal into magnitude and phase spectra.
Independent language modeling architecture for end-to-end ASR
To address this problem, in this work, we propose a new architecture that separates the decoder subnet from the encoder output.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
I4U Submission to NIST SRE 2018: Leveraging from a Decade of Shared Experiences
The I4U consortium was established to facilitate a joint entry to NIST speaker recognition evaluations (SRE).
Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data
The lack of code-switch training data is one of the major concerns in the development of end-to-end code-switching automatic speech recognition (ASR) models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Enriching Rare Word Representations in Neural Language Models by Embedding Matrix Augmentation
However, learning the representation of rare words is a challenging problem causing the NLM to produce unreliable probability estimates.
Optimization of Speaker Extraction Neural Network with Magnitude and Temporal Spectrum Approximation Loss
The SpeakerBeam-FE (SBF) method is proposed for speaker extraction.
On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition
Code-switching (CS) refers to a linguistic phenomenon where a speaker uses different languages in an utterance or between alternating utterances.
Named-Entity Tagging and Domain adaptation for Better Customized Translation
Customized translation need pay spe-cial attention to the target domain ter-minology especially the named-entities for the domain.
Unsupervised and Efficient Vocabulary Expansion for Recurrent Neural Network Language Models in ASR
Additionally, we propose to generate the list of OOS words to expand vocabulary in unsupervised manner by automatically extracting them from ASR output.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition
In this paper, we present our overall efforts to improve the performance of a code-switching speech recognition system using semi-supervised training methods from lexicon learning to acoustic modeling, on the South East Asian Mandarin-English (SEAME) data.
Spoofing detection under noisy conditions: a preliminary investigation and an initial database
To simulate the real-life scenarios, we perform a preliminary investigation of spoofing detection under additive noisy conditions, and also describe an initial database for this task.
The NNI Query-by-Example System for MediaEval 2015
This paper describes the system developed by the NNI team for the Query-by-Example Search on Speech Task (QUESST) in the MediaEval 2015 evaluation.
Ranked #9 on
Keyword Spotting
on QUESST
The NNI Query-by-Example System for MediaEval 2014
For both symbolic and DTW search, partial sequence matching is performed to reduce missing rate, especially for query type 2 and 3.
Ranked #6 on
Keyword Spotting
on QUESST
