Search Results for author:
Found 145 papers, 35 papers with code
Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech
In this paper, we propose multi-band MelGAN, a much faster waveform generation model targeting to high-quality text-to-speech.
Sound Audio and Speech Processing
Wake Word Detection with Alignment-Free Lattice-Free MMI
Always-on spoken language interfaces, e. g. personal digital assistants, rely on a wake word to start processing spoken input.
Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition
In this paper, we present a novel two-pass approach to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model.
Ranked #6 on
Speech Recognition
on AISHELL-1
WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit
In this paper, we propose an open source, production first, and production ready speech recognition toolkit called WeNet in which a new two-pass approach is implemented to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model.
The NPU System for the 2020 Personalized Voice Trigger Challenge
This paper describes the system developed by the NPU team for the 2020 personalized voice trigger challenge.
WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit
Recently, we made available WeNet, a production-oriented end-to-end speech recognition toolkit, which introduces a unified two-pass (U2) framework and a built-in runtime to address the streaming and non-streaming decoding modes in a single model.
Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition
Recently, streaming end-to-end automatic speech recognition (E2E-ASR) has gained more and more attention.
Sound Audio and Speech Processing
DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
Speech enhancement has benefited from the success of deep learning in terms of intelligibility and perceptual quality.
Ranked #4 on
Speech Enhancement
on Deep Noise Suppression (DNS) Challenge
(PESQ-NB metric)
Speech Enhancement
Audio and Speech Processing
Sound
The NPU-Elevoc Personalized Speech Enhancement System for ICASSP2023 DNS Challenge
This paper describes our NPU-Elevoc personalized speech enhancement system (NAPSE) for the 5th Deep Noise Suppression Challenge at ICASSP 2023.
Espresso: A Fast End-to-end Neural Speech Recognition Toolkit
We present Espresso, an open-source, modular, extensible end-to-end neural automatic speech recognition (ASR) toolkit based on the deep learning library PyTorch and the popular neural machine translation toolkit fairseq.
Ranked #1 on
Speech Recognition
on Hub5'00 CallHome
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Statistical Parametric Speech Synthesis Using Generative Adversarial Networks Under A Multi-task Learning Framework
In this paper, we aim at improving the performance of synthesized speech in statistical parametric speech synthesis (SPSS) based on a generative adversarial network (GAN).
Sound
WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition
In this paper, we present WenetSpeech, a multi-domain Mandarin corpus consisting of 10000+ hours high-quality labeled speech, 2400+ hours weakly labeled speech, and about 10000 hours unlabeled speech, with 22400+ hours in total.
Ranked #5 on
Speech Recognition
on WenetSpeech
WeKws: A production first small-footprint end-to-end Keyword Spotting Toolkit
Keyword spotting (KWS) enables speech-based user interaction and gradually becomes an indispensable component of smart devices.
Phonetic Posteriorgrams based Many-to-Many Singing Voice Conversion via Adversarial Training
In order to make timbre conversion more stable and controllable, speaker embedding is further decomposed to the weighted sum of a group of trainable vectors representing different timbre clusters.
Uformer: A Unet based dilated complex & real dual-path conformer network for simultaneous speech enhancement and dereverberation
Complex spectrum and magnitude are considered as two major features of speech enhancement and dereverberation.
Channel-wise Subband Input for Better Voice and Accompaniment Separation on High Resolution Music
We aim to address the major issues in CNN-based high-resolution MSS model: high computational cost and weight sharing between distinctly different bands.
Audio and Speech Processing Sound
INTERSPEECH 2021 ConferencingSpeech Challenge: Towards Far-field Multi-Channel Speech Enhancement for Video Conferencing
The ConferencingSpeech 2021 challenge is proposed to stimulate research on far-field multi-channel speech enhancement for video conferencing.
TFGAN: Time and Frequency Domain Based Generative Adversarial Network for High-fidelity Speech Synthesis
On one hand, we propose to discriminate ground-truth waveform from synthetic one in frequency domain for offering more consistency guarantees instead of only in time domain.
DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection
Reconstruction-based approaches have achieved remarkable outcomes in anomaly detection.
Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures
The prediction of physicochemical properties from molecular structures is a crucial task for artificial intelligence aided molecular design.
Ranked #2 on
Drug Discovery
on QM9
Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition
First, we study the effectiveness of different dereverberation networks (the generator in GAN) and find that LSTM leads a significant improvement as compared with feed-forward DNN and CNN in our dataset.
AIPerf: Automated machine learning as an AI-HPC benchmark
The de facto HPC benchmark LINPACK can not reflect AI computing power and I/O performance without representative workload.
Efficient and Accurate Physics-aware Multiplex Graph Neural Networks for 3D Small Molecules and Macromolecule Complexes
On small molecule dataset for predicting quantum chemical properties, PaxNet reduces the prediction error by 15% and uses 73% less memory than the best baseline.
Molecular Property Prediction
Protein-Ligand Affinity Prediction
Physics-aware Graph Neural Network for Accurate RNA 3D Structure Prediction
In this work, we propose a Graph Neural Network (GNN)-based scoring function trained only with the atomic types and coordinates on limited solved RNA 3D structures for distinguishing accurate structural models.
A Universal Framework for Accurate and Efficient Geometric Deep Learning of Molecular Systems
Molecular sciences address a wide range of problems involving molecules of different types and sizes and their complexes.
Ranked #1 on
Drug Discovery
on QM9
Attention-based End-to-End Models for Small-Footprint Keyword Spotting
In this paper, we propose an attention-based end-to-end neural approach for small-footprint keyword spotting (KWS), which aims to simplify the pipelines of building a production-quality KWS system.
IEEE SLT 2021 Alpha-mini Speech Challenge: Open Datasets, Tracks, Rules and Baselines
In this challenge, we open source a sizable speech, keyword, echo and noise corpus for promoting data-driven methods, particularly deep-learning approaches on KWS and SSL.
Sound Audio and Speech Processing
The NPU-ASLP-LiAuto System Description for Visual Speech Recognition in CNVSRC 2023
This paper delineates the visual speech recognition (VSR) system introduced by the NPU-ASLP-LiAuto (Team 237) in the first Chinese Continuous Visual Speech Recognition Challenge (CNVSRC) 2023, engaging in the fixed and open tracks of Single-Speaker VSR Task, and the open track of Multi-Speaker VSR Task.
Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation
To improve the performance of attention-based GNNs, we propose cardinality preserved attention (CPA) models that can be applied to any kind of attention mechanisms.
Ranked #2 on
Graph Classification
on RE-M5K
Look\&Listen: Multi-Modal Correlation Learning for Active Speaker Detection and Speech Enhancement
Active speaker detection and speech enhancement have become two increasingly attractive topics in audio-visual scenario understanding.
Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model
Speaker adaptation aims to estimate a speaker specific acoustic model from a speaker independent one to minimize the mismatch between the training and testing conditions arisen from speaker variabilities.
CODE-AE: A Coherent De-confounding Autoencoder for Predicting Patient-Specific Drug Response From Cell Line Transcriptomics
Thus, CODE-AE provides a useful framework to take advantage of in vitro omics data for developing generalized patient predictive models.
The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines
In the metric aspect, we design the new conversational DER (CDER) evaluation metric, which calculates the SD accuracy at the utterance level.
Auto-KWS 2021 Challenge: Task, Datasets, and Baselines
Auto-KWS 2021 challenge calls for automated machine learning (AutoML) solutions to automate the process of applying machine learning to a customized keyword spotting task.
Multi-Speaker ASR Combining Non-Autoregressive Conformer CTC and Conditional Speaker Chain
Moreover, by including the data of variable numbers of speakers, our model can even better than the PIT-Conformer AR model with only 1/7 latency, obtaining WERs of 19. 9% and 34. 3% on WSJ0-2mix and WSJ0-3mix sets.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Search
We also find that it is important to have sufficient speech segment pairs to train the deep CNN for effective acoustic word embeddings.
Training Augmentation with Adversarial Examples for Robust Speech Recognition
This paper explores the use of adversarial examples in training speech recognition systems to increase robustness of deep neural network acoustic models.
Study of Semi-supervised Approaches to Improving English-Mandarin Code-Switching Speech Recognition
In this paper, we present our overall efforts to improve the performance of a code-switching speech recognition system using semi-supervised training methods from lexicon learning to acoustic modeling, on the South East Asian Mandarin-English (SEAME) data.
Domain Adversarial Training for Accented Speech Recognition
In this paper, we propose a domain adversarial training (DAT) algorithm to alleviate the accented speech recognition problem.
Attention-Based End-to-End Speech Recognition on Voice Search
Previous attempts have shown that applying attention-based encoder-decoder to Mandarin speech recognition was quite difficult due to the logographic orthography of Mandarin, the large vocabulary and the conditional dependency of the attention model.
Empirical Evaluation of Parallel Training Algorithms on Acoustic Modeling
Deep learning models (DLMs) are state-of-the-art techniques in speech recognition.
Automatic Prosody Prediction for Chinese Speech Synthesis using BLSTM-RNN and Embedding Features
Prosody affects the naturalness and intelligibility of speech.
A Waveform Representation Framework for High-quality Statistical Parametric Speech Synthesis
State-of-the-art statistical parametric speech synthesis (SPSS) generally uses a vocoder to represent speech signals and parameterize them into features for subsequent modeling.
Bi-objective Optimization for Robust RGB-D Visual Odometry
This paper considers a new bi-objective optimization formulation for robust RGB-D visual odometry.
Exploring RNN-Transducer for Chinese Speech Recognition
End-to-end approaches have drawn much attention recently for significantly simplifying the construction of an automatic speech recognition (ASR) system.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A New GAN-based End-to-End TTS Training Algorithm
However, the autoregressive module training is affected by the exposure bias, or the mismatch between the different distributions of real and predicted data.
Exploiting Syntactic Features in a Parsed Tree to Improve End-to-End TTS
The end-to-end TTS, which can predict speech directly from a given sequence of graphemes or phonemes, has shown improved performance over the conventional TTS.
Building a mixed-lingual neural TTS system with only monolingual data
When deploying a Chinese neural text-to-speech (TTS) synthesis system, one of the challenges is to synthesize Chinese utterances with English phrases or words embedded.
a novel cross-lingual voice cloning approach with a few text-free samples
In this paper, we present a cross-lingual voice cloning approach.
Time Domain Audio Visual Speech Separation
Audio-visual multi-modal modeling has been demonstrated to be effective in many speech related tasks, such as speech recognition and speech enhancement.
Audio and Speech Processing Sound
Adversarial Feature Learning and Unsupervised Clustering based Speech Synthesis for Found Data with Acoustic and Textual Noise
As for the speech-side noise, we propose to learn a noise-independent feature in the auto-regressive decoder through adversarial training and data augmentation, which does not need an extra speech enhancement model.
Simplified Self-Attention for Transformer-based End-to-End Speech Recognition
Transformer models have been introduced into end-to-end speech recognition with state-of-the-art performance on various tasks owing to their superiority in modeling long-term dependencies.
Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals
This model additionally has a simple and efficient stop criterion for the end of the transduction, making it able to infer the variable number of output sequences.
Ranked #3 on
Speech Separation
on WSJ0-4mix
The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Results
The paper then presents an overview of the results and system performance in the three tracks.
An End-to-end Architecture of Online Multi-channel Speech Separation
Previously, we introduced a sys-tem, calledunmixing, fixed-beamformerandextraction(UFE), that was shown to be effective in addressing the speech over-lap problem in conversation transcription.
A Cross-Level Information Transmission Network for Predicting Phenotype from New Genotype: Application to Cancer Precision Medicine
An unsolved fundamental problem in biology and ecology is to predict observable traits (phenotypes) from a new genetic constitution (genotype) of an organism under environmental perturbations (e. g., drug treatment).
AutoSpeech 2020: The Second Automated Machine Learning Challenge for Speech Classification
The AutoSpeech challenge calls for automated machine learning (AutoML) solutions to automate the process of applying machine learning to speech processing tasks.
The SLT 2021 children speech recognition challenge: Open datasets, rules and baselines
Automatic speech recognition (ASR) has been significantly advanced with the use of deep learning and big data.
Sound Audio and Speech Processing
Cascade RNN-Transducer: Syllable Based Streaming On-device Mandarin Speech Recognition with a Syllable-to-Character Converter
End-to-end models are favored in automatic speech recognition (ASR) because of its simplified system structure and superior performance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Wake Word Detection with Streaming Transformers
Modern wake word detection systems usually rely on neural networks for acoustic modeling.
Regularized Covariance Estimation for Polarization Radar Detection in Compound Gaussian Sea Clutter
This paper investigates regularized estimation of Kronecker-structured covariance matrices (CM) for polarization radar in sea clutter scenarios where the data are assumed to follow the complex, elliptically symmetric (CES) distributions with a Kronecker-structured CM.
The Multi-speaker Multi-style Voice Cloning Challenge 2021
The challenge consists of two tracks, namely few-shot track and one-shot track, where the participants are required to clone multiple target voices with 100 and 5 samples respectively.
Cross-Validated Tuning of Shrinkage Factors for MVDR Beamforming Based on Regularized Covariance Matrix Estimation
We propose leave-one-out cross-validation (LOOCV) choices for the shrinkage factors to optimize the beamforming performance, referred to as $\text{S}^2$CM-CV and STE-CV.
The NNI Query-by-Example System for MediaEval 2014
For both symbolic and DTW search, partial sequence matching is performed to reduce missing rate, especially for query type 2 and 3.
Ranked #6 on
Keyword Spotting
on QUESST
Enriching Source Style Transfer in Recognition-Synthesis based Non-Parallel Voice Conversion
Specifically, prosodic features are used to explicit model prosody, while VAE and reference encoder are used to implicitly model prosody, which take Mel spectrum and bottleneck feature as input respectively.
DCCRN+: Channel-wise Subband DCCRN with SNR Estimation for Speech Enhancement
Deep complex convolution recurrent network (DCCRN), which extends CRN with complex structure, has achieved superior performance in MOS evaluation in Interspeech 2020 deep noise suppression challenge (DNS2020).
The NNI Query-by-Example System for MediaEval 2015
This paper describes the system developed by the NNI team for the Query-by-Example Search on Speech Task (QUESST) in the MediaEval 2015 evaluation.
Ranked #9 on
Keyword Spotting
on QUESST
Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder For High Fidelity Flow-based Speech Synthesis
Current two-stage TTS framework typically integrates an acoustic model with a vocoder -- the acoustic model predicts a low resolution intermediate representation such as Mel-spectrum while the vocoder generates waveform from the intermediate representation.
Controllable Context-aware Conversational Speech Synthesis
Specifically, we use explicit labels to represent two typical spontaneous behaviors filled-pause and prolongation in the acoustic model and develop a neural network based predictor to predict the occurrences of the two behaviors from text.
AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person
In this paper, we present an automatic method to generate synchronized speech and talking-head videos on the basis of text and a single face image of an arbitrary person as input.
VISinger: Variational Inference with Adversarial Learning for End-to-End Singing Voice Synthesis
In this paper, we propose VISinger, a complete end-to-end high-quality singing voice synthesis (SVS) system that directly generates audio waveform from lyrics and musical score.
One-shot Voice Conversion For Style Transfer Based On Speaker Adaptation
One-shot style transfer is a challenging task, since training on one utterance makes model extremely easy to over-fit to training data and causes low speaker similarity and lack of expressiveness.
Exploration of Dark Chemical Genomics Space via Portal Learning: Applied to Targeting the Undruggable Genome and COVID-19 Anti-Infective Polypharmacology
Advances in biomedicine are largely fueled by exploring uncharted territories of human biology.
A Computational Efficient Maximum Likelihood Direct Position Determination Approach for Multiple Emitters Using Angle and Doppler Measurements
Emitter localization is widely applied in the military and civilian _elds.
Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios
Moreover, the explicit prosody features used in the prosody predicting module can increase the diversity of synthetic speech by adjusting the value of prosody features.
IQDUBBING: Prosody modeling based on discrete self-supervised speech representation for expressive voice conversion
Specifically, prosody vector is first extracted from pre-trained VQ-Wav2Vec model, where rich prosody information is embedded while most speaker and environment information are removed effectively by quantization.
Reinforcement Learning for Personalized Drug Discovery and Design for Complex Diseases: A Systems Pharmacology Perspective
The challenges on harnessing reinforcement learning for systems pharmacology and personalized medicine are discussed.
Conversational Speech Recognition By Learning Conversation-level Characteristics
Conversational automatic speech recognition (ASR) is a task to recognize conversational speech including multiple speakers.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Audio-visual speech separation based on joint feature representation with cross-modal attention
Multi-modal based speech separation has exhibited a specific advantage on isolating the target character in multi-talker noisy environments.
Attention-Based Lip Audio-Visual Synthesis for Talking Face Generation in the Wild
Rather than focusing on the unimportant regions of the face image, the proposed AttnWav2Lip model is able to pay more attention on the lip region reconstruction.
An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detection
DeepFake based digital facial forgery is threatening the public media security, especially when lip manipulation has been used in talking face generation, the difficulty of fake video detection is further improved.
Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Dataset
As a Mandarin speech dataset designed for dialog scenarios with high quality and rich annotations, MagicData-RAMC enriches the data diversity in the Mandarin speech community and allows extensive research on a series of speech-related tasks, including automatic speech recognition, speaker diarization, topic detection, keyword search, text-to-speech, etc.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings
Therefore, we propose the second approach, WD-SOT, to address alignment errors by introducing a word-level diarization model, which can get rid of such timestamp alignment dependency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
End-to-End Voice Conversion with Information Perturbation
The ideal goal of voice conversion is to convert the source speaker's speech to sound naturally like the target speaker while maintaining the linguistic content and the prosody of the source speech.
Leveraging Acoustic Contextual Representation by Audio-textual Cross-modal Learning for Conversational ASR
Then, during the training of the conversational ASR system, the extractor will be frozen to extract the textual representation of preceding speech, while such representation is used as context fed to the ASR decoder through attention mechanism.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Intelligent Reflecting Surface-Aided Maneuvering Target Sensing: True Velocity Estimation
Maneuvering target sensing will be an important service of future vehicular networks, where precise velocity estimation is one of the core tasks.
OLLIE: Derivation-based Tensor Program Optimizer
Boosting the runtime performance of deep neural networks (DNNs) is critical due to their wide adoption in real-world tasks.
Iterative Sparse Recovery based Passive Localization in Perceptive Mobile Networks
As a result, most existing methods require a large number of data samples to achieve an accurate estimate of the covariance matrix for the received signals, based on which a power spectrum is constructed for localization purposes.
ParaTTS: Learning Linguistic and Prosodic Cross-sentence Information in Paragraph-based TTS
To alleviate the difficulty in training, we propose to model linguistic and prosodic information by considering cross-sentence, embedded structure in training.
NWPU-ASLP System for the VoicePrivacy 2022 Challenge
Our system consists of four modules, including feature extractor, acoustic model, anonymization module, and neural vocoder.
spatial-dccrn: dccrn equipped with frame-level angle feature and hybrid filtering for multi-channel speech enhancement
Recently, multi-channel speech enhancement has drawn much interest due to the use of spatial information to distinguish target speech from interfering signal.
TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge
In this challenge, we explore three kinds of typical speaker diarization systems, which are spectral clustering(SC) based diarization, target-speaker voice activity detection(TS-VAD) and end-to-end neural diarization(EEND) respectively.
Distinguishable Speaker Anonymization based on Formant and Fundamental Frequency Scaling
By directly scaling the formant and F0, the speaker distinguishability degradation of the anonymized speech caused by the introduction of other speakers is prevented.
Preserving background sound in noise-robust voice conversion via multi-task learning
Background sound is an informative form of art that is helpful in providing a more immersive experience in real-application voice conversion (VC) scenarios.
Expressive-VC: Highly Expressive Voice Conversion with Attention Fusion of Bottleneck and Perturbation Features
We further fuse the linguistic and para-linguistic features through an attention mechanism, where speaker-dependent prosody features are adopted as the attention query, which result from a prosody encoder with target speaker embedding and normalized pitch and energy of source speech as input.
Delivering Speaking Style in Low-resource Voice Conversion with Multi-factor Constraints
Conveying the linguistic content and maintaining the source speech's speaking style, such as intonation and emotion, is essential in voice conversion (VC).
Multi-Speaker Expressive Speech Synthesis via Multiple Factors Decoupling
This paper aims to synthesize the target speaker's speech with desired speaking style and emotion by transferring the style and emotion from reference speech recorded by other speakers.
MSV Challenge 2022: NPU-HC Speaker Verification System for Low-resource Indian Languages
Specifically, the weight transfer fine-tuning aims to constrain the distance of the weights between the pre-trained model and the fine-tuned model, which takes advantage of the previously acquired discriminative ability from the large-scale out-domain datasets and avoids catastrophic forgetting and overfitting at the same time.
A Tribute to Phil Bourne -- Scientist and Human
This Special Issue of Biomolecules, commissioned in honor of Dr. Philip E. Bourne, focuses on a new field of biomolecular data science.
Two Stage Contextual Word Filtering for Context bias in Unified Streaming and Non-streaming Transducer
In this paper, we propose an efficient approach to obtain a high quality contextual list for a unified streaming/non-streaming based E2E model.
Two-step Band-split Neural Network Approach for Full-band Residual Echo Suppression
This paper describes a Two-step Band-split Neural Network (TBNN) approach for full-band acoustic echo cancellation.
Two-stage Neural Network for ICASSP 2023 Speech Signal Improvement Challenge
Achieving 0. 446 in the final score and 0. 517 in the P. 835 score, our system ranks 4th in the non-real-time track.
Joint BS Selection, User Association, and Beamforming Design for Network Integrated Sensing and Communication
Different from conventional radar, the cellular network in the integrated sensing and communication (ISAC) system enables collaborative sensing by multiple sensing nodes, e. g., base stations (BSs).
Multi-level Temporal-channel Speaker Retrieval for Zero-shot Voice Conversion
Specifically, to flexibly adapt to the dynamic-variant speaker characteristic in the temporal and channel axis of the speech, we propose a novel fine-grained speaker modeling method, called temporal-channel retrieval (TCR), to find out when and where speaker information appears in speech.
Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network
In this study, we introduce a contextual phrase prediction network for an attention-based deep bias method.
DualVC: Dual-mode Voice Conversion using Intra-model Knowledge Distillation and Hybrid Predictive Coding
Voice conversion is an increasingly popular technology, and the growing number of real-time applications requires models with streaming conversion capabilities.
DCCRN-KWS: an audio bias based model for noise robust small-footprint keyword spotting
Real-world complex acoustic environments especially the ones with a low signal-to-noise ratio (SNR) will bring tremendous challenges to a keyword spotting (KWS) system.
BA-SOT: Boundary-Aware Serialized Output Training for Multi-Talker ASR
The recently proposed serialized output training (SOT) simplifies multi-talker automatic speech recognition (ASR) by generating speaker transcriptions separated by a special token.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
TranUSR: Phoneme-to-word Transcoder Based Unified Speech Representation Learning for Cross-lingual Speech Recognition
Different from UniSpeech, UniData2vec replaces the quantized discrete representations with continuous and contextual representations from a teacher model for phonetically-aware pre-training.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Adaptive Contextual Biasing for Transducer Based Streaming Speech Recognition
By incorporating additional contextual information, deep biasing methods have emerged as a promising solution for speech recognition of personalized words.
LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Models
An intuitive approach is to follow AudioLM - Tokenizing speech into semantic and acoustic tokens respectively by HuBERT and SoundStream, and converting source semantic tokens to target acoustic tokens conditioned on acoustic tokens of the target speaker.
MSW-Transformer: Multi-Scale Shifted Windows Transformer Networks for 12-Lead ECG Classification
To address these challenges, we propose a single-layer Transformer network called Multi-Scale Shifted Windows Transformer Networks (MSW-Transformer), which uses a multi-window sliding attention mechanism at different scales to capture features in different dimensions.
Bundle-specific Tractogram Distribution Estimation Using Higher-order Streamline Differential Equation
At the global level, the tractography process is simplified as the estimation of bundle-specific tractogram distribution (BTD) coefficients by minimizing the energy optimization model, and is used to characterize the relations between BTD and diffusion tensor vector under the prior guidance by introducing the tractogram bundle information to provide anatomic priors.
Learning Universal and Robust 3D Molecular Representations with Graph Convolutional Networks
To combine the DNP descriptor and chemical features in molecules, we construct the Robust Molecular Graph Convolutional Network (RoM-GCN) which is capable to take both node and edge features into consideration when generating molecule representations.
METTS: Multilingual Emotional Text-to-Speech by Cross-speaker and Cross-lingual Emotion Transfer
However, such data-efficient approaches have ignored synthesizing emotional aspects of speech due to the challenges of cross-speaker cross-lingual emotion transfer - the heavy entanglement of speaker timbre, emotion, and language factors in the speech signal will make a system produce cross-lingual synthetic speech with an undesired foreign accent and weak emotion expressiveness.
Surrogate Empowered Sim2Real Transfer of Deep Reinforcement Learning for ORC Superheat Control
The Organic Rankine Cycle (ORC) is widely used in industrial waste heat recovery due to its simple structure and easy maintenance.
MSM-VC: High-fidelity Source Style Transfer for Non-Parallel Voice Conversion by Multi-scale Style Modeling
In addition to conveying the linguistic content from source speech to converted speech, maintaining the speaking style of source speech also plays an important role in the voice conversion (VC) task, which is essential in many scenarios with highly expressive source speech, such as dubbing and data augmentation.
PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Prompts
In this study, we propose PromptVC, a novel style voice conversion approach that employs a latent diffusion model to generate a style vector driven by natural language prompts.
DualVC 2: Dynamic Masked Convolution for Unified Streaming and Non-Streaming Voice Conversion
Third, the model is unable to effectively address the noise in the unvoiced segments, lowering the sound quality.
SSHR: Leveraging Self-supervised Hierarchical Representations for Multilingual Automatic Speech Recognition
We first analyze the different layers of the SSL model for language-related and content-related information, uncovering layers that show a stronger correlation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
VITS-Based Singing Voice Conversion Leveraging Whisper and multi-scale F0 Modeling
This paper introduces the T23 team's system submitted to the Singing Voice Conversion Challenge 2023.
MBTFNet: Multi-Band Temporal-Frequency Neural Network For Singing Voice Enhancement
A typical neural speech enhancement (SE) approach mainly handles speech and noise mixtures, which is not optimal for singing voice enhancement scenarios.
An Exploration of Task-decoupling on Two-stage Neural Post Filter for Real-time Personalized Acoustic Echo Cancellation
Utilization of speaker representation has extended the frontier of AEC, thus attracting many researchers' interest in personalized acoustic echo cancellation (PAEC).
Spike-Triggered Contextual Biasing for End-to-End Mandarin Speech Recognition
However, unlike shallow fusion methods that directly bias the posterior of the ASR model, deep biasing methods implicitly integrate contextual information, making it challenging to control the degree of bias.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representation
By introducing both cross-modal and conversational representations into the decoder, our model retains context over longer sentences without information loss, achieving relative accuracy improvements of 8. 8% and 23% on Mandarin conversation datasets HKUST and MagicData-RAMC, respectively, compared to the standard Conformer model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multi-Speaker Expressive Speech Synthesis via Semi-supervised Contrastive Learning
This paper aims to build an expressive TTS system for multi-speakers, synthesizing a target speaker's speech with multiple styles and emotions.
Protein Language Model-Powered 3D Ligand Binding Site Prediction from Protein Sequence
Then we compute and update the protein-ligand interaction embedding based on the protein residue-level embeddings and ligand atom-level embeddings, and the geometric constraints in the inferred protein contact map and ligand distance map.
Joint Training or Not: An Exploration of Pre-trained Speech Models in Audio-Visual Speaker Diarization
The scarcity of labeled audio-visual datasets is a constraint for training superior audio-visual speaker diarization systems.
U2-KWS: Unified Two-pass Open-vocabulary Keyword Spotting with Keyword Bias
Open-vocabulary keyword spotting (KWS), which allows users to customize keywords, has attracted increasingly more interest.
SELM: Speech Enhancement Using Discrete Tokens and Language Models
Given the intrinsic similarity between speech generation and speech enhancement, harnessing semantic information holds potential advantages for speech enhancement tasks.
Modality Exchange Network for Retinogeniculate Visual Pathway Segmentation
Secondly, we propose a cross-fusion module that further enhances the fusion of information between the two modalities.
LESEN: Label-Efficient deep learning for Multi-parametric MRI-based Visual Pathway Segmentation
Recent research has shown the potential of deep learning in multi-parametric MRI-based visual pathway (VP) segmentation.
ICMC-ASR: The ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition Challenge
To promote speech processing and recognition research in driving scenarios, we build on the success of the Intelligent Cockpit Speech Recognition Challenge (ICSRC) held at ISCSLP 2022 and launch the ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition (ICMC-ASR) Challenge.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition
While automatic speech recognition (ASR) systems degrade significantly in noisy environments, audio-visual speech recognition (AVSR) systems aim to complement the audio stream with noise-invariant visual cues and improve the system's robustness.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+4
BS-PLCNet: Band-split Packet Loss Concealment Network with Multi-task Learning Framework and Multi-discriminators
Packet loss is a common and unavoidable problem in voice over internet phone (VoIP) systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversion
Specifically, to enable streaming capability, StreamVoice employs a fully causal context-aware LM with a temporal-independent acoustic predictor, while alternately processing semantic and acoustic features at each time step of autoregression which eliminates the dependence on complete source speech.
Approximate Message Passing-Enhanced Graph Neural Network for OTFS Data Detection
Orthogonal time frequency space (OTFS) modulation has emerged as a promising solution to support high-mobility wireless communications, for which, cost-effective data detectors are critical.
Anatomy-guided fiber trajectory distribution estimation for cranial nerves tractography
Diffusion MRI tractography is an important tool for identifying and analyzing the intracranial course of cranial nerves (CNs).
Facilitating Reinforcement Learning for Process Control Using Transfer Learning: Perspectives
This paper provides insights into deep reinforcement learning (DRL) for process control from the perspective of transfer learning.
Enhancing Lip Reading with Multi-Scale Video and Multi-Encoder
Automatic lip-reading (ALR) aims to automatically transcribe spoken content from a speaker's silent lip motion captured in video.
MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection
Recent advancements in anomaly detection have seen the efficacy of CNN- and transformer-based approaches.
Dynamic fault detection and diagnosis of industrial alkaline water electrolyzer process with variational Bayesian dictionary learning
A novel robust dynamic variational Bayesian dictionary learning (RDVDL) monitoring approach is proposed to improve the reliability and safety of AWE operation.
Nonlinear sparse variational Bayesian learning based model predictive control with application to PEMFC temperature control
Variational inference is used by NSVB-MPC to assess the predictive accuracy and make the necessary corrections to quantify system uncertainty.
