Search Results for author:
Found 70 papers, 44 papers with code
FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks
Large language models have revolutionized natural language processing through self-supervised pretraining on massive datasets.
Investigating the Effectiveness of Explainability Methods in Parkinson's Detection from Speech
Speech impairments in Parkinson's disease (PD) provide significant early indicators for diagnosis.
Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve?
Our further token-level perplexity observations reveals that the perplexity degradation is due to a handful of tokens that are not informative about the domain.
Dynamic HumTrans: Humming Transcription Using CNNs and Dynamic Programming
We propose a novel approach for humming transcription that combines a CNN-based architecture with a dynamic programming-based post-processing algorithm, utilizing the recently introduced HumTrans dataset.
What Are They Doing? Joint Audio-Speech Co-Reasoning
In audio and speech processing, tasks usually focus on either the audio or speech modality, even when both sounds and human speech are present in the same audio clip.
LMAC-TD: Producing Time Domain Explanations for Audio Classifiers
Neural networks are typically black-boxes that remain opaque with regards to their decision mechanisms.
ProGRes: Prompted Generative Rescoring on ASR n-Best
Large Language Models (LLMs) have shown their ability to improve the performance of speech recognizers by effectively rescoring the n-best hypotheses generated during the beam search process.
Open-Source Conversational AI with SpeechBrain 1.0
This paper presents SpeechBrain 1. 0, a significant milestone in the evolution of the toolkit, which now has over 200 recipes for speech, audio, and language processing tasks, and more than 100 models available on Hugging Face.
DASB -- Discrete Audio and Speech Benchmark
Discrete audio tokens have recently gained considerable attention for their potential to connect audio and language processing, enabling the creation of modern multimodal large language models.
How Should We Extract Discrete Audio Tokens from Self-Supervised Models?
Discrete audio tokens have recently gained attention for their potential to bridge the gap between audio and language processing.
Phoneme Discretized Saliency Maps for Explainable Detection of AI-Generated Voice
In this paper, we propose Phoneme Discretized Saliency Maps (PDSM), a discretization algorithm for saliency maps that takes advantage of phoneme boundaries for explainable detection of AI-generated voice.
Listenable Maps for Zero-Shot Audio Classifiers
Interpreting the decisions of deep learning models, including audio classifiers, is crucial for ensuring the transparency and trustworthiness of this technology.
Listenable Maps for Audio Classifiers
Despite the impressive performance of deep learning models across diverse tasks, their complexity poses challenges for interpretation.
SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learning
Self-supervised learning (SSL) has achieved remarkable success across various speech-processing tasks.
Focal Modulation Networks for Interpretable Sound Classification
The increasing success of deep neural networks has raised concerns about their inherent black-box nature, posing challenges related to interpretability and trust.
Bayesian Deep Learning for Remaining Useful Life Estimation via Stein Variational Gradient Descent
In particular, we show through experimental studies on simulated run-to-failure turbofan engine degradation data that Bayesian deep learning models trained via Stein variational gradient descent consistently outperform with respect to convergence speed and predictive performance both the same models trained via parametric variational inference and their frequentist counterparts trained via backpropagation.
Are LLMs Robust for Spoken Dialogues?
Large Pre-Trained Language Models have demonstrated state-of-the-art performance in different downstream tasks, including dialogue state tracking and end-to-end response generation.
Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers
Parameter-efficient transfer learning (PETL) methods have emerged as a solid alternative to the standard full fine-tuning approach.
TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch
TorchAudio is an open-source audio and speech processing library built for PyTorch.
CL-MASR: A Continual Learning Benchmark for Multilingual ASR
To the best of our knowledge, CL-MASR is the first continual learning benchmark for the multilingual ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Audio Editing with Non-Rigid Text Prompts
We show that the proposed editing pipeline is able to create audio edits that remain faithful to the input audio.
Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Recently, pre-trained foundation models have enabled significant advancements in multiple fields.
Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Heads
Self-supervised learning (SSL) leverages large datasets of unlabeled speech to reach impressive performance with reduced amounts of annotated data.
Generalization Limits of Graph Neural Networks in Identity Effects Learning
They are usually based on a message-passing mechanism and have gained increasing popularity for their intuitive formulation, which is closely linked to the Weisfeiler-Lehman (WL) test for graph isomorphism to which they have been proven equivalent in terms of expressive power.
Speech Emotion Diarization: Which Emotion Appears When?
Speech Emotion Recognition (SER) typically relies on utterance-level solutions.
RescueSpeech: A German Corpus for Speech Recognition in Search and Rescue Domain
Despite the recent advancements in speech recognition, there are still difficulties in accurately transcribing conversational and emotional speech in noisy and reverberant acoustic environments.
Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?
Self-supervised learning (SSL) has recently allowed leveraging large datasets of unlabeled speech signals to reach impressive performance on speech tasks using only small amounts of annotated data.
Simulated Annealing in Early Layers Leads to Better Generalization
Our principal innovation in this work is to use Simulated annealing in EArly Layers (SEAL) of the network in place of re-initialization of later layers.
Posthoc Interpretation via Quantization
In this paper, we introduce a new approach, called Posthoc Interpretation via Quantization (PIQ), for interpreting decisions made by trained classifiers.
Fine-tuning Strategies for Faster Inference using Speech Self-Supervised Models: A Comparative Study
Self-supervised learning (SSL) has allowed substantial progress in Automatic Speech Recognition (ASR) performance in low-resource settings.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation
End-to-end speech synthesis models directly convert the input characters into an audio representation (e. g., spectrograms).
Resource-Efficient Separation Transformer
Transformers have recently achieved state-of-the-art performance in speech separation.
Learning Representations for New Sound Classes With Continual Self-Supervised Learning
In this paper, we work on a sound recognition system that continually incorporates new sound classes.
Exploring Self-Attention Mechanisms for Speech Separation
In particular, we extend our previous findings on the SepFormer by providing results on more challenging noisy and noisy-reverberant datasets, such as LibriMix, WHAM!, and WHAMR!.
Ranked #1 on
Speech Enhancement
on WHAM!
OSSEM: one-shot speaker adaptive speech enhancement using meta learning
Although deep learning (DL) has achieved notable progress in speech enhancement (SE), further research is still required for a DL-based SE system to adapt effectively and efficiently to particular speakers.
REAL-M: Towards Speech Separation on Real Mixtures
First, we release the REAL-M dataset, a crowd-sourced corpus of real-life mixtures.
MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech
Most of the deep learning-based speech enhancement models are learned in a supervised manner, which implies that pairs of noisy and clean speech are required during training.
Interpretable SincNet-based Deep Learning for Emotion Recognition from EEG brain activity
This result is consistent with recent neuroscience studies on emotion recognition, which found an association between these band suppressions and the behavioral deficits observed in individuals with ASD.
SpeechBrain: A General-Purpose Speech Toolkit
SpeechBrain is an open-source and all-in-one speech toolkit.
MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
The discrepancy between the cost function used for training a speech enhancement model and human auditory perception usually makes the quality of enhanced speech unsatisfactory.
Ranked #23 on
Speech Enhancement
on VoiceBank + DEMAND
Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numbers
This paper introduces Timers and Such, a new open source dataset of spoken English commands for common voice control use cases involving numbers.
Ranked #4 on
Spoken Language Understanding
on Timers and Such
(using extra training data)
ECAPA-TDNN Embeddings for Speaker Diarization
Learning robust speaker embeddings is a crucial step in speaker diarization.
Transformers with Competitive Ensembles of Independent Mechanisms
In this work we explore a way in which the Transformer architecture is deficient: it represents each position with a large monolithic hidden representation and a single set of parameters which are applied over the entire hidden representation.
Attention is All You Need in Speech Separation
Transformers are emerging as a natural alternative to standard RNNs, replacing recurrent computations with a multi-head attention mechanism.
Ranked #7 on
Speech Separation
on WSJ0-3mix
BIRD: Big Impulse Response Dataset
This paper introduces BIRD, the Big Impulse Response Dataset.
Sound Audio and Speech Processing
Quaternion Neural Networks for Multi-channel Distant Speech Recognition
In this paper, we propose to capture these inter- and intra- structural dependencies with quaternion neural networks, which can jointly process multiple signals as whole quaternion entities.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Multi-task self-supervised learning for Robust Speech Recognition
We then propose a revised encoder that better learns short- and long-term speech dynamics with an efficient combination of recurrent and convolutional networks.
Using Speech Synthesis to Train End-to-End Spoken Language Understanding Models
End-to-end models are an attractive new approach to spoken language understanding (SLU) in which the meaning of an utterance is inferred directly from the raw audio without employing the standard pipeline composed of a separately trained speech recognizer and natural language understanding module.
Ranked #7 on
Spoken Language Understanding
on Snips-SmartLights
Retrieving Signals in the Frequency Domain with Deep Complex Extractors
Using the Wall Street Journal Dataset, we compare our phase-aware loss to several others that operate both in the time and frequency domains and demonstrate the effectiveness of our proposed signal extraction method and proposed loss.
Retrieving Signals with Deep Complex Extractors
Building on recent advances, we propose a new deep complex-valued method for signal retrieval and extraction in the frequency domain.
Speech Model Pre-training for End-to-End Spoken Language Understanding
Whereas conventional spoken language understanding (SLU) systems map speech to text, and then text to intent, end-to-end SLU systems map speech directly to intent through a single trainable model.
Ranked #15 on
Spoken Language Understanding
on Fluent Speech Commands
(using extra training data)
Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks
Learning good representations without supervision is still an open issue in machine learning, and is particularly challenging for speech signals, which are often characterized by long sequences with a complex hierarchical structure.
Ranked #2 on
Distant Speech Recognition
on DIRHA English WSJ
Speech and Speaker Recognition from Raw Waveform with SincNet
Deep neural networks can learn complex and abstract representations, that are progressively obtained by combining simpler ones.
Learning Speaker Representations with Mutual Information
Mutual Information (MI) or similar measures of statistical dependence are promising tools for learning these representations in an unsupervised way.
Interpretable Convolutional Filters with SincNet
Deep learning is currently playing a crucial role toward higher levels of artificial intelligence.
Ranked #3 on
Distant Speech Recognition
on DIRHA English WSJ
Speech recognition with quaternion neural networks
Neural network architectures are at the core of powerful automatic speech recognition systems (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
The PyTorch-Kaldi Speech Recognition Toolkit
Experiments, that are conducted on several datasets and tasks, show that PyTorch-Kaldi can effectively be used to develop modern state-of-the-art speech recognizers.
Ranked #1 on
Distant Speech Recognition
on DIRHA English WSJ
Speaker Recognition from Raw Waveform with SincNet
Rather than employing standard hand-crafted features, the latter CNNs learn low-level speech representations from waveforms, potentially allowing the network to better capture important narrow-band speaker characteristics such as pitch and formants.
Quaternion Recurrent Neural Networks
Recurrent neural networks (RNNs) are powerful architectures to model sequential data, due to their capability to learn short and long-term dependencies between the basic elements of a sequence.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Automatic context window composition for distant speech recognition
Distant speech recognition is being revolutionized by deep learning, that has contributed to significantly outperform previous HMM-GMM systems.
Twin Regularization for online speech recognition
Online speech recognition is crucial for developing natural human-machine interfaces.
Light Gated Recurrent Units for Speech Recognition
A field that has directly benefited from the recent advances in deep learning is Automatic Speech Recognition (ASR).
Ranked #6 on
Speech Recognition
on TIMIT
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Deep Learning for Distant Speech Recognition
Deep learning is an emerging technology that is considered one of the most promising directions for reaching higher levels of artificial intelligence.
Realistic multi-microphone data simulation for distant speech recognition
The availability of realistic simulated corpora is of key importance for the future progress of distant speech recognition technology.
Audio and Speech Processing Sound
Contaminated speech training methods for robust DNN-HMM distant speech recognition
Despite the significant progress made in the last years, state-of-the-art speech recognition technologies provide a satisfactory performance only in the close-talking condition.
The DIRHA-English corpus and related tasks for distant-speech recognition in domestic environments
This paper introduces the contents and the possible usage of the DIRHA-ENGLISH multi-microphone corpus, recently realized under the EC DIRHA project.
Improving speech recognition by revising gated recurrent units
First, we suggest to remove the reset gate in the GRU design, resulting in a more efficient single-gate architecture.
Batch-normalized joint training for DNN-based distant speech recognition
Improving distant speech recognition is a crucial step towards flexible human-machine interfaces.
A network of deep neural networks for distant speech recognition
Despite the remarkable progress recently made in distant speech recognition, state-of-the-art technology still suffers from a lack of robustness, especially when adverse acoustic conditions characterized by non-stationary noises and reverberation are met.
The DIRHA simulated corpus
This paper describes a multi-microphone multi-language acoustic corpus being developed under the EC project Distant-speech Interaction for Robust Home Applications (DIRHA).
