Search Results for author:
Found 58 papers, 17 papers with code
Improving Code-switched ASR with Linguistic Information
This paper seeks to improve the performance of automatic speech recognition (ASR) systems operating on code-switched speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Language Bias in Self-Supervised Learning For Automatic Speech Recognition
Self-supervised learning (SSL) is used in deep learning to train on large datasets without the need for expensive labelling of the data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Beyond Oversmoothing: Evaluating DDPM and MSE for Scalable Speech Synthesis in ASR
However, the paradox remains that ASR systems, when trained on TTS output that is judged as natural by humans, continue to perform badly on real speech.
Semi-Supervised Cognitive State Classification from Speech with Multi-View Pseudo-Labeling
The lack of labeled data is a common challenge in speech classification tasks, particularly those requiring extensive subjective assessment, such as cognitive state classification.
Revise, Reason, and Recognize: LLM-Based Emotion Recognition via Emotion-Specific Prompts and ASR Error Correction
Furthermore, we propose a Revise-Reason-Recognize prompting pipeline for robust LLM-based emotion recognition from spoken language with ASR errors.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition
Given recent advances in generative AI technology, a key question is how large language models (LLMs) can enhance acoustic modeling tasks using text decoding results from a frozen, pretrained automatic speech recognition (ASR) model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
TTSDS -- Text-to-Speech Distribution Score
Many recently published Text-to-Speech (TTS) systems produce audio close to real speech.
Speech Emotion Recognition with ASR Transcripts: A Comprehensive Study on Word Error Rate and Fusion Techniques
Text data is commonly utilized as a primary input to enhance Speech Emotion Recognition (SER) performance and reliability.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Phonetic Error Analysis of Raw Waveform Acoustic Models with Parametric and Non-Parametric CNNs
Our raw waveform acoustic models consists of parametric (Sinc2Net) or non-parametric CNNs and Bidirectional LSTMs, achieving down to 13. 7%/15. 2% PERs on TIMIT Dev/Test sets, outperforming reported PERs for raw waveform models in the literature.
1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem
Previous work has utilised class weighted loss for training, but problems remain as it sometimes causes over-fitting for minor classes or under-fitting for major classes.
Explainable Attribute-Based Speaker Verification
This paper proposes a fully explainable approach to speaker verification (SV), a task that fundamentally relies on individual speaker characteristics.
Crossmodal ASR Error Correction with Discrete Speech Units
ASR remains unsatisfactory in scenarios where the speaking style diverges from that used to train ASR systems, resulting in erroneous transcripts.
LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots
We introduce LLM-Personalize, a novel framework with an optimization pipeline designed to personalize LLM planners for household robotics.
Quantifying the perceptual value of lexical and non-lexical channels in speech
Speech is a fundamental means of communication that can be seen to provide two channels for transmitting information: the lexical channel of which words are said, and the non-lexical channel of how they are spoken.
Can We Trust Explainable AI Methods on ASR? An Evaluation on Phoneme Recognition
Explainable AI (XAI) techniques have been widely used to help explain and understand the output of deep learning models in fields such as image classification and Natural Language Processing.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Transfer Learning for Personality Perception via Speech Emotion Recognition
In this work, we investigate the relationship between two affective attributes: personality and emotion, from a transfer learning perspective.
ASR and Emotional Speech: A Word-Level Investigation of the Mutual Impact of Speech and Emotion Recognition
To overcome this challenge, we investigate how Automatic Speech Recognition (ASR) performs on emotional speech by analyzing the ASR performance on emotion corpora and examining the distribution of word errors and confidence scores in ASR transcripts to gain insight into how emotion affects ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition
Fusing multiple modalities has proven effective for multimodal information processing.
The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR
Although the great many advances in English automatic speech recognition (ASR) over the past decades, results are usually reported based on test datasets which fail to represent the diversity of English as spoken today around the globe.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Explanations for Automatic Speech Recognition
We address quality assessment for neural network based ASR by providing explanations that help increase our understanding of the system and ultimately help build trust in the system.
Automatic Speech Recognition
Explainable Artificial Intelligence (XAI)
+5
Transfer Learning for Olfactory Object Detection
We investigate the effect of style and category similarity in multiple datasets used for object detection pretraining.
ODOR: The ICPR2022 ODeuropa Challenge on Olfactory Object Recognition
The Odeuropa Challenge on Olfactory Object Recognition aims to foster the development of object detection in the visual arts and to promote an olfactory perspective on digital heritage.
Evaluating and reducing the distance between synthetic and real speech distributions
While modern Text-to-Speech (TTS) systems can produce natural-sounding speech, they remain unable to reproduce the full diversity found in natural speech data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Exploration of A Self-Supervised Speech Model: A Study on Emotional Corpora
Self-supervised speech models have grown fast during the past few years and have proven feasible for use in various downstream tasks.
ICC++: Explainable Image Retrieval for Art Historical Corpora using Image Composition Canvas
In this work, we present a novel approach called Image Composition Canvas (ICC++) to compare and retrieve images having similar compositional elements.
Mask-combine Decoding and Classification Approach for Punctuation Prediction with real-time Inference Constraints
In this work, we unify several existing decoding strategies for punctuation prediction in one framework and introduce a novel strategy which utilises multiple predictions at each word across different windows.
Deciphering Speech: a Zero-Resource Approach to Cross-Lingual Transfer in ASR
We present a method for cross-lingual training an ASR system using absolutely no transcribed training data from the target language, and with no phonetic knowledge of the language in question.
Fusing ASR Outputs in Joint Training for Speech Emotion Recognition
However, due to the scarcity of emotion labelled data and the difficulty of recognizing emotional speech, it is hard to obtain reliable linguistic features and models in this research area.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
It's not what you said, it's how you said it: discriminative perception of speech as a multichannel communication system
People convey information extremely effectively through spoken interaction using multiple channels of information transmission: the lexical channel of what is said, and the non-lexical channel of how it is said.
Segmenting Subtitles for Correcting ASR Segmentation Errors
Typical ASR systems segment the input audio into utterances using purely acoustic information, which may not resemble the sentence-like units that are expected by conventional machine translation (MT) systems for Spoken Language Translation.
Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers
Although the lower layers of a deep neural network learn features which are transferable across datasets, these layers are not transferable within the same dataset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Enhancing Human Pose Estimation in Ancient Vase Paintings via Perceptually-grounded Style Transfer Learning
(2) To improve the already strong results further, we created a small dataset (ClassArch) consisting of ancient Greek vase paintings from the 6-5th century BCE with person and pose annotations.
Stochastic Attention Head Removal: A simple and effective method for improving Transformer Based ASR Models
To the best of our knowledge, we have achieved state-of-the-art end-to-end Transformer based model performance on Switchboard and AMI.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
On the Usefulness of Self-Attention for Automatic Speech Recognition with Transformers
Self-attention models such as Transformers, which can capture temporal relationships without being limited by the distance between events, have given competitive speech recognition results.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Leveraging speaker attribute information using multi task learning for speaker verification and diarization
On a test set of US Supreme Court recordings, we show that by leveraging two additional forms of speaker attribute information derived respectively from the matched training data, and VoxCeleb corpus, we improve the performance of our deep speaker embeddings for both verification and diarization tasks, achieving a relative improvement of 26. 2% in DER and 6. 7% in EER compared to baselines using speaker labels only.
Subtitles to Segmentation: Improving Low-Resource Speech-to-Text Translation Pipelines
In this work, we focus on improving ASR output segmentation in the context of low-resource language speech-to-text translation.
Understanding Compositional Structures in Art Historical Images using Pose and Gaze Priors
These compositions are useful in analyzing the interactions in an image to study artists and their artworks.
Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview
We present a structured overview of adaptation algorithms for neural network-based speech recognition, considering both hybrid hidden Markov model / neural network systems and end-to-end neural network systems, with a focus on speaker adaptation, domain adaptation, and accent adaptation.
When Can Self-Attention Be Replaced by Feed Forward Layers?
Recently, self-attention models such as Transformers have given competitive results compared to recurrent neural network systems in speech recognition.
Subtitles to Segmentation: Improving Low-Resource Speech-to-TextTranslation Pipelines
In this work, we focus on improving ASR output segmentation in the context of low-resource language speech-to-text translation.
Recognizing Characters in Art History Using Deep Learning
We present experiments and analysis on three different models and show that the model trained on domain related data gives the best performance for recognizing character.
DropClass and DropAdapt: Dropping classes for deep speaker representation learning
The first proposed method, DropClass, works via periodically dropping a random subset of classes from the training data and the output layer throughout training, resulting in a feature extractor trained on many different classification tasks.
Multi-scale Octave Convolutions for Robust Speech Recognition
We propose a multi-scale octave convolution layer to learn robust speech representations efficiently.
Channel adversarial training for speaker verification and diarization
Previous work has encouraged domain-invariance in deep speaker embedding by adversarially classifying the dataset or labelled environment to which the generated features belong.
Speaker Adaptive Training using Model Agnostic Meta-Learning
Speaker adaptive training (SAT) of neural network acoustic models learns models in a way that makes them more suitable for adaptation to test conditions.
Acoustic Model Adaptation from Raw Waveforms with SincNet
Raw waveform acoustic modelling has recently gained interest due to neural networks' ability to learn feature extraction, and the potential for finding better representations for a given scenario than hand-crafted features.
Embeddings for DNN speaker adaptive training
In this work, we investigate the use of embeddings for speaker-adaptive training of DNNs (DNN-SAT) focusing on a small amount of adaptation data per speaker.
Top-down training for neural networks
Interpreting the top layers as a classifier and the lower layers a feature extractor, one can hypothesize that unwanted network convergence may occur when the classifier has overfit with respect to the feature extractor.
Lattice-Based Unsupervised Test-Time Adaptation of Neural Network Acoustic Models
Acoustic model adaptation to unseen test recordings aims to reduce the mismatch between training and testing conditions.
Lattice-based lightly-supervised acoustic model training
This text data can be used for lightly supervised training, in which text matching the audio is selected using an existing speech recognition model.
Analyzing deep CNN-based utterance embeddings for acoustic model adaptation
We analyze the representations learned by deep CNNs and compare them with deep neural network (DNN) representations and i-vectors, in the context of acoustic model adaptation.
Few-shot learning with attention-based sequence-to-sequence models
End-to-end approaches have recently become popular as a means of simplifying the training and deployment of speech recognition systems.
Learning to adapt: a meta-learning approach for speaker adaptation
The performance of automatic speech recognition systems can be improved by adapting an acoustic model to compensate for the mismatch between training and testing conditions, for example by adapting to unseen speakers.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
WERd: Using Social Text Spelling Variants for Evaluating Dialectal Speech Recognition
We study the problem of evaluating automatic speech recognition (ASR) systems that target dialectal speech input.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
The SUMMA Platform Prototype
We present the first prototype of the SUMMA Platform: an integrated platform for multilingual media monitoring.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition
For language modelling, we made available over 110M words crawled from Aljazeera Arabic website Aljazeera. net for a 10 year duration 2000-2011.
Automatic Dialect Detection in Arabic Broadcast Speech
We used these features in a binary classifier to discriminate between Modern Standard Arabic (MSA) and Dialectal Arabic, with an accuracy of 100%.
