Search Results for author:
Found 10 papers, 2 papers with code
CARE: Large Precision Matrix Estimation for Compositional Data
High-dimensional compositional data are prevalent in many applications.
LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speech
Self-supervised learning (SSL) is at the origin of unprecedented improvements in many different domains including computer vision and natural language processing.
SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding
Unfortunately, token mixing with self-attention takes quadratic time in the length of the speech utterance, slowing down inference as well as training and increasing memory consumption.
Cross-Attention is all you need: Real-Time Streaming Transformers for Personalised Speech Enhancement
Personalised speech enhancement (PSE), which extracts only the speech of a target user and removes everything else from a recorded audio clip, can potentially improve users' experiences of audio AI modules deployed in the wild.
Transformer-based Streaming ASR with Cumulative Attention
In this paper, we propose an online attention mechanism, known as cumulative attention (CA), for streaming Transformer-based automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers
Although the lower layers of a deep neural network learn features which are transferable across datasets, these layers are not transferable within the same dataset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Stochastic Attention Head Removal: A simple and effective method for improving Transformer Based ASR Models
To the best of our knowledge, we have achieved state-of-the-art end-to-end Transformer based model performance on Switchboard and AMI.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
On the Usefulness of Self-Attention for Automatic Speech Recognition with Transformers
Self-attention models such as Transformers, which can capture temporal relationships without being limited by the distance between events, have given competitive speech recognition results.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
When Can Self-Attention Be Replaced by Feed Forward Layers?
Recently, self-attention models such as Transformers have given competitive results compared to recurrent neural network systems in speech recognition.
Top-down training for neural networks
Interpreting the top layers as a classifier and the lower layers a feature extractor, one can hypothesize that unwanted network convergence may occur when the classifier has overfit with respect to the feature extractor.
