Speech Recognition

Edit

478 papers with code • 104 benchmarks • 56 datasets

Speech recognition is the task of recognising speech within audio and converting it into text.

( Image credit: SpecAugment )

Benchmarks

Add a Result

Dataset	Best Model	Paper Title	Compare
LibriSpeech test-clean	Conformer + Wav2vec 2.0 + SpecAugment-based Noisy Student Training with Libri-Light	Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition	See all
LibriSpeech test-other	Conformer + Wav2vec 2.0 + SpecAugment-based Noisy Student Training with Libri-Light	Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition	See all
TIMIT	wav2vec 2.0	wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations	See all
Switchboard + Hub500	IBM (LSTM+Conformer encoder-decoder)	On the limit of English conversational speech recognition	See all
WSJ eval92	tdnn + chain	Purely sequence-trained neural networks for ASR based on lattice-free MMI	See all
MediaSpeech	Quartznet	MediaSpeech: Multilanguage ASR Benchmark and Dataset	See all
swb_hub_500 WER fullSWBCH	IBM (LSTM+Conformer encoder-decoder)	On the limit of English conversational speech recognition	See all
WSJ eval93	Deep Speech 2	Deep Speech 2: End-to-End Speech Recognition in English and Mandarin	See all
Hub5'00 SwitchBoard	LAS + SpecAugment (with LM, Switchboard mild policy)	SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition	See all
Libri-Light test-clean	wav2vec 2.0 Large-10h-LV-60k	wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations	See all
Libri-Light test-other	wav2vec 2.0 Large-10h-LV-60k	wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations	See all
AISHELL-1	CTC/Att	Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition	See all
Fongbe audio	Triphone (39 features) + LDA and MLLT + SGMM	First Automatic Fongbe Continuous Speech Recognition System: Development of Acoustic Models and Language Models	See all
Switchboard (300hr)	End-to-end LF-MMI	End-to-end speech recognition using lattice-free MMI	See all
Hub5'00 CallHome	Espresso	Espresso: A Fast End-to-end Neural Speech Recognition Toolkit	See all
CHiME-6 dev_gss12	RNN-T	Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcription	See all
Hub5'00 FISHER-SWBD	CTC-CRF	CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latency	See all
LibriSpeech train-clean-100 test-clean	wav2vec_wav2letter	Self-training and Pre-training are Complementary for Speech Recognition	See all
LibriSpeech train-clean-100 test-other	wav2vec_wav2letter	Self-training and Pre-training are Complementary for Speech Recognition	See all
SPGISpeech	Conformer	SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition	See all
Common Voice	SpeechStew (1B)	SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network	See all

Show all 101 benchmarks

Collapse benchmarks

Datasets

Subtasks

Sequence-To-Sequence Speech Recognition

Accented Speech Recognition

Noisy Speech Recognition

English Conversational Speech Recognition

Greatest papers with code

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

tensorflow/models • • 8 Dec 2015

We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages.

Ranked #1 on Noisy Speech Recognition on CHiME clean

Accented Speech Recognition End-To-End Speech Recognition +1

70,697

Paper
Code

fairseq S2T: Fast Speech-to-Text Modeling with fairseq

huggingface/transformers • • Asian Chapter of the Association for Computational Linguistics 2020

We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T) modeling tasks such as end-to-end speech recognition and speech-to-text translation.

Ranked #4 on Speech-to-Text Translation on MuST-C EN->DE

End-To-End Speech Recognition Machine Translation +3

48,862

Paper
Code

Unsupervised Cross-lingual Representation Learning for Speech Recognition

huggingface/transformers • • 24 Jun 2020

This paper presents XLSR which learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages.

Quantization Representation Learning +1

48,862

Paper
Code

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

huggingface/transformers • • NeurIPS 2020

We show for the first time that learning powerful representations from speech audio alone followed by fine-tuning on transcribed speech can outperform the best semi-supervised methods while being conceptually simpler.

Ranked #1 on Speech Recognition on Libri-Light test-other

Quantization Self-Supervised Learning +1

48,862

Paper
Code

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

mozilla/DeepSpeech • • 18 Apr 2019

On LibriSpeech, we achieve 6. 8% WER on test-other without the use of a language model, and 5. 8% WER with shallow fusion with a language model.

Ranked #1 on Speech Recognition on Hub5'00 SwitchBoard

Data Augmentation End-To-End Speech Recognition +2

17,754

Paper
Code

Deep Speech: Scaling up end-to-end speech recognition

mozilla/DeepSpeech • • 17 Dec 2014

We present a state-of-the-art speech recognition system developed using end-to-end deep learning.

Ranked #2 on Accented Speech Recognition on VoxForge American-Canadian

Accented Speech Recognition End-To-End Speech Recognition

17,754

Paper
Code

Unsupervised Speech Recognition

pytorch/fairseq • • 24 May 2021

Despite rapid progress in the recent past, current speech recognition systems still require labeled training data which limits this technology to a small fraction of the languages spoken around the globe.

Speech Recognition Unsupervised Speech Recognition

13,372

Paper
Code

vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations

pytorch/fairseq • • ICLR 2020

We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task.

Ranked #2 on Speech Recognition on TIMIT (using extra training data)

General Classification Self-Supervised Learning +1

13,372

Paper
Code

wav2vec: Unsupervised Pre-training for Speech Recognition

pytorch/fairseq • • 11 Apr 2019

Our experiments on WSJ reduce WER of a strong character-based log-mel filterbank baseline by up to 36% when only a few hours of transcribed data is available.

Ranked #5 on Speech Recognition on TIMIT (using extra training data)

General Classification Speech Recognition +1