Benchmarks
| TREND | DATASET | BEST METHOD | PAPER TITLE | PAPER | CODE | COMPARE |
|---|---|---|---|---|---|---|
|
||||||
|
||||||
|
||||||
|
||||||
|
Datasets
Greatest papers with code
LipNet: End-to-End Sentence-level Lipreading
• rizkiarm/LipNet
• 
Lipreading is the task of decoding text from the movement of a speaker's mouth.
Ranked #3 on
Lipreading
on GRID corpus (mixed-speech)
End-to-end Audiovisual Speech Recognition
• mpc001/end-to-end-Lipreading
• 
In presence of high levels of noise, the end-to-end audiovisual model significantly outperforms both audio-only models.
Ranked #10 on
Lipreading
on Lip Reading in the Wild
LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild
• Fengdalu/Lipreading-DenseNet3D
•
It has shown a large variation in this benchmark in several aspects, including the number of samples in each class, video resolution, lighting conditions, and speakers' attributes such as pose, age, gender, and make-up.
Ranked #8 on
Lipreading
on CAS-VSR-W1k (LRW-1000)
Deep word embeddings for visual speech recognition
• tstafylakis/Lipreading-ResNet
•
In this paper we present a deep learning architecture for extracting word embeddings for visual speech recognition.
Combining Residual Networks with LSTMs for Lipreading
• tstafylakis/Lipreading-ResNet
•
We propose an end-to-end deep learning architecture for word-level visual speech recognition.
Ranked #12 on
Lipreading
on Lip Reading in the Wild
Lipreading using Temporal Convolutional Networks
• mpc001/Lipreading_using_Temporal_Convolutional_Networks
•
We present results on the largest publicly-available datasets for isolated word recognition in English and Mandarin, LRW and LRW1000, respectively.
Ranked #3 on
Lipreading
on Lip Reading in the Wild
Learn an Effective Lip Reading Model without Pains
• Fengdalu/learn-an-effective-lip-reading-model-without-pains
•
Considering the non-negligible effects of these strategies and the existing tough status to train an effective lip reading model, we perform a comprehensive quantitative study and comparative analysis, for the first time, to show the effects of several different choices for lip reading.
Ranked #1 on
Lipreading
on CAS-VSR-W1k (LRW-1000)
(using extra training data)
Discriminative Multi-modality Speech Recognition
CVPR 2020
• JackSyu/Discriminative-Multi-modality-Speech-Recognition
•
Vision is often used as a complementary modality for audio speech recognition (ASR), especially in the noisy environment where performance of solo audio modality significantly deteriorates.
Ranked #5 on
Lipreading
on Lip Reading in the Wild
AUDIO-VISUAL SPEECH RECOGNITION LIPREADING SPEECH RECOGNITION
Mutual Information Maximization for Effective Lip Reading
By combining these two advantages together, the proposed method is expected to be both discriminative and robust for effective lip reading.
Ranked #6 on
Lipreading
on Lip Reading in the Wild
Deformation Flow Based Two-Stream Network for Lip Reading
• jingyunx/Deformation-Flow-Based-Two-stream-Network
•
Observing on the continuity in adjacent frames in the speaking process, and the consistency of the motion patterns among different speakers when they pronounce the same phoneme, we model the lip movements in the speaking process as a sequence of apparent deformations in the lip region.
Ranked #4 on
Lipreading
on CAS-VSR-W1k (LRW-1000)