Learn an Effective Lip Reading Model without Pains

15 Nov 2020  ·  Dalu Feng, Shuang Yang, Shiguang Shan, Xilin Chen ·

Lip reading, also known as visual speech recognition, aims to recognize the speech content from videos by analyzing the lip dynamics. There have been several appealing progress in recent years, benefiting much from the rapidly developed deep learning techniques and the recent large-scale lip-reading datasets. Most existing methods obtained high performance by constructing a complex neural network, together with several customized training strategies which were always given in a very brief description or even shown only in the source code. We find that making proper use of these strategies could always bring exciting improvements without changing much of the model. Considering the non-negligible effects of these strategies and the existing tough status to train an effective lip reading model, we perform a comprehensive quantitative study and comparative analysis, for the first time, to show the effects of several different choices for lip reading. By only introducing some easy-to-get refinements to the baseline pipeline, we obtain an obvious improvement of the performance from 83.7% to 88.4% and from 38.2% to 55.7% on two largest public available lip reading datasets, LRW and LRW-1000, respectively. They are comparable and even surpass the existing state-of-the-art results.

PDF Abstract

Results from the Paper


 Ranked #1 on Lipreading on CAS-VSR-W1k (LRW-1000) (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Lipreading CAS-VSR-W1k (LRW-1000) 3D-ResNet + Bi-GRU + MixUp + Label Smooth + Cosine LR Top-1 Accuracy 48.3% # 3
Lipreading CAS-VSR-W1k (LRW-1000) 3D-ResNet + Bi-GRU + MixUp + Label Smooth + Cosine LR (Word Boundary) Top-1 Accuracy 55.7% # 1
Lipreading Lip Reading in the Wild 3D-ResNet + Bi-GRU + MixUp + Label Smoothing + Cosine LR Top-1 Accuracy 85.5 # 3
Lipreading Lip Reading in the Wild 3D-ResNet + Bi-GRU + MixUp + Label Smoothing + Cosine LR (Word Boundary) Top-1 Accuracy 88.4 # 2

Methods


No methods listed for this paper. Add relevant methods here