Context Perception Parallel Decoder for Scene Text Recognition

23 Jul 2023  ·  Yongkun Du, Zhineng Chen, Caiyan Jia, Xiaoting Yin, Chenxia Li, Yuning Du, Yu-Gang Jiang ·

Scene text recognition (STR) methods have struggled to attain high accuracy and fast inference speed. Autoregressive (AR)-based models implement the recognition in a character-by-character manner, showing superiority in accuracy but with slow inference speed. Alternatively, parallel decoding (PD)-based models infer all characters in a single decoding pass, offering faster inference speed but generally worse accuracy. We first present an empirical study of AR decoding in STR, and discover that the AR decoder not only models linguistic context, but also provides guidance on visual context perception. Consequently, we propose Context Perception Parallel Decoder (CPPD) to predict the character sequence in a PD pass. CPPD devises a character counting module to infer the occurrence count of each character, and a character ordering module to deduce the content-free reading order and placeholders. Meanwhile, the character prediction task associates the placeholders with characters. They together build a comprehensive recognition context. We construct a series of CPPD models and also plug the proposed modules into existing STR decoders. Experiments on both English and Chinese benchmarks demonstrate that the CPPD models achieve highly competitive accuracy while running approximately 8x faster than their AR-based counterparts. Moreover, the plugged models achieve significant accuracy improvements. Code is at \href{}{this https URL}.

PDF Abstract

Results from the Paper

 Ranked #1 on Scene Text Recognition on CUTE80 (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Scene Text Recognition CUTE80 CPPD Accuracy 99.7 # 1
Scene Text Recognition ICDAR2015 CPPD Accuracy 91.7 # 3
Scene Text Recognition IIIT5k CPPD Accuracy 99.3 # 5
Scene Text Recognition SVT CPPD Accuracy 98.5 # 6
Scene Text Recognition SVTP CPPD Accuracy 96.7 # 7