Convolutional Character Networks

Recent progress has been made on developing a unified framework for joint text detection and recognition in natural images, but existing joint models were mostly built on two-stage framework by involving ROI pooling, which can degrade the performance on recognition task. In this work, we propose convolutional character networks, referred as CharNet, which is an one-stage model that can process two tasks simultaneously in one pass. CharNet directly outputs bounding boxes of words and characters, with corresponding character labels. We utilize character as basic element, allowing us to overcome the main difficulty of existing approaches that attempted to optimize text detection jointly with a RNN-based recognition branch. In addition, we develop an iterative character detection approach able to transform the ability of character detection learned from synthetic data to real-world images. These technical improvements result in a simple, compact, yet powerful one-stage model that works reliably on multi-orientation and curved text. We evaluate CharNet on three standard benchmarks, where it consistently outperforms the state-of-the-art approaches [25, 24] by a large margin, e.g., with improvements of 65.33%->71.08% (with generic lexicon) on ICDAR 2015, and 54.0%->69.23% on Total-Text, on end-to-end text recognition. Code is available at: https://github.com/MalongTech/research-charnet.

PDF Abstract ICCV 2019 PDF ICCV 2019 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Scene Text Detection ICDAR 2015 CharNet H-88 (multi-scale) F-Measure 91.55 # 2
Precision 92.65 # 2
Recall 90.47 # 3
Scene Text Detection ICDAR 2015 CharNet H-88 (single-scale) F-Measure 90.97 # 3
Precision 89.99 # 16
Recall 91.98 # 1
Scene Text Detection ICDAR 2015 CharNet H-57 (single-scale) F-Measure 89.66 # 9
Precision 88.88 # 21
Recall 90.45 # 4
Scene Text Detection ICDAR 2015 CharNet H-50 (single-scale) F-Measure 89.7 # 8
Precision 91.15 # 9
Recall 88.3 # 7
Scene Text Detection ICDAR 2015 CharNet H-57 (multi-scale) F-Measure 90.06 # 6
Precision 91.43 # 7
Recall 88.74 # 6
Scene Text Detection ICDAR 2015 CharNet H-50 (multi-scale) F-Measure 90.16 # 4
Precision 90.9 # 11
Recall 89.44 # 5
Scene Text Detection ICDAR 2017 MLT CharNet R-50 Precision 77.07 # 11
Recall 70.1 # 5
F-Measure 73.42% # 7
Scene Text Detection ICDAR 2017 MLT CharNet H-88 Precision 81.27 # 5
Recall 70.97 # 3
F-Measure 75.77% # 3
Scene Text Detection Total-Text CharNet H-88 (multi-scale) F-Measure 86.5% # 7
Precision 88 # 13
Recall 85 # 6
Scene Text Detection Total-Text CharNet H-88 F-Measure 85.6% # 11
Precision 89.9 # 5
Recall 81.7 # 12

Methods


No methods listed for this paper. Add relevant methods here