PP-LiteSeg: A Superior Real-Time Semantic Segmentation Model

Real-world applications have high demands for semantic segmentation methods. Although semantic segmentation has made remarkable leap-forwards with deep learning, the performance of real-time methods is not satisfactory. In this work, we propose PP-LiteSeg, a novel lightweight model for the real-time semantic segmentation task. Specifically, we present a Flexible and Lightweight Decoder (FLD) to reduce computation overhead of previous decoder. To strengthen feature representations, we propose a Unified Attention Fusion Module (UAFM), which takes advantage of spatial and channel attention to produce a weight and then fuses the input features with the weight. Moreover, a Simple Pyramid Pooling Module (SPPM) is proposed to aggregate global context with low computation cost. Extensive evaluations demonstrate that PP-LiteSeg achieves a superior trade-off between accuracy and speed compared to other methods. On the Cityscapes test set, PP-LiteSeg achieves 72.0% mIoU/273.6 FPS and 77.5% mIoU/102.6 FPS on NVIDIA GTX 1080Ti. Source code and models are available at PaddleSeg: https://github.com/PaddlePaddle/PaddleSeg.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Real-Time Semantic Segmentation CamVid PP-LiteSeg-T mIoU 73.3 # 13
Frame (fps) 222.3 # 1
Real-Time Semantic Segmentation CamVid PP-LiteSeg-B mIoU 75 # 10
Frame (fps) 154.8 # 2
Real-Time Semantic Segmentation Cityscapes test PP-LiteSeg-B1 mIoU 73.9% # 19
Frame (fps) 195.3 # 2
Real-Time Semantic Segmentation Cityscapes test PP-LiteSeg-T1 mIoU 72.0% # 23
Frame (fps) 273.6 # 1
Real-Time Semantic Segmentation Cityscapes test PP-LiteSeg-B2 mIoU 77.5% # 6
Frame (fps) 102.6 # 10
Real-Time Semantic Segmentation Cityscapes test PP-LiteSeg-T2 mIoU 74.9% # 14
Frame (fps) 143.6 # 6
Real-Time Semantic Segmentation Cityscapes val PP-LiteSeg-T1 mIoU 73.1 # 7
Real-Time Semantic Segmentation Cityscapes val PP-LiteSeg-T2 mIoU 76 # 5
Real-Time Semantic Segmentation Cityscapes val PP-LiteSeg-B1 mIoU 75.3 # 6
Real-Time Semantic Segmentation Cityscapes val PP-LiteSeg-B2 mIoU 78.2 # 4

Methods