PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers

CVPR 2023  ยท  Jiacong Xu, Zixiang Xiong, Shankar P. Bhattacharyya ยท

Two-branch network architecture has shown its efficiency and effectiveness in real-time semantic segmentation tasks. However, direct fusion of high-resolution details and low-frequency context has the drawback of detailed features being easily overwhelmed by surrounding contextual information. This overshoot phenomenon limits the improvement of the segmentation accuracy of existing two-branch models. In this paper, we make a connection between Convolutional Neural Networks (CNN) and Proportional-Integral-Derivative (PID) controllers and reveal that a two-branch network is equivalent to a Proportional-Integral (PI) controller, which inherently suffers from similar overshoot issues. To alleviate this problem, we propose a novel three-branch network architecture: PIDNet, which contains three branches to parse detailed, context and boundary information, respectively, and employs boundary attention to guide the fusion of detailed and context branches. Our family of PIDNets achieve the best trade-off between inference speed and accuracy and their accuracy surpasses all the existing models with similar inference speed on the Cityscapes and CamVid datasets. Specifically, PIDNet-S achieves 78.6% mIOU with inference speed of 93.2 FPS on Cityscapes and 80.1% mIOU with speed of 153.7 FPS on CamVid.

PDF Abstract CVPR 2023 PDF CVPR 2023 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semantic Segmentation CamVid PIDNet-Wider Mean IoU 82.0% # 5
Real-Time Semantic Segmentation CamVid PIDNet-M (Cityscapes-Pretrained) mIoU 82.0 # 1
Time (ms) 11.7 # 6
Frame (fps) 85.6(3090) # 19
Real-Time Semantic Segmentation CamVid PIDNet-S (Cityscapes-Pretrained) mIoU 80.1 # 5
Time (ms) 6.5 # 2
Frame (fps) 153.7(3090) # 19
Real-Time Semantic Segmentation Cityscapes test PIDNet-S mIoU 78.6% # 4
Time (ms) 10.7 # 6
Frame (fps) 93.2(3090) # 27
Real-Time Semantic Segmentation Cityscapes test PIDNet-M mIoU 79.8% # 3
Time (ms) 23.7 # 15
Frame (fps) 42.2(3090) # 27
Real-Time Semantic Segmentation Cityscapes test PIDNet-L mIoU 80.6% # 1
Time (ms) 32.2 # 17
Frame (fps) 31.1(3090) # 27
Real-Time Semantic Segmentation Cityscapes val PIDNet-S mIoU 78.8% # 7
Frame (fps) 93.2(3090) # 15
Time (ms) 10.7 # 1
Real-Time Semantic Segmentation Cityscapes val PIDNet-M mIoU 79.9% # 3
Frame (fps) 42.2(3090) # 15
Time (ms) 23.7 # 2
Real-Time Semantic Segmentation Cityscapes val PIDNet-L mIoU 80.9% # 1
Frame (fps) 31.1(3090) # 15
Time (ms) 32.2 # 3

Methods