High-Resolution Representations for Labeling Pixels and Regions

High-resolution representation learning plays an essential role in many vision problems, e.g., pose estimation and semantic segmentation. The high-resolution network (HRNet)~\cite{SunXLW19}, recently developed for human pose estimation, maintains high-resolution representations through the whole process by connecting high-to-low resolution convolutions in \emph{parallel} and produces strong high-resolution representations by repeatedly conducting fusions across parallel convolutions. In this paper, we conduct a further study on high-resolution representations by introducing a simple yet effective modification and apply it to a wide range of vision tasks. We augment the high-resolution representation by aggregating the (upsampled) representations from all the parallel convolutions rather than only the representation from the high-resolution convolution as done in~\cite{SunXLW19}. This simple modification leads to stronger representations, evidenced by superior results. We show top results in semantic segmentation on Cityscapes, LIP, and PASCAL Context, and facial landmark detection on AFLW, COFW, $300$W, and WFLW. In addition, we build a multi-level representation from the high-resolution representation and apply it to the Faster R-CNN object detection framework and the extended frameworks. The proposed approach achieves superior results to existing single-model networks on COCO object detection. The code and models have been publicly available at \url{https://github.com/HRNet}.

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Semantic Segmentation ADE20K HRNetV2 Validation mIoU 43.2 # 202
Face Alignment AFLW-19 HR-Net NME_diag (%, Full) 1.57 # 13
NME_diag (%, Frontal) 1.46 # 9
Semantic Segmentation Cityscapes test HRNet (HRNetV2-W48) Mean IoU (class) 81.6% # 37
Face Alignment COFW HRNet NME (inter-ocular) 3.45% # 12
Semantic Segmentation LIP val HRNetV2 (HRNetV2-W48) mIoU 55.90% # 7

Results from Other Papers

Task Dataset Model Metric Name Metric Value Rank Uses Extra
Training Data
Source Paper Compare
Face Alignment 300W HR-Net NME_inter-ocular (%, Full) 3.32 # 24
NME_inter-ocular (%, Common) 2.87 # 21
NME_inter-ocular (%, Challenge) 5.15 # 24
Semantic Segmentation ADE20K val HRNetV2 (HRNetV2-W48) mIoU 42.99 # 86