Boosting Semantic Segmentation via Feature Enhancement

Semantic segmentation aims to map each pixel of an image into its correspond-ing semantic label. Most existing methods either mainly concentrate on high-levelfeatures or simple combination of low-level and high-level features from backboneconvolutional networks, which may weaken or even ignore the compensation be-tween different levels. To effectively take advantages from both shallow (textural)and deep (semantic) features, this paper proposes a novel plug-and-play module,namelyfeature enhancement module(FEM). The proposed FEM first aligns fea-tures from different stages through a learnable filter to extract desired information,and then enhances target features by taking in the extracted message. Two types ofFEM,i.e.detail FEM and semantic FEM, are customized. Concretely, the formertype strengthens textural information to protect key but tiny/low-contrast detailsfrom suppression/removal, while the other one highlights structural informationto boost segmentation performance. By equipping a given backbone network withFEMs, there might contain two information flows,i.e.detail flow and seman-tic flow. Extensive experiments on Cityscapes, PASCAL Context, and ADE20Kdatasets are conducted to validate the effectiveness of our design, and reveal itssuperiority over other state-of-the-art alternatives.