The semantic segmentation task is to assign a label from a label set to each pixel in an image. In the case of fully supervised setting, the dataset consists of images and their corresponding pixel-level class-specific annotations (expensive pixel-level annotations). However, in the weakly-supervised setting, the dataset consists of images and corresponding annotations that are relatively easy to obtain, such as tags/labels of objects present in the image.
( Image credit: Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing )
For generating the pseudo labels, we first identify confident seed areas of object classes from attention maps of an image classification model, and propagate them to discover the entire instance areas with accurate boundaries.
IMAGE CLASSIFICATION INSTANCE SEGMENTATION WEAKLY-SUPERVISED SEMANTIC SEGMENTATION
Our method is based on the observation that equivariance is an implicit constraint in fully supervised semantic segmentation, whose pixel-level labels take the same spatial transformation as the input images during data augmentation.
At each time step, the system receives as input a video frame, predicts the optical flow based on the current observation and the LSTM memory state as a dense transformation map, and applies it to the current frame to generate the next frame.
MOTION ESTIMATION OPTICAL FLOW ESTIMATION WEAKLY-SUPERVISED SEMANTIC SEGMENTATION
To alleviate this issue, we present a novel framework that generates segmentation labels of images given their image-level class labels.
Deep convolutional neural networks (DCNNs) trained on a large number of images with strong pixel-level annotations have recently significantly pushed the state-of-art in semantic image segmentation.
SEMI-SUPERVISED SEMANTIC SEGMENTATION WEAKLY-SUPERVISED SEMANTIC SEGMENTATION
Inspired by the traditional image segmentation methods of seeded region growing, we propose to train a semantic segmentation network starting from the discriminative regions and progressively increase the pixel-level supervision using by seeded region growing.
We present a weakly supervised model that jointly performs both semantic- and instance-segmentation -- a particularly relevant problem given the substantial cost of obtaining pixel-perfect annotation for these tasks.
Ranked #17 on
Panoptic Segmentation
on Cityscapes val
INSTANCE SEGMENTATION PANOPTIC SEGMENTATION WEAKLY-SUPERVISED INSTANCE SEGMENTATION WEAKLY-SUPERVISED PANOPTIC SEGMENTATION WEAKLY-SUPERVISED SEMANTIC SEGMENTATION
To the best of our knowledge, the method of [Pathak et al., 2015] is the only prior work that addresses deep CNNs with linear constraints in weakly supervised segmentation.
MEDICAL IMAGE SEGMENTATION WEAKLY SUPERVISED SEGMENTATION WEAKLY-SUPERVISED SEMANTIC SEGMENTATION
Existing weakly-supervised semantic segmentation methods using image-level annotations typically rely on initial responses to locate object regions.
Moreover, our approach ranked 1st place in the Weakly-Supervised Semantic Segmentation Track of CVPR2020 Learning from Imperfect Data Challenge.