Video Panoptic Segmentation

17 papers with code • 3 benchmarks • 4 datasets

Video Panoptic Segmentation is a computer vision task that extends panoptic segmentation by incorporating temporal dimension. That is, given a video sequence, the goal is to predict the semantic class of each pixel while consistently tracking object instances. Here, the pixels belonging to the same object instance should be assigned the same instance ID throughout the video sequence.


Use these libraries to find Video Panoptic Segmentation models and implementations

Most implemented papers

Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation

lxtgh/tube-link ICCV 2023

Our framework is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks.

Video Panoptic Segmentation

mcahny/vps CVPR 2020

In this paper, we propose and explore a new video extension of this task, called video panoptic segmentation.

ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation

joe-siyuan-qiao/ViP-DeepLab CVPR 2021

We name this joint task as Depth-aware Video Panoptic Segmentation, and propose a new evaluation metric along with two derived datasets for it, which will be made available to the public.

STEP: Segmenting and Tracking Every Pixel

google-research/deeplab2 23 Feb 2021

The task of assigning semantic classes and track identities to every pixel in a video is called video panoptic segmentation.

PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentation

harboryuan/polyphonicformer 5 Dec 2021

The Depth-aware Video Panoptic Segmentation (DVPS) is a new challenging vision problem that aims to predict panoptic segmentation and depth in a video simultaneously.

Large-Scale Video Panoptic Segmentation in the Wild: A Benchmark

vipseg-dataset/vipseg-dataset CVPR 2022

In contrast, our large-scale VIdeo Panoptic Segmentation in the Wild (VIPSeg) dataset provides 3, 536 videos and 84, 750 frames with pixel-level panoptic annotations, covering a wide range of real-world scenarios and categories.

Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation

lxtgh/video-k-net CVPR 2022

We hope this simple, yet effective method can serve as a new, flexible baseline in unified video segmentation design.

Waymo Open Dataset: Panoramic Video Panoptic Segmentation

google-research/deeplab2 15 Jun 2022

We therefore present the Waymo Open Dataset: Panoramic Video Panoptic Segmentation Dataset, a large-scale dataset that offers high-quality panoptic segmentation labels for autonomous driving.

PVO: Panoptic Visual Odometry

zju3dv/PVO CVPR 2023

We present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information.

Context-Aware Relative Object Queries To Unify Video Instance and Panoptic Segmentation

AnwesaChoudhuri/CAROQ CVPR 2023

We evaluate the proposed approach across three challenging tasks: video instance segmentation, multi-object tracking and segmentation, and video panoptic segmentation.