Video Panoptic Segmentation
17 papers with code • 3 benchmarks • 4 datasets
Video Panoptic Segmentation is a computer vision task that extends panoptic segmentation by incorporating temporal dimension. That is, given a video sequence, the goal is to predict the semantic class of each pixel while consistently tracking object instances. Here, the pixels belonging to the same object instance should be assigned the same instance ID throughout the video sequence.
Benchmarks
These leaderboards are used to track progress in Video Panoptic Segmentation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
Libraries
Use these libraries to find Video Panoptic Segmentation models and implementations
Cityscapes-VPS
VIPSeg
KITTI-STEP
LaRS
Most implemented papers
Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation
Our framework is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks.
Video Panoptic Segmentation
mcahny/vps
•
•
CVPR 2020
In this paper, we propose and explore a new video extension of this task, called video panoptic segmentation.
ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation
We name this joint task as Depth-aware Video Panoptic Segmentation, and propose a new evaluation metric along with two derived datasets for it, which will be made available to the public.
STEP: Segmenting and Tracking Every Pixel
The task of assigning semantic classes and track identities to every pixel in a video is called video panoptic segmentation.
PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentation
harboryuan/polyphonicformer
•
•
The Depth-aware Video Panoptic Segmentation (DVPS) is a new challenging vision problem that aims to predict panoptic segmentation and depth in a video simultaneously.
Large-Scale Video Panoptic Segmentation in the Wild: A Benchmark
vipseg-dataset/vipseg-dataset
•
•
CVPR 2022
In contrast, our large-scale VIdeo Panoptic Segmentation in the Wild (VIPSeg) dataset provides 3, 536 videos and 84, 750 frames with pixel-level panoptic annotations, covering a wide range of real-world scenarios and categories.
Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation
lxtgh/video-k-net
•
•
CVPR 2022
We hope this simple, yet effective method can serve as a new, flexible baseline in unified video segmentation design.
Waymo Open Dataset: Panoramic Video Panoptic Segmentation
We therefore present the Waymo Open Dataset: Panoramic Video Panoptic Segmentation Dataset, a large-scale dataset that offers high-quality panoptic segmentation labels for autonomous driving.
PVO: Panoptic Visual Odometry
We present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information.
Context-Aware Relative Object Queries To Unify Video Instance and Panoptic Segmentation
AnwesaChoudhuri/CAROQ
•
•
CVPR 2023
We evaluate the proposed approach across three challenging tasks: video instance segmentation, multi-object tracking and segmentation, and video panoptic segmentation.
