4D Panoptic Segmentation
7 papers with code • 1 benchmarks • 3 datasets
4D Panoptic Segmentation is a computer vision task that extends video panoptic segmentation to point cloud sequences. That is, given a point cloud sequence, the goal is to predict the semantic class of each point while consistently tracking object instances. Here, the points belonging to the same object instance should be assigned the same instance ID throughout the point cloud sequence. LSTQ metric is used to evaluate the performance of this task. Video credit: Mask4Former
Benchmarks
These leaderboards are used to track progress in 4D Panoptic Segmentation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
SemanticKITTI
4D-OR
MM-OR
Most implemented papers
4D Panoptic Scene Graph Generation
Jingkang50/OpenPSG
•
•
NeurIPS 2023
To facilitate research in this new area, we build a richly annotated PSG-4D dataset consisting of 3K RGB-D videos with a total of 1M frames, each of which is labeled with 4D panoptic segmentation masks as well as fine-grained, dynamic scene graphs.
4D Panoptic LiDAR Segmentation
mehmetaygun/4d-pls
•
•
CVPR 2021
In this paper, we propose 4D panoptic LiDAR segmentation to assign a semantic class and a temporally-consistent instance ID to a sequence of 3D points.
Contrastive Instance Association for 4D Panoptic Segmentation using Sequences of 3D LiDAR Scans
prbonn/contrastive_association
•
•
IRAL 2021
We propose a novel approach that builds on top of an arbitrary single-scan panoptic segmentation network and extends it to the temporal domain by associating instances across time.
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
hongfz16/DS-Net
•
•
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
4D-StOP: Panoptic Segmentation of 4D LiDAR using Spatio-temporal Object Proposal Generation and Aggregation
Our voting-based tracklet generation method followed by geometric feature-based aggregation generates significantly improved panoptic LiDAR segmentation quality when compared to modeling the entire 4D volume using Gaussian probability distributions.
Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences
Panoptic segmentation of 3D LiDAR scans allows us to semantically describe a vehicle’s environment by predicting semantic classes for each 3D point and to identify individual instances through different instance IDs.
Mask4Former: Mask Transformer for 4D Panoptic Segmentation
With this intention, we propose Mask4Former for the challenging task of 4D panoptic segmentation of LiDAR point clouds.
