OpenSUN3D: 1st Workshop Challenge on Open-Vocabulary 3D Scene Understanding

no code implementations • 23 Feb 2024 • Francis Engelmann, Ayca Takmaz, Jonas Schult, Elisabetta Fedele, Johanna Wald, Songyou Peng, Xi Wang, Or Litany, Siyu Tang, Federico Tombari, Marc Pollefeys, Leonidas Guibas, Hongbo Tian, Chunjie Wang, Xiaosheng Yan, Bingwen Wang, Xuanyang Zhang, Xiao Liu, Phuc Nguyen, Khoi Nguyen, Anh Tran, Cuong Pham, Zhening Huang, Xiaoyang Wu, Xi Chen, Hengshuang Zhao, Lei Zhu, Joan Lasenby

This report provides an overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with ICCV 2023.

Scene Understanding

ControlRoom3D: Room Generation using Semantic Proxy Rooms

no code implementations • 8 Dec 2023 • Jonas Schult, Sam Tsai, Lukas Höllein, Bichen Wu, Jialiang Wang, Chih-Yao Ma, Kunpeng Li, Xiaofang Wang, Felix Wimbauer, Zijian He, Peizhao Zhang, Bastian Leibe, Peter Vajda, Ji Hou

Central to our approach is a user-defined 3D semantic proxy room that outlines a rough room layout based on semantic bounding boxes and a textual description of the overall room style.

Mask4Former: Mask Transformer for 4D Panoptic Segmentation

1 code implementation • 28 Sep 2023 • Kadir Yilmaz, Jonas Schult, Alexey Nekrasov, Bastian Leibe

With this intention, we propose Mask4Former for the challenging task of 4D panoptic segmentation of LiDAR point clouds.

4D Panoptic Segmentation Instance Segmentation +2

AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation

no code implementations • 1 Jun 2023 • Yuanwen Yue, Sabarinath Mahadevan, Jonas Schult, Francis Engelmann, Bastian Leibe, Konrad Schindler, Theodora Kontogianni

In an iterative process, the model assigns each data point to an object (or the background), while the user corrects errors in the resulting segmentation and feeds them back into the model.

Binary Classification Interactive Segmentation +2

Point2Vec for Self-Supervised Representation Learning on Point Clouds

1 code implementation • 29 Mar 2023 • Karim Abou Zeid, Jonas Schult, Alexander Hermans, Bastian Leibe

Recently, the self-supervised learning framework data2vec has shown inspiring performance for various modalities using a masked student-teacher approach.

3D Part Segmentation Few-Shot 3D Point Cloud Classification +3

3D Segmentation of Humans in Point Clouds with Synthetic Data

no code implementations • ICCV 2023 • Ayça Takmaz, Jonas Schult, Irem Kaftan, Mertcan Akçay, Bastian Leibe, Robert Sumner, Francis Engelmann, Siyu Tang

We address this challenge and propose a framework for generating training data of synthetic humans interacting with real 3D scenes.

Instance Segmentation Segmentation +2

Mask3D: Mask Transformer for 3D Semantic Instance Segmentation

1 code implementation • 6 Oct 2022 • Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, Bastian Leibe

Modern 3D semantic instance segmentation approaches predominantly rely on specialized voting mechanisms followed by carefully designed geometric clustering techniques.

3D Instance Segmentation 3D Semantic Instance Segmentation +1

Mix3D: Out-of-Context Data Augmentation for 3D Scenes

3 code implementations • 5 Oct 2021 • Alexey Nekrasov, Jonas Schult, Or Litany, Bastian Leibe, Francis Engelmann

Since scene context helps reasoning about object semantics, current works focus on models with large capacity and receptive fields that can fully capture the global context of an input 3D scene.

3D Semantic Segmentation

DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes

1 code implementation • CVPR 2020 • Jonas Schult, Francis Engelmann, Theodora Kontogianni, Bastian Leibe

That is, the convolutional kernel weights are mapped to the local surface of a given mesh.

3D Semantic Segmentation Scene Segmentation

Know What Your Neighbors Do: 3D Semantic Segmentation of Point Clouds

no code implementations • 2 Oct 2018 • Francis Engelmann, Theodora Kontogianni, Jonas Schult, Bastian Leibe

In this paper, we present a deep learning architecture which addresses the problem of 3D semantic segmentation of unstructured point clouds.

3D Semantic Segmentation Segmentation