Search Results for author: Xinge Zhu

Found 56 papers, 24 papers with code

A Unified Framework for Human-centric Point Cloud Video Understanding

no code implementations CVPR 2024 Yiteng Xu, Kecheng Ye, Xiao Han, Yiming Ren, Xinge Zhu, Yuexin Ma

Human-centric Point Cloud Video Understanding (PVU) is an emerging field focused on extracting and interpreting human-related features from sequences of human point clouds, further advancing downstream human-centric tasks and applications.

3D Pose Estimation Action Recognition +4

LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment

1 code implementation20 Mar 2024 Peishan Cong, Ziyi Wang, Zhiyang Dou, Yiming Ren, Wei Yin, Kai Cheng, Yujing Sun, Xiaoxiao Long, Xinge Zhu, Yuexin Ma

Language-guided scene-aware human motion generation has great significance for entertainment and robotics.

OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries

1 code implementation6 Dec 2023 Yuhang Lu, Xinge Zhu, Tai Wang, Yuexin Ma

Occupancy prediction has increasingly garnered attention in recent years for its fine-grained understanding of 3D scenes.

Learning to Adapt SAM for Segmenting Cross-domain Point Clouds

no code implementations13 Oct 2023 Xidong Peng, Runnan Chen, Feng Qiao, Lingdong Kong, Youquan Liu, Tai Wang, Xinge Zhu, Yuexin Ma

Unsupervised domain adaptation (UDA) in 3D segmentation tasks presents a formidable challenge, primarily stemming from the sparse and unordered nature of point cloud data.

General Knowledge Image Segmentation +4

Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training

no code implementations29 Sep 2023 Runnan Chen, Xinge Zhu, Nenglun Chen, Dawei Wang, Wei Li, Yuexin Ma, Ruigang Yang, Tongliang Liu, Wenping Wang

In this paper, we propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages.

3D Semantic Segmentation Object

Cross-modal and Cross-domain Knowledge Transfer for Label-free 3D Segmentation

no code implementations19 Sep 2023 Jingyu Zhang, Huitong Yang, Dai-Jie Wu, Jacky Keung, Xuesong Li, Xinge Zhu, Yuexin Ma

Current state-of-the-art point cloud-based perception methods usually rely on large-scale labeled data, which requires expensive manual annotations.

Domain Adaptation Semantic Segmentation +1

Human-centric Scene Understanding for 3D Large-scale Scenarios

1 code implementation ICCV 2023 Yiteng Xu, Peishan Cong, Yichen Yao, Runnan Chen, Yuenan Hou, Xinge Zhu, Xuming He, Jingyi Yu, Yuexin Ma

Human-centric scene understanding is significant for real-world applications, but it is extremely challenging due to the existence of diverse human poses and actions, complex human-environment interactions, severe occlusions in crowds, etc.

Action Recognition Scene Understanding +1

ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Clouds

no code implementations25 Apr 2023 Xiangze Jia, Hui Zhou, Xinge Zhu, Yandong Guo, Ji Zhang, Yuexin Ma

In this paper, we propose a novel self-supervised motion estimator for LiDAR-based autonomous driving via BEV representation.

Autonomous Driving Contrastive Learning +2

WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language

no code implementations12 Apr 2023 Zhenxiang Lin, Xidong Peng, Peishan Cong, Yuenan Hou, Xinge Zhu, Sibei Yang, Yuexin Ma

We introduce the task of 3D visual grounding in large-scale dynamic scenes based on natural linguistic descriptions and online captured multi-modal visual data, including 2D images and 3D LiDAR point clouds.

3D visual grounding Autonomous Driving +1

One Training for Multiple Deployments: Polar-based Adaptive BEV Perception for Autonomous Driving

no code implementations2 Apr 2023 Huitong Yang, Xuyang Bai, Xinge Zhu, Yuexin Ma

Current on-board chips usually have different computing power, which means multiple training processes are needed for adapting the same learning-based algorithm to different chips, costing huge computing resources.

Autonomous Driving

SCPNet: Semantic Scene Completion on Point Cloud

1 code implementation CVPR 2023 Zhaoyang Xia, Youquan Liu, Xin Li, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao

We propose a simple yet effective label rectification strategy, which uses off-the-shelf panoptic segmentation labels to remove the traces of dynamic objects in completion labels, greatly improving the performance of deep models especially for those moving objects.

3D Semantic Scene Completion Knowledge Distillation +3

Rethinking Range View Representation for LiDAR Segmentation

no code implementations ICCV 2023 Lingdong Kong, Youquan Liu, Runnan Chen, Yuexin Ma, Xinge Zhu, Yikang Li, Yuenan Hou, Yu Qiao, Ziwei Liu

We show that, for the first time, a range view method is able to surpass the point, voxel, and multi-view fusion counterparts in the competing LiDAR semantic and panoptic segmentation benchmarks, i. e., SemanticKITTI, nuScenes, and ScribbleKITTI.

3D Semantic Segmentation Autonomous Driving +4

CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

1 code implementation CVPR 2023 Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, Wenping Wang

For the first time, our pre-trained network achieves annotation-free 3D semantic segmentation with 20. 8% and 25. 08% mIoU on nuScenes and ScanNet, respectively.

3D Semantic Segmentation Contrastive Learning +4

CL3D: Unsupervised Domain Adaptation for Cross-LiDAR 3D Detection

1 code implementation1 Dec 2022 Xidong Peng, Xinge Zhu, Yuexin Ma

Second, we present Temporal Motion Alignment module to utilize motion features in sequential frames of data to match two domains.

Pseudo Label Unsupervised Domain Adaptation

Zero-shot point cloud segmentation by transferring geometric primitives

no code implementations18 Oct 2022 Runnan Chen, Xinge Zhu, Nenglun Chen, Wei Li, Yuexin Ma, Ruigang Yang, Wenping Wang

To this end, we propose a novel framework to learn the geometric primitives shared in seen and unseen categories' objects and employ a fine-grained alignment between language and the learned geometric primitives.

Point Cloud Segmentation Semantic Segmentation

Rethinking Trajectory Prediction via "Team Game"

no code implementations17 Oct 2022 Zikai Wei, Xinge Zhu, Bo Dai, Dahua Lin

To accurately predict trajectories in multi-agent settings, e. g. team games, it is important to effectively model the interactions among agents.

Trajectory Prediction

GANet: Goal Area Network for Motion Forecasting

1 code implementation20 Sep 2022 Mingkun Wang, Xinge Zhu, Changqian Yu, Wei Li, Yuexin Ma, Ruochun Jin, Xiaoguang Ren, Dongchun Ren, Mingxu Wang, Wenjing Yang

In view of this, we propose a new goal area-based framework, named Goal Area Network (GANet), for motion forecasting, which models goal areas rather than exact goal coordinates as preconditions for trajectory prediction, performing more robustly and accurately.

Motion Forecasting Trajectory Prediction

Vision-Centric BEV Perception: A Survey

1 code implementation4 Aug 2022 Yuexin Ma, Tai Wang, Xuyang Bai, Huitong Yang, Yuenan Hou, Yaming Wang, Yu Qiao, Ruigang Yang, Dinesh Manocha, Xinge Zhu

In recent years, vision-centric Bird's Eye View (BEV) perception has garnered significant interest from both industry and academia due to its inherent advantages, such as providing an intuitive representation of the world and being conducive to data fusion.

MV-FCOS3D++: Multi-View Camera-Only 4D Object Detection with Pretrained Monocular Backbones

1 code implementation26 Jul 2022 Tai Wang, Qing Lian, Chenming Zhu, Xinge Zhu, Wenwei Zhang

In this technical report, we present our solution, dubbed MV-FCOS3D++, for the Camera-Only 3D Detection track in Waymo Open Dataset Challenge 2022.

object-detection Object Detection +1

Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation

no code implementations CVPR 2022 Yuenan Hou, Xinge Zhu, Yuexin Ma, Chen Change Loy, Yikang Li

This article addresses the problem of distilling knowledge from a large teacher model to a slim student network for LiDAR semantic segmentation.

Ranked #8 on LIDAR Semantic Segmentation on nuScenes (val mIoU metric)

3D Semantic Segmentation Knowledge Distillation +1

STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes

1 code implementation CVPR 2022 Peishan Cong, Xinge Zhu, Feng Qiao, Yiming Ren, Xidong Peng, Yuenan Hou, Lan Xu, Ruigang Yang, Dinesh Manocha, Yuexin Ma

In addition, considering the property of sparse global distribution and density-varying local distribution of pedestrians, we further propose a novel method, Density-aware Hierarchical heatmap Aggregation (DHA), to enhance pedestrian perception in crowded scenes.

Pedestrian Detection Sensor Fusion

TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers

1 code implementation CVPR 2022 Xuyang Bai, Zeyu Hu, Xinge Zhu, Qingqiu Huang, Yilun Chen, Hongbo Fu, Chiew-Lan Tai

The attention mechanism of the transformer enables our model to adaptively determine where and what information should be taken from the image, leading to a robust and effective fusion strategy.

3D Object Detection Autonomous Driving +3

Self-supervised Point Cloud Completion on Real Traffic Scenes via Scene-concerned Bottom-up Mechanism

no code implementations20 Mar 2022 Yiming Ren, Peishan Cong, Xinge Zhu, Yuexin Ma

In this paper, we propose a self-supervised point cloud completion method (TraPCC) for vehicles in real traffic scenes without any complete data.

Point Cloud Completion

LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network

1 code implementation14 Mar 2022 Fangzhou Hong, Hui Zhou, Xinge Zhu, Hongsheng Li, Ziwei Liu

In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.

4D Panoptic Segmentation Autonomous Driving +3

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

no code implementations9 Dec 2021 Xiao Song, Guorun Yang, Xinge Zhu, Hui Zhou, Yuexin Ma, Zhe Wang, Jianping Shi

Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline.

Domain Adaptation Stereo Matching

Referring Self-supervised Learning on 3D Point Cloud

no code implementations29 Sep 2021 Runnan Chen, Xinge Zhu, Nenglun Chen, Dawei Wang, Wei Li, Yuexin Ma, Ruigang Yang, Wenping Wang

In this paper, we study a new problem named Referring Self-supervised Learning (RSL) on 3D scene understanding: Given the 3D synthetic models with labels and the unlabeled 3D real scene scans, our goal is to distinguish the identical semantic objects on an unseen scene according to the referring synthetic 3D models.

Scene Understanding Self-Supervised Learning

SIDE: Center-based Stereo 3D Detector with Structure-aware Instance Depth Estimation

no code implementations22 Aug 2021 Xidong Peng, Xinge Zhu, Tai Wang, Yuexin Ma

Due to the information sparsity of local cost volume, we further introduce match reweighting and structure-aware attention, to make the depth information more concentrated.

Depth Estimation

LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation

no code implementations17 Aug 2021 Lin Zhao, Hui Zhou, Xinge Zhu, Xiao Song, Hongsheng Li, Wenbing Tao

However, two major issues of the fusion between camera and LiDAR hinder its performance, \ie, how to effectively fuse these two modalities and how to precisely align them (suffering from the weak spatiotemporal synchronization problem).

Autonomous Driving LIDAR Semantic Segmentation +1

Probabilistic and Geometric Depth: Detecting Objects in Perspective

1 code implementation29 Jul 2021 Tai Wang, Xinge Zhu, Jiangmiao Pang, Dahua Lin

As the preliminary depth estimation of each instance is usually inaccurate in this ill-posed setting, we incorporate a probabilistic representation to capture the uncertainty.

Attribute Depth Estimation +2

FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection

8 code implementations22 Apr 2021 Tai Wang, Xinge Zhu, Jiangmiao Pang, Dahua Lin

In this paper, we study this problem with a practice built on a fully convolutional single-stage detector and propose a general framework FCOS3D.

Autonomous Driving Monocular 3D Object Detection +2

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

no code implementations26 Mar 2021 Peishan Cong, Xinge Zhu, Yuexin Ma

A thorough and holistic scene understanding is crucial for autonomous vehicles, where LiDAR semantic segmentation plays an indispensable role.

Autonomous Vehicles LIDAR Semantic Segmentation +2

SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks

no code implementations19 Oct 2020 Yan Xu, Zhaoyang Huang, Kwan-Yee Lin, Xinge Zhu, Jianping Shi, Hujun Bao, Guofeng Zhang, Hongsheng Li

To suit our network to self-supervised learning, we design several novel loss functions that utilize the inherent properties of LiDAR point clouds.

Self-Supervised Learning

Channel-wise Alignment for Adaptive Object Detection

no code implementations7 Sep 2020 Hang Yang, Shan Jiang, Xinge Zhu, Mingyang Huang, Zhiqiang Shen, Chunxiao Liu, Jianping Shi

Existing methods on this task usually draw attention on the high-level alignment based on the whole image or object of interest, which naturally, cannot fully utilize the fine-grained channel information.

Instance Segmentation Object +3

Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation

3 code implementations4 Aug 2020 Hui Zhou, Xinge Zhu, Xiao Song, Yuexin Ma, Zhe Wang, Hongsheng Li, Dahua Lin

A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space.

3D Semantic Segmentation LIDAR Semantic Segmentation

Tensor Low-Rank Reconstruction for Semantic Segmentation

no code implementations ECCV 2020 Wanli Chen, Xinge Zhu, Ruoqi Sun, Junjun He, Ruiyu Li, Xiaoyong Shen, Bei Yu

Then we use these rank-1 tensors to recover the high-rank context features through our proposed tensor reconstruction module (TRM).

Semantic Segmentation

AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching

no code implementations CVPR 2021 Xiao Song, Guorun Yang, Xinge Zhu, Hui Zhou, Zhe Wang, Jianping Shi

Compared to previous methods for adaptive stereo matching, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline.

Domain Adaptation Stereo Matching

Reconfigurable Voxels: A New Representation for LiDAR-Based Point Clouds

no code implementations6 Apr 2020 Tai Wang, Xinge Zhu, Dahua Lin

LiDAR is an important method for autonomous driving systems to sense the environment.

Autonomous Driving

SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds

1 code implementation6 Apr 2020 Xinge Zhu, Yuexin Ma, Tai Wang, Yan Xu, Jianping Shi, Dahua Lin

Multi-class 3D object detection aims to localize and classify objects of multiple categories from point clouds.

3D Object Detection object-detection

Adversarial Attacks on Monocular Depth Estimation

no code implementations23 Mar 2020 Ziqi Zhang, Xinge Zhu, Yingwei Li, Xiangqun Chen, Yao Guo

In order to understand the impact of adversarial attacks on depth estimation, we first define a taxonomy of different attack scenarios for depth estimation, including non-targeted attacks, targeted attacks and universal attacks.

Autonomous Driving Monocular Depth Estimation +3

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

no code implementations ICCV 2019 Yan Xu, Xinge Zhu, Jianping Shi, Guofeng Zhang, Hujun Bao, Hongsheng Li

Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises.

Autonomous Driving Decoder +1

TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents

1 code implementation6 Nov 2018 Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, Dinesh Manocha

To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).

Autonomous Vehicles Navigate +2

Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation

no code implementations ECCV 2018 Xinge Zhu, Hui Zhou, Ceyuan Yang, Jianping Shi, Dahua Lin

Due to the expensive and time-consuming annotations (e. g., segmentation) for real-world images, recent works in computer vision resort to synthetic data.

Domain Adaptation Segmentation +1

Generative Adversarial Frontal View to Bird View Synthesis

no code implementations1 Aug 2018 Xinge Zhu, Zhichao Yin, Jianping Shi, Hongsheng Li, Dahua Lin

Due to the large gap and severe deformation between the frontal view and bird view, generating a bird view image from a single frontal view is challenging.

Bird View Synthesis Homography Estimation +1

Pose Guided Human Video Generation

no code implementations ECCV 2018 Ceyuan Yang, Zhe Wang, Xinge Zhu, Chen Huang, Jianping Shi, Dahua Lin

Human pose, on the other hand, can represent motion patterns intrinsically and interpretably, and impose the geometric constraints regardless of appearance.

Generative Adversarial Network motion prediction +1

Cannot find the paper you are looking for? You can Submit a new open access paper.