Search Results for author:
Found 68 papers, 28 papers with code
A Unified Framework for Human-centric Point Cloud Video Understanding
Human-centric Point Cloud Video Understanding (PVU) is an emerging field focused on extracting and interpreting human-related features from sequences of human point clouds, further advancing downstream human-centric tasks and applications.
RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method
Comprehensive capturing of human motions requires both accurate captures of complex poses and precise localization of the human within scenes.
Gaze-guided Hand-Object Interaction Synthesis: Benchmark and Method
Here, the object motion diffusion model generates sequences of object motions based on gaze conditions, while the hand motion diffusion model produces hand motions based on the generated object motion.
LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment
Language-guided scene-aware human motion generation has great significance for entertainment and robotics.
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image
We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes, e. g., depth and normals, from single images.
HUNTER: Unsupervised Human-centric 3D Detection via Transferring Knowledge from Synthetic Instances to Real Scenes
Human-centric 3D scene understanding has recently drawn increasing attention, driven by its critical impact on robotics.
LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment
For human-centric large-scale scenes, fine-grained modeling for 3D human global pose and shape is significant for scene understanding and can benefit many real-world applications.
GaussianPro: 3D Gaussian Splatting with Progressive Propagation
The advent of 3D Gaussian Splatting (3DGS) has recently brought about a revolution in the field of neural rendering, facilitating high-quality renderings at real-time speed.
RealDex: Towards Human-like Grasping for Robotic Dexterous Hand
In this paper, we introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns, enriched by multi-view and multimodal visual data.
Extreme Two-View Geometry From Object Poses with Diffusion Models
Human has an incredible ability to effortlessly perceive the viewpoint difference between two images containing the same object, even when the viewpoint change is astonishingly vast with no co-visible regions in the images.
HybridGait: A Benchmark for Spatial-Temporal Cloth-Changing Gait Recognition with Hybrid Explorations
Existing gait recognition benchmarks mostly include minor clothing variations in the laboratory environments, but lack persistent changes in appearance over time and space.
OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queries
Occupancy prediction has increasingly garnered attention in recent years for its fine-grained understanding of 3D scenes.
GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
In this paper, we present GaussianShader, a novel method that applies a simplified shading function on 3D Gaussians to enhance the neural rendering in scenes with reflective surfaces while preserving the training and rendering efficiency.
UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving
Multi-camera setups find widespread use across various applications, such as autonomous driving, as they greatly expand sensing capabilities.
Wonder3D: Single Image to 3D using Cross-Domain Diffusion
In this work, we introduce Wonder3D, a novel method for efficiently generating high-fidelity textured meshes from single-view images. Recent methods based on Score Distillation Sampling (SDS) have shown the potential to recover 3D geometry from 2D diffusion priors, but they typically suffer from time-consuming per-shape optimization and inconsistent geometry.
Learning to Adapt SAM for Segmenting Cross-domain Point Clouds
Unsupervised domain adaptation (UDA) in 3D segmentation tasks presents a formidable challenge, primarily stemming from the sparse and unordered nature of point cloud data.
Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training
In this paper, we propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages.
Cross-modal and Cross-domain Knowledge Transfer for Label-free 3D Segmentation
Current state-of-the-art point cloud-based perception methods usually rely on large-scale labeled data, which requires expensive manual annotations.
UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase
Besides, we construct the OpenPCSeg codebase, which is the largest and most comprehensive outdoor LiDAR segmentation codebase.
Ranked #2 on
3D Semantic Segmentation
on SemanticKITTI
(using extra training data)
Human-centric Scene Understanding for 3D Large-scale Scenarios
Human-centric scene understanding is significant for real-world applications, but it is extremely challenging due to the existence of diverse human poses and actions, complex human-environment interactions, severe occlusions in crowds, etc.
See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data
They typically align visual features with semantic features obtained from word embedding by the supervision of seen classes' annotations.
Towards Label-free Scene Understanding by Vision Foundation Models
For nuImages and nuScenes datasets, the performance is 22. 1\% and 26. 8\% with improvements of 3. 5\% and 6. 0\%, respectively.
ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Clouds
In this paper, we propose a novel self-supervised motion estimator for LiDAR-based autonomous driving via BEV representation.
WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language
We introduce the task of 3D visual grounding in large-scale dynamic scenes based on natural linguistic descriptions and online captured multi-modal visual data, including 2D images and 3D LiDAR point clouds.
One Training for Multiple Deployments: Polar-based Adaptive BEV Perception for Autonomous Driving
Current on-board chips usually have different computing power, which means multiple training processes are needed for adapting the same learning-based algorithm to different chips, costing huge computing resources.
CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactions
The core of this dataset is a blending optimization process, which corrects for the pose as it drifts and is affected by the magnetic conditions.
SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments
We present SLOPER4D, a novel scene-aware dataset collected in large urban environments to facilitate the research of global human pose estimation (GHPE) with human-scene interaction in the wild.
SCPNet: Semantic Scene Completion on Point Cloud
We propose a simple yet effective label rectification strategy, which uses off-the-shelf panoptic segmentation labels to remove the traces of dynamic objects in completion labels, greatly improving the performance of deep models especially for those moving objects.
Ranked #1 on
3D Semantic Scene Completion
on SemanticKITTI
Rethinking Range View Representation for LiDAR Segmentation
We show that, for the first time, a range view method is able to surpass the point, voxel, and multi-view fusion counterparts in the competing LiDAR semantic and panoptic segmentation benchmarks, i. e., SemanticKITTI, nuScenes, and ScribbleKITTI.
Ranked #4 on
3D Semantic Segmentation
on SemanticKITTI
IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation
This paper presents an inverse kinematic optimization layer (IKOL) for 3D human pose and shape estimation that leverages the strength of both optimization- and regression-based methods within an end-to-end framework.
Ranked #24 on
3D Human Pose Estimation
on 3DPW
CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
For the first time, our pre-trained network achieves annotation-free 3D semantic segmentation with 20. 8% and 25. 08% mIoU on nuScenes and ScanNet, respectively.
CL3D: Unsupervised Domain Adaptation for Cross-LiDAR 3D Detection
Second, we present Temporal Motion Alignment module to utilize motion features in sequential frames of data to match two domains.
Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDAR
Motivated by this, we propose a monocular camera and single LiDAR-based method for 3D multi-person pose estimation in large-scale scenes, which is easy to deploy and insensitive to light.
LiCamGait: Gait Recognition in the Wild by Using LiDAR and Camera Multi-modal Visual Sensors
LiDAR can capture accurate depth information in large-scale scenarios without the effect of light conditions, and the captured point cloud contains gait-related 3D geometric properties and dynamic motion characteristics.
Monocular BEV Perception of Road Scenes via Front-to-Top View Projection
We propose a front-to-top view projection (FTVP) module, which takes the constraint of cycle consistency between views into account and makes full use of their correlation to strengthen the view transformation and scene understanding.
Zero-shot point cloud segmentation by transferring geometric primitives
To this end, we propose a novel framework to learn the geometric primitives shared in seen and unseen categories' objects and employ a fine-grained alignment between language and the learned geometric primitives.
GANet: Goal Area Network for Motion Forecasting
In view of this, we propose a new goal area-based framework, named Goal Area Network (GANet), for motion forecasting, which models goal areas rather than exact goal coordinates as preconditions for trajectory prediction, performing more robustly and accurately.
Ranked #15 on
Motion Forecasting
on Argoverse CVPR 2020
Vision-Centric BEV Perception: A Survey
In recent years, vision-centric Bird's Eye View (BEV) perception has garnered significant interest from both industry and academia due to its inherent advantages, such as providing an intuitive representation of the world and being conducive to data fusion.
Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation
This article addresses the problem of distilling knowledge from a large teacher model to a slim student network for LiDAR semantic segmentation.
Ranked #8 on
LIDAR Semantic Segmentation
on nuScenes
(val mIoU metric)
LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors
We propose a multi-sensor fusion method for capturing challenging 3D human motions with accurate consecutive local poses and global trajectories in large-scale scenarios, only using single LiDAR and 4 IMUs, which are set up conveniently and worn lightly.
STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes
In addition, considering the property of sparse global distribution and density-varying local distribution of pedestrians, we further propose a novel method, Density-aware Hierarchical heatmap Aggregation (DHA), to enhance pedestrian perception in crowded scenes.
LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Clouds
Quantitative and qualitative experiments show that our method outperforms the techniques based only on RGB images.
Ranked #3 on
3D Human Pose Estimation
on SLOPER4D
(using extra training data)
Towards 3D Scene Understanding by Referring Synthetic Models
Promising performance has been achieved for visual perception on the point cloud.
Self-supervised Point Cloud Completion on Real Traffic Scenes via Scene-concerned Bottom-up Mechanism
In this paper, we propose a self-supervised point cloud completion method (TraPCC) for vehicles in real traffic scenes without any complete data.
HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR
We propose Human-centered 4D Scene Capture (HSC4D) to accurately and efficiently create a dynamic digital world, containing large-scale indoor-outdoor scenes, diverse human motions, and rich interactions between humans and environments.
NIMBLE: A Non-rigid Hand Model with Bones and Muscles
Emerging Metaverse applications demand reliable, accurate, and photorealistic reproductions of human hands to perform sophisticated operations as if in the physical world.
AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach
Compared to previous methods, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline.
Referring Self-supervised Learning on 3D Point Cloud
In this paper, we study a new problem named Referring Self-supervised Learning (RSL) on 3D scene understanding: Given the 3D synthetic models with labels and the unlabeled 3D real scene scans, our goal is to distinguish the identical semantic objects on an unseen scene according to the referring synthetic 3D models.
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
In this paper, we benchmark our model on these three tasks.
SIDE: Center-based Stereo 3D Detector with Structure-aware Instance Depth Estimation
Due to the information sparsity of local cost volume, we further introduce match reweighting and structure-aware attention, to make the depth information more concentrated.
Structure-Aware Long Short-Term Memory Network for 3D Cephalometric Landmark Detection
Detecting 3D landmarks on cone-beam computed tomography (CBCT) is crucial to assessing and quantifying the anatomical abnormalities in 3D cephalometric analysis.
Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-View Transformation
Furthermore, our model runs at 35 FPS on a single GPU, which is efficient and applicable for real-time panorama HD map reconstruction.
Autonomous Driving
Monocular Cross-View Road Scene Parsing(Road)
+2
Semi-supervised Anatomical Landmark Detection via Shape-regulated Self-training
The global shape constraint is the inherent property of anatomical landmarks that provides valuable guidance for more consistent pseudo labelling of the unlabeled data, which is ignored in the previously semi-supervised methods.
SportsCap: Monocular 3D Human Motion Capture and Fine-grained Understanding in Challenging Sports Videos
In this paper, we propose SportsCap -- the first approach for simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input.
Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation
A thorough and holistic scene understanding is crucial for autonomous vehicles, where LiDAR semantic segmentation plays an indispensable role.
ChallenCap: Monocular 3D Capture of Challenging Human Performances using Multi-Modal References
We propose a hybrid motion inference stage with a generation network, which utilizes a temporal encoder-decoder to extract the motion details from the pair-wise sparse-view reference, as well as a motion discriminator to utilize the unpaired marker-based references to extract specific challenging motion characteristics in a data-driven manner.
Category Disentangled Context: Turning Category-irrelevant Features Into Treasures
Deep neural networks have achieved great success in computer vision, thanks to their ability in extracting category-relevant semantic features.
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited.
Ranked #2 on
3D Semantic Segmentation
on ScribbleKITTI
Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space.
Ranked #11 on
LIDAR Semantic Segmentation
on nuScenes
AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points
Then we aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos.
CVPR 2019 WAD Challenge on Trajectory Prediction and 3D Perception
This paper reviews the CVPR 2019 challenge on Autonomous Driving.
SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds
Multi-class 3D object detection aims to localize and classify objects of multiple categories from point clouds.
Decision Propagation Networks for Image Classification
High-level (e. g., semantic) features encoded in the latter layers of convolutional neural networks are extensively exploited for image classification, leaving low-level (e. g., color) features in the early layers underexplored.
Cephalometric Landmark Detection by Attentive Feature Pyramid Fusion and Regression-Voting
Marking anatomical landmarks in cephalometric radiography is a critical operation in cephalometric analysis.
Cephalometric Landmark Detection by AttentiveFeature Pyramid Fusion and Regression-Voting
Marking anatomical landmarks in cephalometric radiography is a critical operation in cephalometric analysis.
AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms
Our augmented approach combines the flexibility in a virtual environment (e. g., vehicle movements) with the richness of the real world to allow effective simulation of anywhere on earth.
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).
Ranked #1 on
Trajectory Prediction
on Apolloscape Trajectory
Efficient Reciprocal Collision Avoidance between Heterogeneous Agents Using CTMAT
We present a novel algorithm for reciprocal collision avoidance between heterogeneous agents of different shapes and sizes.
