Search Results for author:
Found 66 papers, 44 papers with code
LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes
LiDAR data pretraining offers a promising approach to leveraging large-scale, readily available datasets for enhanced data utilization.
Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives
Additionally, we highlight the potential of leveraging VLMs' awareness of corruptions to enhance their reliability, offering a roadmap for developing more trustworthy and interpretable decision-making systems in real-world autonomous driving contexts.
LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving
We introduce several key innovations: i) VFM-driven superpixel generation for detailed semantic representation, ii) a VFM-assisted contrastive learning strategy to align multimodal features, iii) superpoint temporal consistency to maintain stable representations across time, and iv) multi-source data pretraining to generalize across various LiDAR configurations.
OVGaussian: Generalizable 3D Gaussian Segmentation with Open Vocabularies
We first construct a large-scale 3D scene dataset based on 3DGS, dubbed \textbf{SegGaussian}, which provides detailed semantic and instance annotations for both Gaussian points and multi-view images.
3D Semantic Segmentation
Open Vocabulary Semantic Segmentation
+2
A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motions
Extracting physically plausible 3D human motion from videos is a critical task.
ChatDyn: Language-Driven Multi-Actor Dynamics Generation in Street Scenes
Generating realistic and interactive dynamics of traffic participants according to specific instruction is critical for street scene simulation.
SIMS: Simulating Human-Scene Interactions with Real World Script Planning
On the one hand, films and shows with stylish human locomotions or interactions with scenes are abundantly available on the internet, providing a rich source of data for script planning.
GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data
2) With the help of the pretrained human prior models, the Geometry Initialization-&-Sculpting pipeline is leveraged to recover high-quality 3D human geometry given a single image.
MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
Furthermore, MVPaint employs a UVR module to improve the texture quality in the UV space, which first performs a UV-space Super-Resolution, followed by a Spatial-aware Seam-Smoothing algorithm for revising spatial texturing discontinuities caused by UV unwrapping.
DynamicCity: Large-Scale LiDAR Generation from Dynamic Scenes
2) A DiT-based diffusion model for HexPlane generation.
AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation
2) For the ''how'' challenge, we introduce correspondence-aware motion optimization that constructs motion fields for both human and object models using the linear blend skinning function from SMPL-X.
Human-Object Interaction Detection
Human-Object Interaction Generation
+2
GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting
To achieve effective LoRA training, we introduce a reference-driven image editing approach that enables the simultaneous editing of multi-view images while ensuring consistency.
3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion
The increasing demand for high-quality 3D assets across various industries necessitates efficient and automated 3D content creation.
Point Transformer V3 Extreme: 1st Place Solution for 2024 Waymo Open Dataset Challenge in Semantic Segmentation
In this technical report, we detail our first-place solution for the 2024 Waymo Open Dataset Challenge's semantic segmentation track.
4D Contrastive Superflows are Dense 3D Representation Learners
In the realm of autonomous driving, accurate 3D perception is the foundation.
Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving
In this study, we present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms.
The RoboDrive Challenge: Drive Anytime Anywhere in Any Condition
In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles.
DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation
Therefore, we introduce a diffusion-based feed-forward framework to address these challenges with a single model.
Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving
Efficient data utilization is crucial for advancing 3D scene understanding in autonomous driving, where reliance on heavily human-annotated LiDAR point clouds challenges fully supervised methods.
Multi-Space Alignments Towards Universal LiDAR Segmentation
A unified and versatile LiDAR segmentation model with strong robustness and generalizability is desirable for safe autonomous driving perception.
Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption
Specifically, we first design a latent representation based on Vector Quantised-Variational AutoEncoder (VQ-VAE) to model human interaction.
Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding
Safety-critical 3D scene understanding tasks necessitate not only accurate but also confident predictions from 3D perception models.
3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping.
Exploiting Hierarchical Interactions for Protein Surface Learning
In this paper, we present a principled framework based on deep learning techniques, namely Hierarchical Chemical and Geometric Feature Interaction Network (HCGNet), for protein surface analysis by bridging chemical and geometric features with hierarchical interactions.
InsActor: Instruction-driven Physics-based Characters
Generating animation of physics-based characters with intuitive control has long been a desirable task with numerous applications.
DreamGaussian4D: Generative 4D Gaussian Splatting
Specifically, we propose an integral framework with two major modules: 1) Image-to-4D GS - we initially generate static GS with DreamGaussianHD, followed by HexPlane-based dynamic generation with Gaussian deformation; and 2) Video-to-Video Texture Refinement - we refine the generated UV-space texture maps and meanwhile enhance their temporal consistency by utilizing a pre-trained image-to-video diffusion model.
Digital Life Project: Autonomous 3D Characters with Social Intelligence
In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment.
Ranked #3 on
Motion Synthesis
on InterHuman
Large-Vocabulary 3D Diffusion Model with Transformer
To this end, we propose a novel triplane-based 3D-aware Diffusion model with TransFormer, DiffTF, for handling challenges via three aspects.
PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds
To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture.
Towards Real-World Visual Tracking with Temporal Contexts
To handle those problems, we propose a two-level framework (TCTrack) that can exploit temporal contexts efficiently.
HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
In this work, we propose HumanLiff, the first layer-wise 3D human generative model with a unified diffusion process.
Synthesizing Physically Plausible Human Motions in 3D Scenes
Experimental results demonstrate that our framework can synthesize physically plausible long-term human motions in complex 3D scenes.
Temporally-Adaptive Models for Efficient Video Understanding
Spatial convolutions are extensively used in numerous deep video models.
Ranked #4 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
Recent advancements in vision foundation models (VFMs) have opened up new possibilities for versatile and efficient visual perception.
Variational Relational Point Completion Network for Robust 3D Classification
Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details.
RoboBEV: Towards Robust Bird's Eye View Perception under Corruptions
Our experiments further demonstrate that pre-training and depth-free BEV transformation has the potential to enhance out-of-distribution robustness.
DiffMimic: Efficient Motion Mimicking with Differentiable Physics
Motion mimicking is a foundational task in physics-based character animation.
Generative Diffusion Prior for Unified Image Restoration and Enhancement
Besides, we devise hierarchical guidance and patch-based methods, enabling the GDP to generate images of arbitrary resolutions.
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
However, the performance on more diverse motions remains unsatisfactory.
Ranked #1 on
Motion Synthesis
on KIT Motion-Language
Robo3D: Towards Robust and Reliable 3D Perception against Corruptions
The robustness of 3D perception systems under natural corruptions from environments and sensors is pivotal for safety-critical applications.
SHERF: Generalizable Human NeRF from a Single Image
To this end, we propose a bank of 3D-aware hierarchical features, including global, point-level, and pixel-aligned features, to facilitate informative encoding.
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
Recent advances in modeling 3D objects mostly rely on synthetic datasets due to the lack of large-scale realscanned 3D databases.
EVA3D: Compositional 3D Human Generation from 2D Image Collections
At the core of EVA3D is a compositional human NeRF representation, which divides the human body into local parts.
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected.
Ranked #21 on
Motion Synthesis
on KIT Motion-Language
Exploring Point-BEV Fusion for 3D Point Cloud Object Tracking with Transformer
In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in consecutive frames given an object template.
TransPillars: Coarse-to-Fine Aggregation for Multi-Frame 3D Object Detection
3D object detection using point clouds has attracted increasing attention due to its wide applications in autonomous driving and robotics.
LaserMix for Semi-Supervised LiDAR Semantic Segmentation
Densely annotating LiDAR point clouds is costly, which restrains the scalability of fully-supervised learning methods.
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture
Our key-idea is to use real physical supervisions to train a target pose distribution prior for sampling-based motion control to capture physically plausible human motion.
Versatile Multi-Modal Pre-Training for Human-Centric Perception
To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency.
TCTrack: Temporal Contexts for Aerial Tracking
Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers.
Benchmarking and Analyzing Point Cloud Classification under Corruptions
3D perception, especially point cloud classification, has achieved substantial progress.
Ranked #7 on
Point Cloud Classification
on PointCloud-C
Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results
Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration.
Garment4D: Garment Reconstruction from Point Cloud Sequences
The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts.
A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion
The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud.
PTTR: Relational 3D Point Cloud Object Tracking with Transformer
In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in the current search point cloud given a template point cloud.
Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Robust Partial-to-Partial Point Cloud Registration in a Full Range
\textbf{3)} Based on a synergy of hierarchical graph networks and graphical modeling, we propose the {H}ierarchical {G}raphical {M}odeling (\textbf{HGM}) architecture to encode robust descriptors consisting of i) a unary term learned from {\textit{RI}} features; and ii) multiple smoothness terms encoded from neighboring point relations at different scales through our TPT modules.
Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
TAda! Temporally-Adaptive Convolutions for Video Understanding
This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos.
Ranked #67 on
Action Recognition
on Something-Something V2
(using extra training data)
CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing
Generating an interpretable and compact representation of 3D shapes from point clouds is an important and challenging problem.
Unsupervised 3D Shape Completion through GAN Inversion
In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time.
Variational Relational Point Completion Network
In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds.
Ranked #1 on
Point Cloud Completion
on Completion3D
Robust 6D Object Pose Estimation by Learning RGB-D Features
Accurate 6D object pose estimation is fundamental to robotic manipulation and grasping.
PointAtrousGraph: Deep Hierarchical Encoder-Decoder with Point Atrous Convolution for Unorganized 3D Points
Motivated by the success of encoding multi-scale contextual information for image analysis, we propose our PointAtrousGraph (PAG) - a deep permutation-invariant hierarchical encoder-decoder for efficiently exploiting multi-scale edge features in point clouds.
