Search Results for author:
Found 70 papers, 34 papers with code
IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection
HSF applies Point-to-Grid and Grid-to-Region transformers to capture the multimodal scene context at different granularities.
DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection
Specifically, DMA builds a domain-mixing 3D instance bank for the teacher and student models during training, resulting in aligned data representation.
Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training
In this paper, we propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages.
Language-Guided 3D Object Detection in Point Cloud for Autonomous Driving
In this work, we propose a new multi-modal visual grounding task, termed LiDAR Grounding.
LWSIS: LiDAR-guided Weakly Supervised Instance Segmentation for Autonomous Driving
In this paper, we present a more artful framework, LiDAR-guided Weakly Supervised Instance Segmentation (LWSIS), which leverages the off-the-shelf 3D data, i. e., Point Cloud, together with the 3D boxes, as natural weak supervisions for training the 2D image instance segmentation models.
SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloud
However, these UDA solutions just yield unsatisfactory 3D detection results when there is a severe domain shift, e. g., from Waymo (64-beam) to nuScenes (32-beam).
Transformation-Equivariant 3D Object Detection for Autonomous Driving
However, it is difficult to apply such networks to 3D object detection in autonomous driving due to its large computation cost and slow reasoning speed.
Zero-shot point cloud segmentation by transferring geometric primitives
To this end, we propose a novel framework to learn the geometric primitives shared in seen and unseen categories' objects and employ a fine-grained alignment between language and the learned geometric primitives.
Vision-Centric BEV Perception: A Survey
In recent years, vision-centric Bird's Eye View (BEV) perception has garnered significant interest from both industry and academia due to its inherent advantages, such as providing an intuitive representation of the world and being conducive to data fusion.
Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds
In this paper, we propose to detect 3D objects by exploiting temporal information in multiple frames, i. e., the point cloud videos.
STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes
In addition, considering the property of sparse global distribution and density-varying local distribution of pedestrians, we further propose a novel method, Density-aware Hierarchical heatmap Aggregation (DHA), to enhance pedestrian perception in crowded scenes.
Towards 3D Scene Understanding by Referring Synthetic Models
Promising performance has been achieved for visual perception on the point cloud.
An Intelligent Self-driving Truck System For Highway Transportation
Recently, there have been many advances in autonomous driving society, attracting a lot of attention from academia and industry.
FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction
By training on FaceScape data, a novel algorithm is proposed to predict elaborate riggable 3D face models from a single image input.
Referring Self-supervised Learning on 3D Point Cloud
In this paper, we study a new problem named Referring Self-supervised Learning (RSL) on 3D scene understanding: Given the 3D synthetic models with labels and the unlabeled 3D real scene scans, our goal is to distinguish the identical semantic objects on an unseen scene according to the referring synthetic 3D models.
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
In this paper, we benchmark our model on these three tasks.
Detailed Avatar Recovery from Single Image
In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation.
Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks
In this paper, we present the first study of security issues of MSF-based perception in AD systems.
Semantic Distribution-aware Contrastive Adaptation for Semantic Segmentation
Specifically, we first design a pixel-wise contrastive loss by considering the correspondences between semantic distributions and pixel-wise representations from both domains.
Adaptive Surface Normal Constraint for Depth Estimation
We present a novel method for single image depth estimation using surface normal constraints.
LEAD: LiDAR Extender for Autonomous Driving
In this paper, we propose LEAD, i. e., LiDAR Extender for Autonomous Driving, to extend the MEMS LiDAR by coupled image w. r. t both FoV and range.
Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation
For autonomous driving, the dynamics and states of vehicle parts such as doors, the trunk, and the bonnet can provide meaningful semantic information and interaction states, which are essential to ensuring the safety of the self-driving vehicle.
Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses
In this paper, we propose a novel approach to convert given speech audio to a photo-realistic speaking video of a specific person, where the output video has synchronized, realistic, and expressive rich body dynamics.
DVI: Depth Guided Video Inpainting for Autonomous Driving
To get clear street-view and photo-realistic simulation in autonomous driving, we present an automatic video inpainting algorithm that can remove traffic agents from videos and synthesize missing regions with the guidance of depth/point cloud.
Ranked #1 on
Image Inpainting
on ApolloScape
PerMO: Perceiving More at Once from a Single Image for Autonomous Driving
We present a novel approach to detect, segment, and reconstruct complete textured 3D models of vehicles from a single image for autonomous driving.
AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points
Then we aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos.
ODE-CNN: Omnidirectional Depth Extension Networks
Omnidirectional 360{\deg} camera proliferates rapidly for autonomous robots since it significantly enhances the perception ability by widening the field of view(FoV).
SparseFusion: Dynamic Human Avatar Modeling from Sparse RGBD Images
First, based on a generative human template, for every two frames having sufficient overlap, an initial pairwise alignment is performed; It is followed by a global non-rigid registration procedure, in which partial results from RGBD frames are collected into a unified 3D shape, under the guidance of correspondences from the pairwise alignment; Finally, the texture map of the reconstructed human model is optimized to deliver a clear and spatially consistent texture.
Channel Attention based Iterative Residual Learning for Depth Map Super-Resolution
Second, we propose a new framework for real-world DSR, which consists of four modules : 1) An iterative residual learning module with deep supervision to learn effective high-frequency components of depth maps in a coarse-to-fine manner; 2) A channel attention strategy to enhance channels with abundant high-frequency components; 3) A multi-stage fusion module to effectively re-exploit the results in the coarse-to-fine process; and 4) A depth refinement module to improve the depth map by TGV regularization and input loss.
CVPR 2019 WAD Challenge on Trajectory Prediction and 3D Perception
This paper reviews the CVPR 2019 challenge on Autonomous Driving.
LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attention
In this paper, we propose an end-to-end online 3D video object detector that operates on point cloud sequences.
FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction
In this paper, we present a large-scale detailed 3D face dataset, FaceScape, and propose a novel algorithm that is able to predict elaborate riggable 3D face models from a single image input.
A Unified Object Motion and Affinity Model for Online Multi-Object Tracking
In this paper, we propose a novel MOT framework that unifies object motion and affinity model into a single network, named UMA, in order to learn a compact feature that is discriminative for both object motion and affinity measure.
Learning 2D-3D Correspondences To Solve The Blind Perspective-n-Point Problem
Conventional absolute camera pose via a Perspective-n-Point (PnP) solver often assumes that the correspondences between 2D image pixels and 3D points are given.
Gated Path Selection Network for Semantic Segmentation
Semantic segmentation is a challenging task that needs to handle large scale variations, deformations and different viewpoints.
Domain-invariant Stereo Matching Networks
State-of-the-art stereo matching networks have difficulties in generalizing to new unseen environments due to significant domain differences, such as color, illumination, contrast, and texture.
AutoRemover: Automatic Object Removal for Autonomous Driving Videos
To deal with shadows, we build up an autonomous driving shadow dataset and design a deep neural network to detect shadows automatically.
CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion
In this paper, we propose CSPN++, which further improves its effectiveness and efficiency by learning adaptive convolutional kernel sizes and the number of iterations for the propagation, thus the context and computational resources needed at each pixel could be dynamically assigned upon requests.
Learning Resilient Behaviors for Navigation Under Uncertainty
Deep reinforcement learning has great potential to acquire complex, adaptive behaviors for autonomous agents automatically.
Improved Techniques for Training Adaptive Deep Networks
Adaptive inference is a promising technique to improve the computational efficiency of deep models at test time.
IoU Loss for 2D/3D Object Detection
In 2D/3D object detection task, Intersection-over-Union (IoU) has been widely employed as an evaluation metric to evaluate the performance of different detectors in the testing stage.
Adversarial Objects Against LiDAR-Based Autonomous Driving Systems
Deep neural networks (DNNs) are found to be vulnerable against adversarial examples, which are carefully crafted inputs with a small magnitude of perturbation aiming to induce arbitrarily incorrect predictions.
Detailed Human Shape Estimation from a Single Image by Hierarchical Mesh Deformation
This paper presents a novel framework to recover detailed human body shapes from a single image.
GA-Net: Guided Aggregation Net for End-to-end Stereo Matching
In the stereo matching task, matching cost aggregation is crucial in both traditional methods and deep neural network models in order to accurately estimate disparities.
Ranked #4 on
Stereo Depth Estimation
on Spring
AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms
Our augmented approach combines the flexibility in a virtual environment (e. g., vehicle movements) with the richness of the real world to allow effective simulation of anywhere on earth.
Human Pose Estimation with Spatial Contextual Information
With this principle, we present two conceptually simple and yet computational efficient modules, namely Cascade Prediction Fusion (CPF) and Pose Graph Neural Network (PGNN), to exploit underlying contextual information.
Ranked #10 on
Pose Estimation
on MPII Human Pose
ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving
Specifically, we first segment each car with a pre-trained Mask R-CNN, and then regress towards its 3D pose and shape based on a deformable 3D car model with or without using semantic keypoints.
Part-level Car Parsing and Reconstruction from Single Street View
As it is labor-intensive to annotate semantic parts on real street views, we propose a specific approach to implicitly transfer part features from synthesized images to real street views.
Augmented LiDAR Simulator for Autonomous Driving
Instead, we can simply deploy a vehicle with a LiDAR scanner to sweep the street of interests to obtain the background point cloud, based on which annotated point cloud can be automatically generated.
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).
Ranked #1 on
Trajectory Prediction
on Apolloscape Trajectory
Learning Depth with Convolutional Spatial Propagation Network
In this paper, we propose a simple yet effective convolutional spatial propagation network (CSPN) to learn the affinity matrix for various depth estimation tasks.
Safe Navigation with Human Instructions in Complex Scenes
In this paper, we present a robotic navigation algorithm with natural language interfaces, which enables a robot to safely walk through a changing environment with moving persons by following human instructions such as "go to the restaurant and keep away from people".
RealPoint3D: Point Cloud Generation from a Single Image with Complex Background
Then, the image together with the retrieved shape model is fed into the proposed network to generate the fine-grained 3D point cloud.
Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network
Depth estimation from a single image is a fundamental problem in computer vision.
A Network Structure to Explicitly Reduce Confusion Errors in Semantic Segmentation
Confusing classes that are ubiquitous in real world often degrade performance for many vision related applications like object detection, classification, and segmentation.
Identity Preserving Face Completion for Large Ocular Region Occlusion
Different from the state-of-the-art face inpainting methods that have no control over the synthesized content and can only handle frontal face pose, our approach can faithfully recover the missing content under various head poses while preserving the identity.
DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map
The uniqueness of our design is a sensor fusion scheme which integrates camera videos, motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robustness and efficiency of the system.
Learning Warped Guidance for Blind Face Restoration
For better recovery of fine facial details, we modify the problem setting by taking both the degraded observation and a high-quality guided image of the same identity as input to our guided face restoration network (GFRNet).
Ranked #1 on
Image Super-Resolution
on WebFace - 8x upscaling
View Extrapolation of Human Body from a Single Image
We study how to synthesize novel views of human body from a single image.
The ApolloScape Open Dataset for Autonomous Driving and its Application
In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving.
Detailed Surface Geometry and Albedo Recovery from RGB-D Video Under Natural Illumination
In this paper we present a novel approach for depth map enhancement from an RGB-D video sequence.
Mask-off: Synthesizing Face Images in the Presence of Head-mounted Displays
The main purpose of this paper is to synthesize realistic face images without occlusions based on the images captured by these cameras.
Interactive Visual Hull Refinement for Specular and Transparent Object Surface Reconstruction
We discovered that these internal contours, which are results of convex parts on an object's surface, can lead to a tighter fit than the original visual hull.
3D Reconstruction in the Presence of Glasses by Acoustic and Stereo Fusion
We present a novel sensor fusion algorithm that first segments the depth map into different categories such as opaque/transparent/infinity (e. g., too far to measure) and then updates the depth map based on the segmentation outcome.
Simultaneous Time-of-Flight Sensing and Photometric Stereo With a Single ToF Sensor
We present a novel system which incorporates photometric stereo with the Time-of-Flight depth sensor.
Quality Dynamic Human Body Modeling Using a Single Low-cost Depth Camera
In this paper we present a novel autonomous pipeline to build a personalized parametric model (pose-driven avatar) using a single depth sensor.
Data-driven Flower Petal Modeling with Botany Priors
Each segmented petal is then fitted with a scale-invariant morphable petal shape model, which is constructed from individually scanned exemplar petals.
Real-time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera
In this paper we present a novel real-time algorithm for simultaneous pose and shape estimation for articulated objects, such as human beings and animals.
Video Enhancement of People Wearing Polarized Glasses: Darkening Reversal and Reflection Reduction
With the wide-spread of consumer 3D-TV technology, stereoscopic videoconferencing systems are emerging.
