Search Results for author:
Found 66 papers, 19 papers with code
EMMA: End-to-End Multimodal Model for Autonomous Driving
We show that co-training EMMA with planner trajectories, object detection, and road graph tasks yields improvements across all three domains, highlighting EMMA's potential as a generalist model for autonomous driving applications.
PVTransformer: Point-to-Voxel Transformer for Scalable 3D Object Detection
3D object detectors for point clouds often rely on a pooling-based PointNet to encode sparse points into grid-like voxels or pillars.
MoST: Multi-modality Scene Tokenization for Motion Prediction
This symbolic representation is a high-level abstraction of the real world, which may render the motion prediction model vulnerable to perception errors (e. g., failures in detecting open-vocabulary obstacles) while missing salient information from the scene context (e. g., poor road conditions).
LEF: Late-to-Early Temporal Fusion for LiDAR 3D Object Detection
We propose a late-to-early recurrent feature fusion scheme for 3D object detection using temporal LiDAR point clouds.
Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving
Closed-set 3D perception models trained on only a pre-defined set of object categories can be inadequate for safety critical applications such as autonomous driving where new object types can be encountered after deployment.
3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels
We show that by training on a large training set from Waymo Open Dataset without any human annotated keypoints, we are able to achieve reasonable performance as compared to the fully supervised approach.
MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion
We present MotionDiffuser, a diffusion based representation for the joint distribution of future trajectories over multiple agents.
MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences
The MoDAR modality propagates object information from temporal contexts to a target frame, represented as a set of virtual points, one for each object from a waypoint on a forecasted trajectory.
The Waymo Open Sim Agents Challenge
Simulation with realistic, interactive agents represents a key task for autonomous vehicle software development.
WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting
To study the effect of these modular approaches, design new paradigms that mitigate these limitations, and accelerate the development of end-to-end motion forecasting models, we augment the Waymo Open Motion Dataset (WOMD) with large-scale, high-quality, diverse LiDAR data for the motion forecasting task.
GINA-3D: Learning to Generate Implicit Neural Assets in the Wild
Modeling the 3D world from sensor data for simulation is a scalable way of developing testing and validation environments for robotic learning problems such as autonomous driving.
Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
To our knowledge, this is the first application of a combined imitation and reinforcement learning approach in autonomous driving that utilizes large amounts of real-world human driving data.
JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving
We propose JFP, a Joint Future Prediction model that can learn to generate accurate and consistent multi-agent future trajectories.
HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Driving
Autonomous driving is an exciting new industry, posing important research questions.
NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors
Formulating single-view reconstruction as an image-conditioned 3D generation problem, we optimize the NeRF representations by minimizing a diffusion loss on its arbitrary view renderings with a pretrained image diffusion model under the input-view constraint.
LidarAugment: Searching for Scalable 3D LiDAR Data Augmentations
Data augmentations are important in training high-performance 3D object detectors for point clouds.
PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds
To alleviate the cost of hyperparameter tuning and iterative pseudo labeling, we develop a population-based data augmentation framework for 3D detection, named AutoPseudoAugment.
Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving
We demonstrate the first large-scale application of model-based generative adversarial imitation learning (MGAIL) to the task of dense urban self-driving.
CramNet: Camera-Radar Fusion with Ray-Constrained Cross-Attention for Robust 3D Object Detection
Fusing camera and radar data is challenging, however, as each of the sensors lacks information along a perpendicular axis, that is, depth is unknown to camera and elevation is unknown to radar.
Improving the Intra-class Long-tail in 3D Detection via Rare Example Mining
Continued improvements in deep learning architectures have steadily advanced the overall performance of 3D object detectors to levels on par with humans for certain tasks and datasets, where the overall performance is mostly driven by common examples.
LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds
Our method co-designs an efficient labeling process with semi/weakly supervised learning and is applicable to nearly any 3D semantic segmentation backbones.
Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving
Learning-based perception and prediction modules in modern autonomous driving systems typically rely on expensive human annotation and are designed to perceive only a handful of predefined object categories.
SWFormer: Sparse Window Transformer for 3D Object Detection in Point Clouds
3D object detection in point clouds is a core component for modern robotics and autonomous driving systems.
LidarNAS: Unifying and Searching Neural Architectures for 3D Point Clouds
Developing neural models that accurately understand objects in 3D point clouds is essential for the success of robotics and autonomous driving.
Waymo Open Dataset: Panoramic Video Panoptic Segmentation
We therefore present the Waymo Open Dataset: Panoramic Video Panoptic Segmentation Dataset, a large-scale dataset that offers high-quality panoptic segmentation labels for autonomous driving.
LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection
However, camera-only detectors have limited depth accuracy, which may cause otherwise reasonable predictions that suffer from such longitudinal localization errors to be treated as false positives.
Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking
Monocular image-based 3D perception has become an active research area in recent years owing to its applications in autonomous driving.
RIDDLE: Lidar Data Compression with Range Image Deep Delta Encoding
Lidars are depth measuring sensors widely used in autonomous driving and augmented reality.
StopNet: Scalable Trajectory and Occupancy Prediction for Urban Autonomous Driving
A whole-scene sparse input representation allows StopNet to scale to predicting trajectories for hundreds of road agents with reliable latency.
Multi-Class 3D Object Detection with Single-Class Supervision
While multi-class 3D detectors are needed in many robotics applications, training them with fully labeled datasets can be expensive in labeling cost.
Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulation
The beam search refines these policies on the fly by pruning branches that are unfavourably evaluated by a discriminator.
PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions
Cross-entropy loss and focal loss are the most common choices when training deep neural networks for classification problems.
Ranked #100 on
Image Classification
on ImageNet
Occupancy Flow Fields for Motion Forecasting in Autonomous Driving
We propose Occupancy Flow Fields, a new representation for motion forecasting of multiple agents, an important task in autonomous driving.
GradTail: Learning Long-Tailed Data Using Gradient-based Sample Weighting
We propose GradTail, an algorithm that uses gradients to improve model performance on the fly in the face of long-tailed training data distributions.
Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Driving
3D human pose estimation (HPE) in autonomous vehicles (AV) differs from other use cases in many factors, including the 3D resolution and range of data, absence of dense depth maps, failure modes for LiDAR, relative location between the camera and LiDAR, and a high bar for estimation accuracy.
Revisiting 3D Object Detection From an Egocentric Perspective
Given the insight that SDE would benefit from more accurate geometry descriptions, we propose to represent objects as amodal contours, specifically amodal star-shaped polygons, and devise a simple model, StarPoly, to predict such contours.
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction
Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving.
Ranked #16 on
Motion Forecasting
on Argoverse CVPR 2020
Lidar Range Image Compression with Deep Delta Encoding
However, most prior work focus on the generic point cloud representation, neglecting the spatial patterns of the points from lidar range images.
SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation
On the Waymo Open Dataset and KITTI, SPG improves 3D detection results of these two methods across all categories.
Ranked #6 on
3D Object Detection
on KITTI Cars Easy
HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps
To address this issue, we introduce a new challenging task to generate HD maps.
RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detection
These larger detection ranges require more efficient and accurate detection models.
To the Point: Efficient 3D Object Detection in the Range Image with Graph Convolution Kernels
3D object detection is vital for many robotics applications.
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
Offboard 3D Object Detection from Point Cloud Sequences
While current 3D object recognition research mostly focuses on the real-time, onboard scenario, there are many offboard use cases of perception that are largely under-explored, such as using machines to automatically generate high-quality 3D labels.
Large Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset
Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout
The vast majority of deep models use multiple gradient signals, typically corresponding to a sum of multiple loss terms, to update a shared set of trainable weights.
TNT: Target-driveN Trajectory Prediction
Our key insight is that for prediction within a moderate time horizon, the future modes can be effectively captured by a set of target states.
SoDA: Multi-Object Tracking with Soft Data Association
Robust multi-object tracking (MOT) is a prerequisite fora safe deployment of self-driving cars.
Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detection
This paper presents a novel 3D object detection framework that processes LiDAR data directly on its native representation: range images.
STINet: Spatio-Temporal-Interactive Network for Pedestrian Detection and Trajectory Prediction
For the Waymo Open Dataset, we achieve a bird-eyes-view (BEV) detection AP of 80. 73 and trajectory prediction average displacement error (ADE) of 33. 67cm for pedestrians, which establish the state-of-the-art for both tasks.
VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
Behavior prediction in dynamic, multi-agent systems is an important problem in the context of self-driving cars, due to the complex representations and interactions of road components, including moving agents (e. g. pedestrians and vehicles) and road context information (e. g. lanes, traffic lights).
SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Driving
In such scenarios, the ability to accurately simulate the vehicle sensors such as cameras, lidar or radar is essential.
Improving 3D Object Detection through Progressive Population Based Augmentation
Data augmentation has been widely adopted for object detection in 3D point clouds.
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
In an effort to help align the research community's contributions with real-world self-driving problems, we introduce a new large scale, high quality, diverse dataset.
End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds
In this paper, we aim to synergize the birds-eye view and the perspective view and propose a novel end-to-end multi-view fusion (MVF) algorithm, which can effectively learn to utilize the complementary information from both.
MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction
Predicting human behavior is a difficult and crucial task required for motion planning.
Ranked #2 on
Trajectory Prediction
on PAID
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
We present PointFusion, a generic 3D object detection method that leverages both image and 3D point cloud information.
3D Bounding Box Estimation Using Deep Learning and Geometry
In contrast to current techniques that only regress the 3D orientation of an object, our method first regresses relatively stable 3D object properties using a deep convolutional neural network and then combines these estimates with geometric constraints provided by a 2D object bounding box to produce a complete 3D bounding box.
Ranked #9 on
Vehicle Pose Estimation
on KITTI Cars Hard
SSD: Single Shot MultiBox Detector
Experimental results on the PASCAL VOC, MS COCO, and ILSVRC datasets confirm that SSD has comparable accuracy to methods that utilize an additional object proposal step and is much faster, while providing a unified framework for both training and inference.
Ranked #3 on
Object Detection
on PASCAL VOC 2012
Training Deep Neural Networks on Noisy Labels with Bootstrapping
On MNIST handwritten digits, we show that our model is robust to label corruption.
Self-informed neural network structure learning
We study the problem of large scale, multi-label visual recognition with a large number of possible classes.
Scalable, High-Quality Object Detection
Using the multi-scale convolutional MultiBox (MSC-MultiBox) approach, we substantially advance the state-of-the-art on the ILSVRC 2014 detection challenge data set, with $0. 5$ mAP for a single model and $0. 52$ mAP for an ensemble of two models.
Going Deeper with Convolutions
We propose a deep convolutional neural network architecture codenamed "Inception", which was responsible for setting the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC 2014).
Self-taught Object Localization with Deep Networks
This paper introduces self-taught object localization, a novel approach that leverages deep convolutional networks trained for whole-image recognition to localize objects in images without additional human supervision, i. e., without using any ground-truth bounding boxes for training.
Capturing Long-tail Distributions of Object Subcategories
We argue that object subcategories follow a long-tail distribution: a few subcategories are common, while many are rare.
Scalable Object Detection using Deep Neural Networks
Deep convolutional neural networks have recently achieved state-of-the-art performance on a number of image recognition benchmarks, including the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC-2012).
