Search Results for author:
Found 49 papers, 23 papers with code
DreamDrive: Generative 4D Scene Modeling from Street View Images
In this paper, we present DreamDrive, a 4D spatial-temporal scene generation approach that combines the merits of generation and reconstruction, to synthesize generalizable 4D driving scenes and dynamic driving videos with 3D consistency.
STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes
We present STORM, a spatio-temporal reconstruction model designed for reconstructing dynamic outdoor scenes from sparse observations.
LoRA3D: Low-Rank Self-Calibration of 3D Geometric Foundation Models
Emerging 3D geometric foundation models, such as DUSt3R, offer a promising approach for in-the-wild 3D vision tasks.
Extrapolated Urban View Synthesis Benchmark
Our results show that Gaussian Splatting is prone to overfitting to training views.
Large Spatial Model: End-to-end Unposed Images to Semantic 3D
To tackle the scarcity of labeled 3D semantic data and enable natural language-driven scene manipulation, we incorporate a pre-trained 2D language-based segmentation model into a 3D-consistent semantic feature field.
System-Level Analysis of Module Uncertainty Quantification in the Autonomy Pipeline
We present a novel perspective on the design, use, and role of uncertainty measures for learned modules in an autonomous system.
Distributed NeRF Learning for Collaborative Multi-Robot Perception
We show the effectiveness of our method through an extensive set of experiments on datasets containing challenging real-world scenes, achieving performance comparable to centralized mapping of the environment where data is sent to a central server for processing.
Promptable Closed-loop Traffic Simulation
Simulation stands as a cornerstone for safe and efficient autonomous driving development.
OmniRe: Omni Urban Scene Reconstruction
To that end, we propose a comprehensive 3DGS framework for driving scenes, named OmniRe, that allows for accurate, full-length reconstruction of diverse dynamic objects in a driving log.
Wolf: Captioning Everything with a World Summarization Framework
Finally, we establish a benchmark for video captioning and introduce a leaderboard, aiming to accelerate advancements in video understanding, captioning, and data alignment.
Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attention
Understanding road geometry is a critical component of the autonomous vehicle (AV) stack.
Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
The autonomous driving industry is increasingly adopting end-to-end learning from sensory inputs to minimize human biases in system design.
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
On a large set of challenging scenarios, we observe that simple methods with moderate compute requirements such as TransFuser can match recent large-scale end-to-end driving architectures such as UniAD.
DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features
Our method is a generalizable feedforward model that predicts a rich neural scene representation from sparse, single-frame multi-view camera inputs with limited view overlap, and is trained self-supervised with differentiable rendering to reconstruct RGB, depth, or feature images.
Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses
Leveraging this rich dataset, we further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes, such as crash types, severity and number of injuries, based on contextual and environmental factors.
Language-Image Models with 3D Understanding
Our experiments on outdoor benchmarks demonstrate that Cube-LLM significantly outperforms existing baselines by 21. 3 points of AP-BEV on the Talk2Car dataset for 3D grounded reasoning and 17. 7 points on the DriveLM dataset for complex reasoning about driving scenarios, respectively.
InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds
InstantSplat adopts a self-supervised framework that bridges the gap between 2D images and 3D representations using Gaussian Bundle Adjustment (GauBA) and can be optimized in an end-to-end manner.
Producing and Leveraging Online Map Uncertainty in Trajectory Prediction
High-definition (HD) maps have played an integral role in the development of modern autonomous vehicle (AV) stacks, albeit with high associated labeling and maintenance costs.
Parallelized Spatiotemporal Binding
While modern best practices advocate for scalable architectures that support long-range interactions, object-centric models are yet to fully embrace these architectures.
Driving Everywhere with Large Language Model Policy Adaptation
Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs).
PARA-Drive: Parallelized Architecture for Real-time Autonomous Driving
Recent works have proposed end-to-end autonomous vehicle (AV) architectures comprised of differentiable modules achieving state-of-the-art driving performance.
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision
We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes.
Reinforcement Learning with Human Feedback for Realistic Traffic Simulation
This works aims to address this by developing a framework that employs reinforcement learning with human preference (RLHF) to enhance the realism of existing traffic models.
trajdata: A Unified Interface to Multiple Human Trajectory Datasets
The field of trajectory forecasting has grown significantly in recent years, partially owing to the release of numerous large-scale, real-world human trajectory datasets for autonomous vehicles (AVs) and pedestrian motion tracking.
Language Conditioned Traffic Generation
In this work, we turn to language as a source of supervision for dynamic traffic scene generation.
Language-Guided Traffic Simulation via Scene-Level Diffusion
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development.
Partial-View Object View Synthesis via Filtered Inversion
At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.
DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles
To enable the joint optimization of AV stacks while retaining modularity, we present DiffStack, a differentiable and modular stack for prediction, planning, and control.
Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models
Reasoning with occluded traffic agents is a significant open challenge for planning for autonomous vehicles.
Robust and Controllable Object-Centric Learning through Energy-based Models
Humans are remarkably good at understanding and reasoning about complex visual scenes.
Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learning
Learning-based behavior prediction methods are increasingly being deployed in real-world autonomous systems, e. g., in fleets of self-driving vehicles, which are beginning to commercially operate in major cities across the world.
BITS: Bi-level Imitation for Traffic Simulation
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets and show that BITS achieves balanced traffic simulation performance in realism, diversity, and long-horizon stability.
ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning
In this work, we present ScePT, a policy planning-based trajectory prediction model that generates accurate, scene-consistent trajectory predictions suitable for autonomous system motion planning.
Whose Track Is It Anyway? Improving Robustness to Tracking Errors With Affinity-Based Trajectory Prediction
This is typically caused by the propagation of errors from tracking to prediction, such as noisy tracks, fragments, and identity switches.
MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation
Recently, there has been tremendous progress in developing each individual module of the standard perception-planning robot autonomy pipeline, including detection, tracking, prediction of other agents' trajectories, and ego-agent trajectory planning.
Propagating State Uncertainty Through Trajectory Forecasting
As a result, perceptual uncertainties are not propagated through forecasting and predictions are frequently overconfident.
Injecting Planning-Awareness into Prediction and Detection Evaluation
Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios entailing human-robot interaction such as autonomous driving.
Sample-Efficient Safety Assurances using Conformal Prediction
When deploying machine learning models in high-stakes robotics applications, the ability to detect unsafe situations is crucial.
Rethinking Trajectory Forecasting Evaluation
Forecasting the behavior of other agents is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios with human-robot interaction, such as autonomous driving.
Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty
Reasoning about the future behavior of other agents is critical to safe robot navigation.
Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions
To achieve seamless human-robot interactions, robots need to intimately reason about complex interaction dynamics and future human behaviors within their motion planning process.
Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoders
Discrete latent spaces in variational autoencoders have been shown to effectively capture the data distribution for many real-world problems such as natural language understanding, human intent prediction, and visual scene representation.
MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control
Reasoning about human motion is a core component of modern human-robot interactive systems.
Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction
This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure.
Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach
Human behavior prediction models enable robots to anticipate how humans may react to their actions, and hence are instrumental to devising safe and proactive robot planning algorithms.
Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data
Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation.
Ranked #2 on
Trajectory Prediction
on ETH
The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs
Developing safe human-robot interaction systems is a necessary step towards the widespread integration of autonomous agents in society.
BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning
Our Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance.
Generative Modeling of Multimodal Multi-Human Behavior
This work presents a methodology for modeling and predicting human behavior in settings with N humans interacting in highly multimodal scenarios (i. e. where there are many possible highly-distinct futures).
Robotics Human-Computer Interaction
