Search Results for author:
Found 169 papers, 76 papers with code
Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning
FORTRESS outperforms on-the-fly prompting of slow reasoning models in safety classification accuracy on synthetic benchmarks and real-world ANYmal robot data, and further improves system safety and planning success in simulation and on quadrotor hardware for urban navigation.
Generative AI for Autonomous Driving: Frontiers and Opportunities
This survey delivers a comprehensive and critical synthesis of the emerging role of GenAI across the autonomous driving stack.
Vision Foundation Model Embedding-Based Semantic Anomaly Detection
This work explores semantic anomaly detection by leveraging the semantic priors of state-of-the-art vision foundation models, operating directly on the image.
Flight Validation of Learning-Based Trajectory Optimization for the Astrobee Free-Flyer
Although widely used in commercial and industrial robotics, trajectory optimization has seen limited use in space applications due to its high computational demands.
Describe Anything: Detailed Localized Image and Video Captioning
We introduce the Describe Anything Model (DAM), a model designed for detailed localized captioning (DLC).
Robo-taxi Fleet Coordination at Scale via Reinforcement Learning
Fleets of robo-taxis offering on-demand transportation services, commonly known as Autonomous Mobility-on-Demand (AMoD) systems, hold significant promise for societal benefits, such as reducing pollution, energy consumption, and urban congestion.
Data Scaling Laws for End-to-End Autonomous Driving
Autonomous vehicle (AV) stacks have traditionally relied on decomposed approaches, with separate modules handling perception, prediction, and planning.
Taming High-Dimensional Dynamics: Learning Optimal Projections onto Spectral Submanifolds
High-dimensional nonlinear systems pose considerable challenges for modeling and control across many domains, from fluid mechanics to advanced robotics.
Scaling Vision Pre-Training to 4K Resolution
We introduce PS3 that scales CLIP-style vision pre-training to 4K resolution with a near-constant cost.
Surprise Potential as a Measure of Interactivity in Driving Scenarios
Identifying interactive scenarios in real-world driving logs enables the curation of datasets that amplify critical signals and provide a more accurate assessment of an AV's performance.
DreamDrive: Generative 4D Scene Modeling from Street View Images
In this paper, we present DreamDrive, a 4D spatial-temporal scene generation approach that combines the merits of generation and reconstruction, to synthesize generalizable 4D driving scenes and dynamic driving videos with 3D consistency.
STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes
We present STORM, a spatio-temporal reconstruction model designed for reconstructing dynamic outdoor scenes from sparse observations.
LoRA3D: Low-Rank Self-Calibration of 3D Geometric Foundation Models
Emerging 3D geometric foundation models, such as DUSt3R, offer a promising approach for in-the-wild 3D vision tasks.
Extrapolated Urban View Synthesis Benchmark
Photorealistic simulators are essential for the training and evaluation of vision-centric autonomous vehicles (AVs).
Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models
Traffic simulation aims to learn a policy for traffic agents that, when unrolled in closed-loop, faithfully recovers the joint distribution of trajectories observed in the real world.
Training an Open-Vocabulary Monocular 3D Object Detection Model without 3D Data
However, training 3D models with labels directly derived from pseudo-LiDAR is inadequate due to imprecise boxes estimated from noisy point clouds and severely occluded objects.
Learning Multiple Initial Solutions to Optimization Problems
Sequentially solving similar optimization problems under strict runtime constraints is essential for many applications, such as robot control, autonomous driving, and portfolio management.
Transformer-based Model Predictive Control: Trajectory Optimization via Sequence Modeling
Model predictive control (MPC) has established itself as the primary methodology for constrained control, enabling general-purpose robot autonomy in diverse real-world scenarios.
Large Spatial Model: End-to-end Unposed Images to Semantic 3D
To tackle the scarcity of labeled 3D semantic data and enable natural language-driven scene manipulation, we incorporate a pre-trained 2D language-based segmentation model into a 3D-consistent semantic feature field.
Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusion
An important paradigm in 3D object detection is the use of multiple modalities to enhance accuracy in both normal and challenging conditions, particularly for long-tail scenarios.
System-Level Analysis of Module Uncertainty Quantification in the Autonomy Pipeline
We present a novel perspective on the design, use, and role of uncertainty measures for learned modules in an autonomous system.
Generalizable Spacecraft Trajectory Generation via Multimodal Learning with Transformers
Effective trajectory generation is essential for reliable on-board spacecraft autonomy.
LoRD: Adapting Differentiable Driving Policies to Distribution Shifts
Distribution shifts between operational domains can severely affect the performance of learned models in self-driving vehicles (SDVs).
Offline Hierarchical Reinforcement Learning via Inverse Optimization
In this work, we propose OHIO: a framework for offline reinforcement learning (RL) of hierarchical policies.
Towards Robust Spacecraft Trajectory Optimization via Transformers
Future multi-spacecraft missions require robust autonomous trajectory optimization capabilities to ensure safe and efficient rendezvous operations.
LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning
This paper presents an advanced mathematical problem-solving framework, LLaMA-Berry, for enhancing the mathematical reasoning ability of Large Language Models (LLMs).
Promptable Closed-loop Traffic Simulation
Simulation stands as a cornerstone for safe and efficient autonomous driving development.
OmniRe: Omni Urban Scene Reconstruction
We introduce OmniRe, a comprehensive system for efficiently creating high-fidelity digital twins of dynamic real-world scenes from on-device logs.
Fast Grid Emissions Sensitivities using Parallel Decentralized Implicit Differentiation
Marginal emissions rates -- the sensitivity of carbon emissions to electricity demand -- are important for evaluating the impact of emissions mitigation measures.
Space-LLaVA: a Vision-Language Model Adapted to Extraterrestrial Applications
Foundation Models (FMs), e. g., large language models, possess attributes of intelligence which offer promise to endow a robot with the contextual understanding necessary to navigate complex, unstructured tasks in the wild.
Diagnostic Runtime Monitoring with Martingales
Machine learning systems deployed in safety-critical robotics settings must be robust to distribution shifts.
Wolf: Captioning Everything with a World Summarization Framework
Finally, we establish a benchmark for video captioning and introduce a leaderboard, aiming to accelerate advancements in video understanding, captioning, and data alignment.
VILA$^2$: VILA Augmented VILA
In the self-augment step, the instruction-finetuned VLM recaptions its pretraining caption datasets and then retrains from scratch leveraging refined data.
Ranked #63 on
Visual Question Answering
on MM-Vet
Real-Time Anomaly Detection and Reactive Planning with Large Language Models
Foundation models, e. g., large language models (LLMs), trained on internet-scale data possess zero-shot generalization capabilities that make them a promising technology towards detecting and mitigating out-of-distribution failure modes of robotic systems.
Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attention
Understanding road geometry is a critical component of the autonomous vehicle (AV) stack.
Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
The autonomous driving industry is increasingly adopting end-to-end learning from sensory inputs to minimize human biases in system design.
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
On a large set of challenging scenarios, we observe that simple methods with moderate compute requirements such as TransFuser can match recent large-scale end-to-end driving architectures such as UniAD.
DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features
Our method is a generalizable feedforward model that predicts a rich neural scene representation from sparse, single-frame multi-view camera inputs with limited view overlap, and is trained self-supervised with differentiable rendering to reconstruct RGB, depth, or feature images.
Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses
Leveraging this rich dataset, we further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes, such as crash types, severity and number of injuries, based on contextual and environmental factors.
Memorize What Matters: Emergent Scene Decomposition from Multitraverse
Humans naturally retain memories of permanent elements, while ephemeral moments often slip through the cracks of memory.
DiffuBox: Refining 3D Object Detection with Point Diffusion
Ensuring robust 3D object detection and localization is crucial for many applications in robotics and autonomous driving.
RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shifts
Modern motion planners for autonomous driving frequently use imitation learning (IL) to draw from expert driving logs.
Language-Image Models with 3D Understanding
Our experiments on outdoor benchmarks demonstrate that Cube-LLM significantly outperforms existing baselines by 21. 3 points of AP-BEV on the Talk2Car dataset for 3D grounded reasoning and 17. 7 points on the DriveLM dataset for complex reasoning about driving scenarios, respectively.
InstantSplat: Sparse-view SfM-free Gaussian Splatting in Seconds
InstantSplat adopts a self-supervised framework that bridges the gap between 2D images and 3D representations using Gaussian Bundle Adjustment (GauBA) and can be optimized in an end-to-end manner.
Producing and Leveraging Online Map Uncertainty in Trajectory Prediction
High-definition (HD) maps have played an integral role in the development of modern autonomous vehicle (AV) stacks, albeit with high associated labeling and maintenance costs.
Credit vs. Discount-Based Congestion Pricing: A Comparison Study
Tolling, or congestion pricing, offers a promising traffic management policy for regulating congestion, but has also attracted criticism for placing outsized financial burdens on low-income users.
Q-SLAM: Quadric Representations for Monocular SLAM
In this paper, we reimagine volumetric representations through the lens of quadrics.
Mapping High-level Semantic Regions in Indoor Environments without Object Recognition
Robots require a semantic understanding of their surroundings to operate in an efficient and explainable way in human environments.
Contingency Planning Using Bi-level Markov Decision Processes for Space Missions
This work focuses on autonomous contingency planning for scientific missions by enabling rapid policy computation from any off-nominal point in the state space in the event of a delay or deviation from the nominal mission plan.
Parallelized Spatiotemporal Binding
While modern best practices advocate for scalable architectures that support long-range interactions, object-centric models are yet to fully embrace these architectures.
Catch Me If You Can: Combatting Fraud in Artificial Currency Based Government Benefits Programs
To address the problem of misreporting fraud in artificial currency based benefits programs, we introduce an audit mechanism that induces a two-stage game between an administrator and users.
Driving Everywhere with Large Language Model Policy Adaptation
Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs).
Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate
The student learners are trained by the main model and, in turn, provide feedback to help the main model capture more generalizable and imitable correlations.
PARA-Drive: Parallelized Architecture for Real-time Autonomous Driving
Recent works have proposed end-to-end autonomous vehicle (AV) architectures comprised of differentiable modules achieving state-of-the-art driving performance.
RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios
Simulation plays a crucial role in the development of autonomous vehicles (AVs) due to the potential risks associated with real-world testing.
Learning for CasADi: Data-driven Models in Numerical Optimization
While real-world problems are often challenging to analyze analytically, deep learning excels in modeling complex processes from data.
PAC-Bayes Generalization Certificates for Learned Inductive Conformal Prediction
To remedy this, recent work has proposed learning model and score function parameters using data to directly optimize the efficiency of the ICP prediction sets.
Dolphins: Multimodal Language Model for Driving
The quest for fully autonomous vehicles (AVs) capable of navigating complex real-world scenarios with human-like understanding and responsiveness.
Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latent
Adept traffic models are critical to both planning and closed-loop simulation for autonomous vehicles (AV), and key design objectives include accuracy, diverse multimodal behaviors, interpretability, and downstream compatibility.
A Language Agent for Autonomous Driving
Human-level driving is an ultimate goal of autonomous driving.
Real-time Control of Electric Autonomous Mobility-on-Demand Systems via Graph Reinforcement Learning
Operators of Electric Autonomous Mobility-on-Demand (E-AMoD) fleets need to make several real-time decisions such as matching available vehicles to ride requests, rebalancing idle vehicles to areas of high demand, and charging vehicles to ensure sufficient range.
Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps
We propose a novel framework to integrate SD maps into online map prediction and propose a Transformer-based encoder, SD Map Encoder Representations from transFormers, to leverage priors in SD maps for the lane-topology prediction task.
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision
We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes.
Interactive Joint Planning for Autonomous Vehicles
In particular, IJP jointly optimizes over the behavior of the ego and the surrounding agents and leverages deep-learned prediction models as prediction priors that the join trajectory optimization tries to stay close to.
Closing the Loop on Runtime Monitors with Fallback-Safe MPC
When we rely on deep-learned models for robotic perception, we must recognize that these models may behave unreliably on inputs dissimilar from the training data, compromising the closed-loop system's safety.
Robust Nonlinear Reduced-Order Model Predictive Control
Real-world systems are often characterized by high-dimensional nonlinear dynamics, making them challenging to control in real time.
Reinforcement Learning with Human Feedback for Realistic Traffic Simulation
This works aims to address this by developing a framework that employs reinforcement learning with human preference (RLHF) to enhance the realism of existing traffic models.
trajdata: A Unified Interface to Multiple Human Trajectory Datasets
The field of trajectory forecasting has grown significantly in recent years, partially owing to the release of numerous large-scale, real-world human trajectory datasets for autonomous vehicles (AVs) and pedestrian motion tracking.
Language Conditioned Traffic Generation
In this work, we turn to language as a source of supervision for dynamic traffic scene generation.
Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictors
Trajectory prediction modules are key enablers for safe and efficient planning of autonomous vehicles (AVs), particularly in highly interactive traffic scenarios.
Language-Guided Traffic Simulation via Scene-Level Diffusion
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development.
Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models
Recently, reward-conditioned reinforcement learning (RCRL) has gained popularity due to its simplicity, flexibility, and off-policy nature.
Graph Reinforcement Learning for Network Control via Bi-Level Optimization
Optimization problems over dynamic networks have been extensively studied and widely used in the past decades to formulate numerous real-world problems.
Partial-View Object View Synthesis via Filtered Inversion
At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.
Convex Hulls of Reachable Sets
We study the convex hulls of reachable sets of nonlinear systems with bounded disturbances and uncertain initial conditions.
Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation
Geometric uncertainty propagation, on the other, propagates the geometric constraints on the keypoints to the 6D object pose, leading to a Pose UnceRtainty SEt (PURSE) that guarantees coverage of the groundtruth pose with the same probability.
FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization
One is to regularize the frequency range of NeRF's inputs, while the other is to penalize the near-camera density fields.
Dynamic locational marginal emissions via implicit differentiation
The method is model agnostic; it can compute LMEs for any convex optimization-based dispatch model, including some of the complex dispatch models employed by system operators in real electricity systems.
Learning Control-Oriented Dynamical Structure from Data
Even for known nonlinear dynamical systems, feedback controller synthesis is a difficult problem that often requires leveraging the particular structure of the dynamics to induce a stable closed-loop system.
A System-Level View on Out-of-Distribution Data in Robotics
When testing conditions differ from those represented in training data, so-called out-of-distribution (OOD) inputs can mar the reliability of learned components in the modern robot autonomy stack.
Hybrid Multi-agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems
We consider the sequential decision-making problem of making proactive request assignment and rejection decisions for a profit-maximizing operator of an autonomous mobility on demand system.
DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles
To enable the joint optimization of AV stacks while retaining modularity, we present DiffStack, a differentiable and modular stack for prediction, planning, and control.
Adaptive Robust Model Predictive Control via Uncertainty Cancellation
We propose a learning-based robust predictive control algorithm that compensates for significant uncertainty in the dynamics for a class of discrete-time systems that are nominally linear with an additive nonlinear component.
Online Distribution Shift Detection via Recency Prediction
When deploying modern machine learning-enabled robotic systems in high-stakes applications, detecting distribution shift is critical.
Foundation Models for Semantic Novelty in Reinforcement Learning
Effectively exploring the environment is a key challenge in reinforcement learning (RL).
Guided Conditional Diffusion for Controllable Traffic Simulation
Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles.
Differentially Private Stochastic Convex Optimization for Network Routing Applications
In this paper, we present a differentially private algorithm for network routing problems.
Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models
Reasoning with occluded traffic agents is a significant open challenge for planning for autonomous vehicles.
Robust and Controllable Object-Centric Learning through Energy-based Models
Humans are remarkably good at understanding and reasoning about complex visual scenes.
Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learning
Learning-based behavior prediction methods are increasingly being deployed in real-world autonomous systems, e. g., in fleets of self-driving vehicles, which are beginning to commercially operate in major cities across the world.
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors.
Data Lifecycle Management in Evolving Input Distributions for Learning-based Aerospace Applications
Algorithms within this framework are evaluated based on (1) model performance throughout mission lifetime and (2) cumulative costs associated with labeling and model retraining.
BITS: Bi-level Imitation for Traffic Simulation
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets and show that BITS achieves balanced traffic simulation performance in realism, diversity, and long-horizon stability.
Robust Trajectory Prediction against Adversarial Attacks
We demonstrate that our method is able to improve the performance by 46% on adversarial data and at the cost of only 3% performance degradation on clean data, compared to the model trained with clean data.
ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning
In this work, we present ScePT, a policy planning-based trajectory prediction model that generates accurate, scene-consistent trajectory predictions suitable for autonomous system motion planning.
Second-Order Sensitivity Analysis for Bilevel Optimization
Many existing approaches to bilevel optimization employ first-order sensitivity analysis, based on the implicit function theorem (IFT), for the lower problem to derive a gradient of the lower problem solution with respect to its parameters; this IFT gradient is then used in a first-order optimization method for the upper problem.
Safe Reinforcement Learning Using Black-Box Reachability Analysis
Reinforcement learning (RL) is capable of sophisticated motion planning and control for robots in uncertain environments.
Control-oriented meta-learning
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Online Learning for Traffic Routing under Unknown Preferences
at each period, we show that our approach obtains an expected regret and road capacity violation of $O(\sqrt{T})$, where $T$ is the number of periods over which tolls are updated.
Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms
Our experiments, performed in simulation and the real world onboard a highly agile quadrotor platform, demonstrate the capabilities of the described system to run learned models with, previously infeasible, large modeling capacity using gradient-based online optimization MPC.
Motron: Multimodal Probabilistic Human Motion Forecasting
We present Motron, a multimodal, probabilistic, graph-structured model, that captures human's multimodality using probabilistic methods while being able to output deterministic maximum-likelihood motions and corresponding confidence values for each mode.
A Unified View of SDP-based Neural Network Verification through Completely Positive Programming
Verifying that input-output relationships of a neural network conform to prescribed operational specifications is a key enabler towards deploying these networks in safety-critical applications.
Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demand
Autonomous Mobility-on-Demand (AMoD) systems represent an attractive alternative to existing transportation paradigms, currently challenged by urbanization and increasing travel needs.
Matching with Transfers under Distributional Constraints
To this end, we first consider the setting when the number of institutions (e. g., firms in a labor market) is one and show that equilibrium arrangements exist irrespective of the nature of the constraint structure or the agents' preferences.
Data-Driven Chance Constrained Control using Kernel Distribution Embeddings
We present a data-driven algorithm for efficiently computing stochastic control policies for general joint chance constrained optimal control problems.
Whose Track Is It Anyway? Improving Robustness to Tracking Errors With Affinity-Based Trajectory Prediction
This is typically caused by the propagation of errors from tracking to prediction, such as noisy tracks, fragments, and identity switches.
A Simple and Efficient Sampling-based Algorithm for General Reachability Analysis
In this work, we analyze an efficient sampling-based algorithm for general-purpose reachability analysis, which remains a notoriously challenging problem with applications ranging from neural network verification to safety analysis of dynamical systems.
Personalized Federated Learning of Driver Prediction Models for Autonomous Driving
Autonomous vehicles (AVs) must interact with a diverse set of human drivers in heterogeneous geographic areas.
On the Problem of Reformulating Systems with Uncertain Dynamics as a Stochastic Differential Equation
We identify an issue in recent approaches to learning-based control that reformulate systems with uncertain dynamics using a stochastic differential equation.
MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation
Recently, there has been tremendous progress in developing each individual module of the standard perception-planning robot autonomy pipeline, including detection, tracking, prediction of other agents' trajectories, and ego-agent trajectory planning.
Coordinated Multi-Agent Pathfinding for Drones and Trucks over Road Networks
The second stage solves only for drones, by routing them over a composite of the road network and the transit network defined by truck paths from the first stage.
Propagating State Uncertainty Through Trajectory Forecasting
As a result, perceptual uncertainties are not propagated through forecasting and predictions are frequently overconfident.
Injecting Planning-Awareness into Prediction and Detection Evaluation
Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios entailing human-robot interaction such as autonomous driving.
Data Sharing and Compression for Cooperative Networked Control
Sharing forecasts of network timeseries data, such as cellular or electricity load patterns, can improve independent control applications ranging from traffic scheduling to power generation.
Sample-Efficient Safety Assurances using Conformal Prediction
When deploying machine learning models in high-stakes robotics applications, the ability to detect unsafe situations is crucial.
Tube-Certified Trajectory Tracking for Nonlinear Systems With Robust Control Contraction Metrics
This paper presents an approach towards guaranteed trajectory tracking for nonlinear control-affine systems subject to external disturbances based on robust control contraction metrics (CCM) that aims to minimize the $\mathcal L_\infty$ gain from the disturbances to nominal-actual trajectory deviations.
On the Interplay between Self-Driving Cars and Public Transportation
Cities worldwide struggle with overloaded transportation systems and their externalities.
Towards Data-Driven Synthesis of Autonomous Vehicle Safety Concepts
As safety-critical autonomous vehicles (AVs) will soon become pervasive in our society, a number of safety concepts for trusted AV deployment have recently been proposed throughout industry and academia.
Bayesian Embeddings for Few-Shot Open World Recognition
As autonomous decision-making agents move from narrow operating environments to unstructured worlds, learning systems must move from a closed-world formulation to an open-world and few-shot setting in which agents continuously learn new classes from small amounts of information.
Rethinking Trajectory Forecasting Evaluation
Forecasting the behavior of other agents is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios with human-robot interaction, such as autonomous driving.
Joint Optimization of Autonomous Electric Vehicle Fleet Operations and Charging Station Siting
Charging infrastructure is the coupling link between power and transportation networks, thus determining charging station siting is necessary for planning of power and transportation systems.
Analysis and Control of Autonomous Mobility-on-Demand Systems
We provide a comprehensive review of methods and tools to model and solve problems related to autonomous mobility-on-demand systems.
Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty
Reasoning about the future behavior of other agents is critical to safe robot navigation.
Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems
Autonomous mobility-on-demand (AMoD) systems represent a rapidly developing mode of transportation wherein travel requests are dynamically handled by a coordinated fleet of robotic, self-driving vehicles.
Adaptive Robust Model Predictive Control with Matched and Unmatched Uncertainty
We propose a learning-based robust predictive control algorithm that compensates for significant uncertainty in the dynamics for a class of discrete-time systems that are nominally linear with an additive nonlinear component.
Particle MPC for Uncertain and Learning-Based Control
As robotic systems move from highly structured environments to open worlds, incorporating uncertainty from dynamics learning or state estimation into the control pipeline is essential for robust performance.
Balancing Fairness and Efficiency in Traffic Routing via Interpolated Traffic Assignment
To address the inherent unfairness of SO routing, we study the ${\beta}$-fair SO problem whose goal is to minimize the total travel time while guaranteeing a ${\beta\geq 1}$ level of unfairness, which specifies the maximum possible ratio between the travel times of different users with shared origins and destinations.
Adaptive-Control-Oriented Meta-Learning for Nonlinear Systems
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks
In order to safely deploy Deep Neural Networks (DNNs) within the perception pipelines of real-time decision making systems, there is a need for safeguards that can detect out-of-training-distribution (OoD) inputs both efficiently and accurately.
Local Calibration: Metrics and Recalibration
In this work, we propose the local calibration error (LCE) to span the gap between average and individual reliability.
Risk-sensitive safety analysis using Conditional Value-at-Risk
In addition, we propose a second definition for risk-sensitive safe sets and provide a tractable method for their estimation without using a parameter-dependent upper bound.
Sampling Training Data for Continual Learning Between Robots and the Cloud
Today's robotic fleets are increasingly measuring high-volume video and LIDAR sensory streams, which can be mined for valuable training data, such as rare scenes of road construction sites, to steadily improve robotic perception models.
Linear Reduced Order Model Predictive Control
Model predictive controllers use dynamics models to solve constrained optimal control problems.
Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions
To achieve seamless human-robot interactions, robots need to intimately reason about complex interaction dynamics and future human behaviors within their motion planning process.
Task-relevant Representation Learning for Networked Robotic Perception
However, today's representations for sensory data are mostly designed for human, not robotic, perception and thus often waste precious compute or wireless network resources to transmit unimportant parts of a scene that are unnecessary for a high-level robotic task.
Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoders
Discrete latent spaces in variational autoencoders have been shown to effectively capture the data distribution for many real-world problems such as natural language understanding, human intent prediction, and visual scene representation.
CoCo: Learning Strategies for Online Mixed-Integer Control
Mixed-integer convex programming (MICP) is a popular modeling framework for solving discrete and combinatorial optimization problems arising in various settings.
MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control
Reasoning about human motion is a core component of modern human-robot interactive systems.
Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction
This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure.
Safe Active Dynamics Learning and Control: A Sequential Exploration-Exploitation Framework
In this work, we propose a practical and theoretically-justified approach to maintaining safety in the presence of dynamics uncertainty.
Co-Design to Enable User-Friendly Tools to Assess the Impact of Future Mobility Solutions
This requires tools to study such a coupling and co-design mobility systems in terms of different objectives.
Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach
Human behavior prediction models enable robots to anticipate how humans may react to their actions, and hence are instrumental to devising safe and proactive robot planning algorithms.
Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods
This paper presents a technique, named STLCG, to compute the quantitative semantics of Signal Temporal Logic (STL) formulas using computation graphs.
Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data
Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation.
Ranked #2 on
Trajectory Prediction
on ETH
Continuous Meta-Learning without Tasks
In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.
Efficient Large-Scale Multi-Drone Delivery Using Transit Networks
Our results show that the framework computes solutions typically within a few seconds on commodity hardware, and that drones travel up to $360 \%$ of their flight range with public transit.
Learning Stabilizable Nonlinear Dynamics with Contraction-Based Regularization
We propose a novel framework for learning stabilizable nonlinear dynamical systems for continuous control tasks in robotics.
High-Dimensional Optimization in Adaptive Random Subspaces
We propose a new randomized optimization method for high-dimensional problems which can be seen as a generalization of coordinate descent to random subspaces.
On the Interaction between Autonomous Mobility on Demand Systems and Power Distribution Networks -- An Optimal Power Flow Approach
Specifically, we extend previous results on an optimization-based modeling approach for electric AMoD systems to jointly control an electric AMoD fleet and a series of PDNs, and analyze the benefit of coordination under load balancing constraints.
Network Offloading Policies for Cloud Robotics: a Learning-based Approach
In this paper, we formulate a novel Robot Offloading Problem --- how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication?
Robust and Adaptive Planning under Model Uncertainty
The first, RAMCP-F, converges to an optimal risk-sensitive policy without having to rebuild the search tree as the underlying belief over models is perturbed.
The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs
Developing safe human-robot interaction systems is a necessary step towards the widespread integration of autonomous agents in society.
Risk-Sensitive Generative Adversarial Imitation Learning
We then derive two different versions of our RS-GAIL optimization problem that aim at matching the risk profiles of the agent and the expert w. r. t.
Learning Stabilizable Dynamical Systems via Control Contraction Metrics
We propose a novel framework for learning stabilizable nonlinear dynamical systems for continuous control tasks in robotics.
Meta-Learning Priors for Efficient Online Bayesian Regression
However, this approach suffers from two main drawbacks: (1) it is computationally inefficient, as computation scales poorly with the number of samples; and (2) it can be data inefficient, as encoding prior knowledge that can aid the model through the choice of kernel and associated hyperparameters is often challenging and unintuitive.
BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning
Our Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance.
Safe Motion Planning in Unknown Environments: Optimality Benchmarks and Tractable Policies
This paper addresses the problem of planning a safe (i. e., collision-free) trajectory from an initial state to a goal region when the obstacle space is a-priori unknown and is incrementally revealed online, e. g., through line-of-sight perception.
Generative Modeling of Multimodal Multi-Human Behavior
This work presents a methodology for modeling and predicting human behavior in settings with N humans interacting in highly multimodal scenarios (i. e. where there are many possible highly-distinct futures).
Robotics Human-Computer Interaction
Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods
The literature on Inverse Reinforcement Learning (IRL) typically assumes that humans take actions in order to minimize the expected value of a cost function, i. e., that humans are risk neutral.
How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics
We discuss general representation theorems that precisely characterize the class of metrics that satisfy these axioms (referred to as distortion risk metrics), and provide instantiations that can be used in applications.
Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction
This paper presents a method for constructing human-robot interaction policies in settings where multimodality, i. e., the possibility of multiple highly distinct futures, plays a critical role in decision making.
Data-Driven Model Predictive Control of Autonomous Mobility-on-Demand Systems
The goal of this paper is to present an end-to-end, data-driven framework to control Autonomous Mobility-on-Demand systems (AMoD, i. e. fleets of self-driving vehicles).
Robotics Multiagent Systems Systems and Control Applications
Learning Sampling Distributions for Robot Motion Planning
This paper proposes a methodology for non-uniform sampling, whereby a sampling distribution is learned from demonstrations, and then used to bias sampling.
On the interaction between Autonomous Mobility-on-Demand systems and the power network: models and coordination algorithms
We assess the performance of the mode by studying a hypothetical AMoD system in Dallas-Fort Worth and its impact on the Texas power network.
Systems and Control Multiagent Systems Robotics
Mixed Strategy for Constrained Stochastic Optimal Control
We found that the same result holds for stochastic optimal control problems with continuous state and action spaces. Furthermore, we show the randomization of control input can result in reduced cost when the optimization problem is nonconvex, and the cost reduction is equal to the duality gap.
Risk-Constrained Reinforcement Learning with Percentile Risk Criteria
In many sequential decision-making problems one is interested in minimizing an expected cumulative cost while taking into account \emph{risk}, i. e., increased awareness of events of small probability and high consequences.
Two Phase $Q-$learning for Bidding-based Vehicle Sharing
We consider one-way vehicle sharing systems where customers can rent a car at one station and drop it off at another.
Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach
Our first contribution is to show that a CVaR objective, besides capturing risk sensitivity, has an alternative interpretation as expected cost under worst-case modeling errors, for a given error budget.
Monte Carlo Motion Planning for Robot Trajectory Optimization Under Uncertainty
MCMP applies this CP estimation procedure to motion planning by iteratively (i) computing an (approximately) optimal path for the deterministic version of the problem (here, using the FMT* algorithm), (ii) computing the CP of this path, and (iii) inflating or deflating the obstacles by a common factor depending on whether the CP is higher or lower than a target value.
Robotics
