Search Results for author:
Found 111 papers, 52 papers with code
Dolphins: Multimodal Language Model for Driving
The quest for fully autonomous vehicles (AVs) capable of navigating complex real-world scenarios with human-like understanding and responsiveness.
Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latent
Adept traffic models are critical to both planning and closed-loop simulation for autonomous vehicles (AV), and key design objectives include accuracy, diverse multimodal behaviors, interpretability, and downstream compatibility.
A Language Agent for Autonomous Driving
Our approach, termed Agent-Driver, transforms the traditional autonomous driving pipeline by introducing a versatile tool library accessible via function calls, a cognitive memory of common sense and experiential knowledge for decision-making, and a reasoning engine capable of chain-of-thought reasoning, task planning, motion planning, and self-reflection.
Real-time Control of Electric Autonomous Mobility-on-Demand Systems via Graph Reinforcement Learning
Operators of Electric Autonomous Mobility-on-Demand (E-AMoD) fleets need to make several real-time decisions such as matching available cars to ride requests, rebalancing idle cars to areas of high demand, and charging vehicles to ensure sufficient range.
Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps
We propose a novel framework to integrate SD maps into online map prediction and propose a Transformer-based encoder, SD Map Encoder Representations from transFormers, to leverage priors in SD maps for the lane-topology prediction task.
EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision
We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes.
Interactive Joint Planning for Autonomous Vehicles
In particular, IJP jointly optimizes over the behavior of the ego and the surrounding agents and leverages deep-learned prediction models as prediction priors that the join trajectory optimization tries to stay close to.
Closing the Loop on Runtime Monitors with Fallback-Safe MPC
When we rely on deep-learned models for robotic perception, we must recognize that these models may behave unreliably on inputs dissimilar from the training data, compromising the closed-loop system's safety.
Robust Nonlinear Reduced-Order Model Predictive Control
Real-world systems are often characterized by high-dimensional nonlinear dynamics, making them challenging to control in real time.
Reinforcement Learning with Human Feedback for Realistic Traffic Simulation
This works aims to address this by developing a framework that employs reinforcement learning with human preference (RLHF) to enhance the realism of existing traffic models.
Language Conditioned Traffic Generation
In this work, we turn to language as a source of supervision for dynamic traffic scene generation.
Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictors
Trajectory prediction modules are key enablers for safe and efficient planning of autonomous vehicles (AVs), particularly in highly interactive traffic scenarios.
Language-Guided Traffic Simulation via Scene-Level Diffusion
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development.
Bayesian Reparameterization of Reward-Conditioned Reinforcement Learning with Energy-based Models
Recently, reward-conditioned reinforcement learning (RCRL) has gained popularity due to its simplicity, flexibility, and off-policy nature.
Graph Reinforcement Learning for Network Control via Bi-Level Optimization
Optimization problems over dynamic networks have been extensively studied and widely used in the past decades to formulate numerous real-world problems.
Partial-View Object View Synthesis via Filtered Inversion
At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.
Exact Characterization of the Convex Hulls of Reachable Sets
We study the convex hulls of reachable sets of nonlinear systems with bounded disturbances.
Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation
Geometric uncertainty propagation, on the other, propagates the geometric constraints on the keypoints to the 6D object pose, leading to a Pose UnceRtainty SEt (PURSE) that guarantees coverage of the groundtruth pose with the same probability.
FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization
One is to regularize the frequency range of NeRF's inputs, while the other is to penalize the near-camera density fields.
Dynamic locational marginal emissions via implicit differentiation
The method is model agnostic; it can compute LMEs for any convex optimization-based dispatch model, including some of the complex dispatch models employed by system operators in real electricity systems.
Learning Control-Oriented Dynamical Structure from Data
Even for known nonlinear dynamical systems, feedback controller synthesis is a difficult problem that often requires leveraging the particular structure of the dynamics to induce a stable closed-loop system.
A System-Level View on Out-of-Distribution Data in Robotics
When testing conditions differ from those represented in training data, so-called out-of-distribution (OOD) inputs can mar the reliability of learned components in the modern robot autonomy stack.
Hybrid Multi-agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems
We consider the sequential decision-making problem of making proactive request assignment and rejection decisions for a profit-maximizing operator of an autonomous mobility on demand system.
DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles
To enable the joint optimization of AV stacks while retaining modularity, we present DiffStack, a differentiable and modular stack for prediction, planning, and control.
Adaptive Robust Model Predictive Control via Uncertainty Cancellation
We propose a learning-based robust predictive control algorithm that compensates for significant uncertainty in the dynamics for a class of discrete-time systems that are nominally linear with an additive nonlinear component.
Online Distribution Shift Detection via Recency Prediction
When deploying modern machine learning-enabled robotic systems in high-stakes applications, detecting distribution shift is critical.
Foundation Models for Semantic Novelty in Reinforcement Learning
Effectively exploring the environment is a key challenge in reinforcement learning (RL).
Guided Conditional Diffusion for Controllable Traffic Simulation
Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles.
Differentially Private Stochastic Convex Optimization for Network Routing Applications
In this paper, we present a differentially private algorithm for network routing problems.
Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models
Reasoning with occluded traffic agents is a significant open challenge for planning for autonomous vehicles.
Robust and Controllable Object-Centric Learning through Energy-based Models
Humans are remarkably good at understanding and reasoning about complex visual scenes.
Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learning
Learning-based behavior prediction methods are increasingly being deployed in real-world autonomous systems, e. g., in fleets of self-driving vehicles, which are beginning to commercially operate in major cities across the world.
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors.
Data Lifecycle Management in Evolving Input Distributions for Learning-based Aerospace Applications
Algorithms within this framework are evaluated based on (1) model performance throughout mission lifetime and (2) cumulative costs associated with labeling and model retraining.
BITS: Bi-level Imitation for Traffic Simulation
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets and show that BITS achieves balanced traffic simulation performance in realism, diversity, and long-horizon stability.
Robust Trajectory Prediction against Adversarial Attacks
We demonstrate that our method is able to improve the performance by 46% on adversarial data and at the cost of only 3% performance degradation on clean data, compared to the model trained with clean data.
ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning
In this work, we present ScePT, a policy planning-based trajectory prediction model that generates accurate, scene-consistent trajectory predictions suitable for autonomous system motion planning.
Second-Order Sensitivity Analysis for Bilevel Optimization
Many existing approaches to bilevel optimization employ first-order sensitivity analysis, based on the implicit function theorem (IFT), for the lower problem to derive a gradient of the lower problem solution with respect to its parameters; this IFT gradient is then used in a first-order optimization method for the upper problem.
Safe Reinforcement Learning Using Black-Box Reachability Analysis
Reinforcement learning (RL) is capable of sophisticated motion planning and control for robots in uncertain environments.
Control-oriented meta-learning
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Online Learning for Traffic Routing under Unknown Preferences
at each period, we show that our approach obtains an expected regret and road capacity violation of $O(\sqrt{T})$, where $T$ is the number of periods over which tolls are updated.
Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms
Our experiments, performed in simulation and the real world onboard a highly agile quadrotor platform, demonstrate the capabilities of the described system to run learned models with, previously infeasible, large modeling capacity using gradient-based online optimization MPC.
Motron: Multimodal Probabilistic Human Motion Forecasting
We present Motron, a multimodal, probabilistic, graph-structured model, that captures human's multimodality using probabilistic methods while being able to output deterministic maximum-likelihood motions and corresponding confidence values for each mode.
A Unified View of SDP-based Neural Network Verification through Completely Positive Programming
Verifying that input-output relationships of a neural network conform to prescribed operational specifications is a key enabler towards deploying these networks in safety-critical applications.
Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demand
Autonomous Mobility-on-Demand (AMoD) systems represent an attractive alternative to existing transportation paradigms, currently challenged by urbanization and increasing travel needs.
Matching with Transfers under Distributional Constraints
To this end, we first consider the setting when the number of institutions (e. g., firms in a labor market) is one and show that equilibrium arrangements exist irrespective of the nature of the constraint structure or the agents' preferences.
Data-Driven Chance Constrained Control using Kernel Distribution Embeddings
We present a data-driven algorithm for efficiently computing stochastic control policies for general joint chance constrained optimal control problems.
Whose Track Is It Anyway? Improving Robustness to Tracking Errors With Affinity-Based Trajectory Prediction
This is typically caused by the propagation of errors from tracking to prediction, such as noisy tracks, fragments, and identity switches.
A Simple and Efficient Sampling-based Algorithm for General Reachability Analysis
In this work, we analyze an efficient sampling-based algorithm for general-purpose reachability analysis, which remains a notoriously challenging problem with applications ranging from neural network verification to safety analysis of dynamical systems.
Personalized Federated Learning of Driver Prediction Models for Autonomous Driving
Autonomous vehicles (AVs) must interact with a diverse set of human drivers in heterogeneous geographic areas.
On the Problem of Reformulating Systems with Uncertain Dynamics as a Stochastic Differential Equation
We identify an issue in recent approaches to learning-based control that reformulate systems with uncertain dynamics using a stochastic differential equation.
MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation
Recently, there has been tremendous progress in developing each individual module of the standard perception-planning robot autonomy pipeline, including detection, tracking, prediction of other agents' trajectories, and ego-agent trajectory planning.
Coordinated Multi-Agent Pathfinding for Drones and Trucks over Road Networks
The second stage solves only for drones, by routing them over a composite of the road network and the transit network defined by truck paths from the first stage.
Propagating State Uncertainty Through Trajectory Forecasting
As a result, perceptual uncertainties are not propagated through forecasting and predictions are frequently overconfident.
Injecting Planning-Awareness into Prediction and Detection Evaluation
Detecting other agents and forecasting their behavior is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios entailing human-robot interaction such as autonomous driving.
Data Sharing and Compression for Cooperative Networked Control
Sharing forecasts of network timeseries data, such as cellular or electricity load patterns, can improve independent control applications ranging from traffic scheduling to power generation.
Sample-Efficient Safety Assurances using Conformal Prediction
When deploying machine learning models in high-stakes robotics applications, the ability to detect unsafe situations is crucial.
Tube-Certified Trajectory Tracking for Nonlinear Systems With Robust Control Contraction Metrics
This paper presents an approach towards guaranteed trajectory tracking for nonlinear control-affine systems subject to external disturbances based on robust control contraction metrics (CCM) that aims to minimize the $\mathcal L_\infty$ gain from the disturbances to nominal-actual trajectory deviations.
On the Interplay between Self-Driving Cars and Public Transportation
Cities worldwide struggle with overloaded transportation systems and their externalities, such as traffic congestion and emissions.
Towards Data-Driven Synthesis of Autonomous Vehicle Safety Concepts
As safety-critical autonomous vehicles (AVs) will soon become pervasive in our society, a number of safety concepts for trusted AV deployment have recently been proposed throughout industry and academia.
Bayesian Embeddings for Few-Shot Open World Recognition
As autonomous decision-making agents move from narrow operating environments to unstructured worlds, learning systems must move from a closed-world formulation to an open-world and few-shot setting in which agents continuously learn new classes from small amounts of information.
Rethinking Trajectory Forecasting Evaluation
Forecasting the behavior of other agents is an integral part of the modern robotic autonomy stack, especially in safety-critical scenarios with human-robot interaction, such as autonomous driving.
Joint Optimization of Autonomous Electric Vehicle Fleet Operations and Charging Station Siting
Charging infrastructure is the coupling link between power and transportation networks, thus determining charging station siting is necessary for planning of power and transportation systems.
Analysis and Control of Autonomous Mobility-on-Demand Systems
We provide a comprehensive review of methods and tools to model and solve problems related to autonomous mobility-on-demand systems.
Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty
Reasoning about the future behavior of other agents is critical to safe robot navigation.
Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems
Autonomous mobility-on-demand (AMoD) systems represent a rapidly developing mode of transportation wherein travel requests are dynamically handled by a coordinated fleet of robotic, self-driving vehicles.
Adaptive Robust Model Predictive Control with Matched and Unmatched Uncertainty
We propose a learning-based robust predictive control algorithm that compensates for significant uncertainty in the dynamics for a class of discrete-time systems that are nominally linear with an additive nonlinear component.
Particle MPC for Uncertain and Learning-Based Control
As robotic systems move from highly structured environments to open worlds, incorporating uncertainty from dynamics learning or state estimation into the control pipeline is essential for robust performance.
Balancing Fairness and Efficiency in Traffic Routing via Interpolated Traffic Assignment
To address the inherent unfairness of SO routing, we study the ${\beta}$-fair SO problem whose goal is to minimize the total travel time while guaranteeing a ${\beta\geq 1}$ level of unfairness, which specifies the maximum possible ratio between the travel times of different users with shared origins and destinations.
Adaptive-Control-Oriented Meta-Learning for Nonlinear Systems
Real-time adaptation is imperative to the control of robots operating in complex, dynamic environments.
Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks
In order to safely deploy Deep Neural Networks (DNNs) within the perception pipelines of real-time decision making systems, there is a need for safeguards that can detect out-of-training-distribution (OoD) inputs both efficiently and accurately.
Local Calibration: Metrics and Recalibration
In this work, we propose the local calibration error (LCE) to span the gap between average and individual reliability.
Risk-sensitive safety analysis using Conditional Value-at-Risk
In addition, we propose a second definition for risk-sensitive safe sets and provide a tractable method for their estimation without using a parameter-dependent upper bound.
Sampling Training Data for Continual Learning Between Robots and the Cloud
Today's robotic fleets are increasingly measuring high-volume video and LIDAR sensory streams, which can be mined for valuable training data, such as rare scenes of road construction sites, to steadily improve robotic perception models.
Linear Reduced Order Model Predictive Control
Model predictive controllers use dynamics models to solve constrained optimal control problems.
Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions
To achieve seamless human-robot interactions, robots need to intimately reason about complex interaction dynamics and future human behaviors within their motion planning process.
Task-relevant Representation Learning for Networked Robotic Perception
However, today's representations for sensory data are mostly designed for human, not robotic, perception and thus often waste precious compute or wireless network resources to transmit unimportant parts of a scene that are unnecessary for a high-level robotic task.
Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoders
Discrete latent spaces in variational autoencoders have been shown to effectively capture the data distribution for many real-world problems such as natural language understanding, human intent prediction, and visual scene representation.
CoCo: Learning Strategies for Online Mixed-Integer Control
Mixed-integer convex programming (MICP) is a popular modeling framework for solving discrete and combinatorial optimization problems arising in various settings.
MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control
Reasoning about human motion is a core component of modern human-robot interactive systems.
Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction
This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure.
Safe Active Dynamics Learning and Control: A Sequential Exploration-Exploitation Framework
In this work, we propose a practical and theoretically-justified approach to maintaining safety in the presence of dynamics uncertainty.
Co-Design to Enable User-Friendly Tools to Assess the Impact of Future Mobility Solutions
This requires tools to study such a coupling and co-design mobility systems in terms of different objectives.
Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach
Human behavior prediction models enable robots to anticipate how humans may react to their actions, and hence are instrumental to devising safe and proactive robot planning algorithms.
Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods
This paper presents a technique, named STLCG, to compute the quantitative semantics of Signal Temporal Logic (STL) formulas using computation graphs.
Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data
Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation.
Ranked #2 on
Trajectory Prediction
on ETH
Continuous Meta-Learning without Tasks
In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task.
Efficient Large-Scale Multi-Drone Delivery Using Transit Networks
Our results show that the framework computes solutions typically within a few seconds on commodity hardware, and that drones travel up to $360 \%$ of their flight range with public transit.
Learning Stabilizable Nonlinear Dynamics with Contraction-Based Regularization
We propose a novel framework for learning stabilizable nonlinear dynamical systems for continuous control tasks in robotics.
High-Dimensional Optimization in Adaptive Random Subspaces
We propose a new randomized optimization method for high-dimensional problems which can be seen as a generalization of coordinate descent to random subspaces.
On the Interaction between Autonomous Mobility on Demand Systems and Power Distribution Networks -- An Optimal Power Flow Approach
Specifically, we extend previous results on an optimization-based modeling approach for electric AMoD systems to jointly control an electric AMoD fleet and a series of PDNs, and analyze the benefit of coordination under load balancing constraints.
Network Offloading Policies for Cloud Robotics: a Learning-based Approach
In this paper, we formulate a novel Robot Offloading Problem --- how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication?
Robust and Adaptive Planning under Model Uncertainty
The first, RAMCP-F, converges to an optimal risk-sensitive policy without having to rebuild the search tree as the underlying belief over models is perturbed.
The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs
Developing safe human-robot interaction systems is a necessary step towards the widespread integration of autonomous agents in society.
Risk-Sensitive Generative Adversarial Imitation Learning
We then derive two different versions of our RS-GAIL optimization problem that aim at matching the risk profiles of the agent and the expert w. r. t.
Learning Stabilizable Dynamical Systems via Control Contraction Metrics
We propose a novel framework for learning stabilizable nonlinear dynamical systems for continuous control tasks in robotics.
Meta-Learning Priors for Efficient Online Bayesian Regression
However, this approach suffers from two main drawbacks: (1) it is computationally inefficient, as computation scales poorly with the number of samples; and (2) it can be data inefficient, as encoding prior knowledge that can aid the model through the choice of kernel and associated hyperparameters is often challenging and unintuitive.
BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning
Our Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance.
Safe Motion Planning in Unknown Environments: Optimality Benchmarks and Tractable Policies
This paper addresses the problem of planning a safe (i. e., collision-free) trajectory from an initial state to a goal region when the obstacle space is a-priori unknown and is incrementally revealed online, e. g., through line-of-sight perception.
Generative Modeling of Multimodal Multi-Human Behavior
This work presents a methodology for modeling and predicting human behavior in settings with N humans interacting in highly multimodal scenarios (i. e. where there are many possible highly-distinct futures).
Robotics Human-Computer Interaction
Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods
The literature on Inverse Reinforcement Learning (IRL) typically assumes that humans take actions in order to minimize the expected value of a cost function, i. e., that humans are risk neutral.
How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics
We discuss general representation theorems that precisely characterize the class of metrics that satisfy these axioms (referred to as distortion risk metrics), and provide instantiations that can be used in applications.
Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction
This paper presents a method for constructing human-robot interaction policies in settings where multimodality, i. e., the possibility of multiple highly distinct futures, plays a critical role in decision making.
Data-Driven Model Predictive Control of Autonomous Mobility-on-Demand Systems
The goal of this paper is to present an end-to-end, data-driven framework to control Autonomous Mobility-on-Demand systems (AMoD, i. e. fleets of self-driving vehicles).
Robotics Multiagent Systems Systems and Control Applications
Learning Sampling Distributions for Robot Motion Planning
This paper proposes a methodology for non-uniform sampling, whereby a sampling distribution is learned from demonstrations, and then used to bias sampling.
On the interaction between Autonomous Mobility-on-Demand systems and the power network: models and coordination algorithms
We assess the performance of the mode by studying a hypothetical AMoD system in Dallas-Fort Worth and its impact on the Texas power network.
Systems and Control Multiagent Systems Robotics
Mixed Strategy for Constrained Stochastic Optimal Control
We found that the same result holds for stochastic optimal control problems with continuous state and action spaces. Furthermore, we show the randomization of control input can result in reduced cost when the optimization problem is nonconvex, and the cost reduction is equal to the duality gap.
Risk-Constrained Reinforcement Learning with Percentile Risk Criteria
In many sequential decision-making problems one is interested in minimizing an expected cumulative cost while taking into account \emph{risk}, i. e., increased awareness of events of small probability and high consequences.
Two Phase $Q-$learning for Bidding-based Vehicle Sharing
We consider one-way vehicle sharing systems where customers can rent a car at one station and drop it off at another.
Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach
Our first contribution is to show that a CVaR objective, besides capturing risk sensitivity, has an alternative interpretation as expected cost under worst-case modeling errors, for a given error budget.
Monte Carlo Motion Planning for Robot Trajectory Optimization Under Uncertainty
MCMP applies this CP estimation procedure to motion planning by iteratively (i) computing an (approximately) optimal path for the deterministic version of the problem (here, using the FMT* algorithm), (ii) computing the CP of this path, and (iii) inflating or deflating the obstacles by a common factor depending on whether the CP is higher or lower than a target value.
Robotics
