Search Results for author: Mykel J. Kochenderfer

Found 143 papers, 64 papers with code

Entropy-regularized Point-based Value Iteration

1 code implementation14 Feb 2024 Harrison Delecki, Marcell Vazquez-Chanlatte, Esen Yel, Kyle Wray, Tomer Arnon, Stefan Witwicki, Mykel J. Kochenderfer

However, model-based planners may be brittle under these types of uncertainty because they rely on an exact model and tend to commit to a single optimal behavior.

Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation

no code implementations22 Jan 2024 Jiachen Li, Chuanbo Hua, Hengbo Ma, Jinkyoo Park, Victoria Dax, Mykel J. Kochenderfer

In this paper, we propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures, and we demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.

Relational Reasoning Robot Navigation +2

The Synergy Between Optimal Transport Theory and Multi-Agent Reinforcement Learning

no code implementations18 Jan 2024 Ali Baheri, Mykel J. Kochenderfer

This paper explores the integration of optimal transport (OT) theory with multi-agent reinforcement learning (MARL).

Management Multi-agent Reinforcement Learning

Graph Q-Learning for Combinatorial Optimization

no code implementations11 Jan 2024 Victoria M. Dax, Jiachen Li, Kevin Leahy, Mykel J. Kochenderfer

Graph-structured data is ubiquitous throughout natural and social sciences, and Graph Neural Networks (GNNs) have recently been shown to be effective at solving prediction and inference problems on graph data.

Combinatorial Optimization Decision Making +1

Disentangled Neural Relational Inference for Interpretable Motion Prediction

no code implementations7 Jan 2024 Victoria M. Dax, Jiachen Li, Enna Sachdeva, Nakul Agarwal, Mykel J. Kochenderfer

The results show superior performance compared to existing methods in modeling spatio-temporal relations, motion prediction, and identifying time-invariant latent features.

Motion Planning motion prediction

Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation

no code implementations27 Nov 2023 Jiachen Li, David Isele, Kanghoon Lee, Jinkyoo Park, Kikuo Fujimura, Mykel J. Kochenderfer

Moreover, we propose an interactivity estimation mechanism based on the difference between predicted trajectories in these two situations, which indicates the degree of influence of the ego agent on other agents.

Autonomous Navigation counterfactual +4

Large-Scale Multi-Robot Assembly Planning for Autonomous Manufacturing

1 code implementation31 Oct 2023 Kyle Brown, Dylan M. Asmar, Mac Schwager, Mykel J. Kochenderfer

Mobile autonomous robots have the potential to revolutionize manufacturing processes.

Constrained Hierarchical Monte Carlo Belief-State Planning

no code implementations30 Oct 2023 Arec Jamgochian, Hugo Buurmeijer, Kyle H. Wray, Anthony Corso, Mykel J. Kochenderfer

Optimal plans in Constrained Partially Observable Markov Decision Processes (CPOMDPs) maximize reward objectives while satisfying hard cost constraints, generalizing safe planning under state and transition uncertainty.

Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments

1 code implementation25 Sep 2023 Bernard Lange, Jiachen Li, Mykel J. Kochenderfer

We introduce the Scene Informer, a unified approach for predicting both observed agent trajectories and inferring occlusions in a partially observable setting.

Autonomous Vehicles Trajectory Prediction

SAVME: Efficient Safety Validation for Autonomous Systems Using Meta-Learning

no code implementations21 Sep 2023 Marc R. Schlichting, Nina V. Boord, Anthony L. Corso, Mykel J. Kochenderfer

The validation can be accelerated by identifying critical failure scenarios for the system under test and by reducing the simulation runtime.


A Holistic Assessment of the Reliability of Machine Learning Systems

no code implementations20 Jul 2023 Anthony Corso, David Karamadian, Romeo Valentin, Mary Cooper, Mykel J. Kochenderfer

As machine learning (ML) systems increasingly permeate high-stakes settings such as healthcare, transportation, military, and national security, concerns regarding their reliability have emerged.

Adversarial Robustness Out-of-Distribution Detection

Robust Driving Policy Learning with Guided Meta Reinforcement Learning

no code implementations19 Jul 2023 Kanghoon Lee, Jiachen Li, David Isele, Jinkyoo Park, Kikuo Fujimura, Mykel J. Kochenderfer

Although deep reinforcement learning (DRL) has shown promising results for autonomous navigation in interactive traffic scenarios, existing work typically adopts a fixed behavior policy to control social vehicles in the training environment.

Autonomous Navigation Meta Reinforcement Learning +1

Efficient Determination of Safety Requirements for Perception Systems

no code implementations3 Jul 2023 Sydney M. Katz, Anthony L. Corso, Esen Yel, Mykel J. Kochenderfer

Perception systems operate as a subcomponent of the general autonomy stack, and perception system designers often need to optimize performance characteristics while maintaining safety with respect to the overall closed-loop system.

Collision Avoidance Gaussian Processes

AVOIDDS: Aircraft Vision-based Intruder Detection Dataset and Simulator

1 code implementation19 Jun 2023 Elysia Q. Smyers, Sydney M. Katz, Anthony L. Corso, Mykel J. Kochenderfer

We also provide an interface that evaluates trained models on slices of this dataset to identify changes in performance with respect to changing environmental conditions.

Collision Avoidance object-detection +1

BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximations

no code implementations31 May 2023 Robert J. Moss, Anthony Corso, Jef Caers, Mykel J. Kochenderfer

BetaZero learns offline approximations that replace heuristics to enable online decision making in long-horizon problems.

Autonomous Driving Decision Making

Model-based Validation as Probabilistic Inference

1 code implementation17 May 2023 Harrison Delecki, Anthony Corso, Mykel J. Kochenderfer

Estimating the distribution over failures is a key step in validating autonomous systems.

Bayesian Inference

Bayesian Safety Validation for Black-Box Systems

1 code implementation3 May 2023 Robert J. Moss, Mykel J. Kochenderfer, Maxime Gariel, Arthur Dubois

Accurately estimating the probability of failure for safety-critical systems is important for certification.

Bayesian Optimization

Optimizing Carbon Storage Operations for Long-Term Safety

1 code implementation19 Apr 2023 Yizheng Wang, Markus Zechner, Gege Wen, Anthony Louis Corso, John Michael Mern, Mykel J. Kochenderfer, Jef Karel Caers

In this study, we address these issues by modeling the decision-making process for carbon storage operations as a partially observable Markov decision process (POMDP).

Decision Making

Inferring Traffic Models in Terminal Airspace from Flight Tracks and Procedures

1 code implementation17 Mar 2023 Soyeon Jung, Mykel J. Kochenderfer

For each segment, a Gaussian mixture model is used to learn the deviations of aircraft trajectories from their procedures.


Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization

1 code implementation28 Dec 2022 Zahra Shahrooei, Mykel J. Kochenderfer, Ali Baheri

Simulation-based falsification is a practical testing method to increase confidence that the system will meet safety requirements.

Bayesian Optimization

Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascent

1 code implementation23 Dec 2022 Arec Jamgochian, Anthony Corso, Mykel J. Kochenderfer

Rather than augmenting rewards with penalties for undesired behavior, Constrained Partially Observable Markov Decision Processes (CPOMDPs) plan safely by imposing inviolable hard constraint value budgets.

Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction

1 code implementation16 Nov 2022 Masha Itkina, Mykel J. Kochenderfer

We propose the use of evidential deep learning to estimate the epistemic uncertainty over a low-dimensional, interpretable latent space in a trajectory prediction setting.

Autonomous Driving Trajectory Prediction +1

A POMDP Model for Safe Geological Carbon Sequestration

no code implementations25 Oct 2022 Anthony Corso, Yizheng Wang, Markus Zechner, Jef Caers, Mykel J. Kochenderfer

This POMDP model can be used as a test bed to drive the development of novel decision-making algorithms for CCS operations.

Decision Making

Optimality Guarantees for Particle Belief Approximation of POMDPs

1 code implementation10 Oct 2022 Michael H. Lim, Tyler J. Becker, Mykel J. Kochenderfer, Claire J. Tomlin, Zachary N. Sunberg

Thus, when combined with sparse sampling MDP algorithms, this approach can yield algorithms for POMDPs that have no direct theoretical dependence on the size of the state and observation spaces.

LOPR: Latent Occupancy PRediction using Generative Models

1 code implementation3 Oct 2022 Bernard Lange, Masha Itkina, Mykel J. Kochenderfer

Environment prediction frameworks are integral for autonomous vehicles, enabling safe navigation in dynamic environments.

Autonomous Driving Representation Learning +1

Prioritizing emergency evacuations under compounding levels of uncertainty

1 code implementation30 Sep 2022 Lisa J. Einstein, Robert J. Moss, Mykel J. Kochenderfer

However, decision makers struggle to determine optimal evacuation policies given the chaos, uncertainty, and value judgments inherent in emergency evacuations.

Decision Making Humanitarian

Backward Reachability Analysis of Neural Feedback Loops: Techniques for Linear and Nonlinear Systems

no code implementations28 Sep 2022 Nicholas Rober, Sydney M. Katz, Chelsea Sidrane, Esen Yel, Michael Everett, Mykel J. Kochenderfer, Jonathan P. How

As neural networks (NNs) become more prevalent in safety-critical applications such as control of vehicles, there is a growing need to certify that systems with NN components are safe.

Collaborative Decision Making Using Action Suggestions

1 code implementation27 Sep 2022 Dylan M. Asmar, Mykel J. Kochenderfer

The assumption of a collaborative environment enables us to use the agent's policy to estimate the distribution over action suggestions.

Decision Making valid

Dynamics-Aware Spatiotemporal Occupancy Prediction in Urban Environments

no code implementations27 Sep 2022 Maneekwan Toyungyernsub, Esen Yel, Jiachen Li, Mykel J. Kochenderfer

Detection and segmentation of moving obstacles, along with prediction of the future occupancy states of the local environment, are essential for autonomous vehicles to proactively make safe and informed decisions.

Autonomous Vehicles Segmentation +1

Sequential Bayesian Optimization for Adaptive Informative Path Planning with Multimodal Sensing

1 code implementation16 Sep 2022 Joshua Ott, Edward Balaban, Mykel J. Kochenderfer

Adaptive Informative Path Planning with Multimodal Sensing (AIPPMS) considers the problem of an agent equipped with multiple sensors, each with different sensing accuracy and energy costs.

Bayesian Optimization

Multi-Objective Policy Gradients with Topological Constraints

no code implementations15 Sep 2022 Kyle Hollins Wray, Stas Tiomkin, Mykel J. Kochenderfer, Pieter Abbeel

Multi-objective optimization models that encode ordered sequential constraints provide a solution to model various challenging problems including encoding preferences, modeling a curriculum, and enforcing measures of safety.

Risk-aware Meta-level Decision Making for Exploration Under Uncertainty

no code implementations12 Sep 2022 Joshua Ott, Sung-Kyun Kim, Amanda Bouman, Oriana Peltzer, Mamoru Sobue, Harrison Delecki, Mykel J. Kochenderfer, Joel Burdick, Ali-akbar Agha-mohammadi

Robotic exploration of unknown environments is fundamentally a problem of decision making under uncertainty where the robot must account for uncertainty in sensor measurements, localization, action execution, as well as many other factors.

Decision Making Decision Making Under Uncertainty

EvolveHypergraph: Group-Aware Dynamic Relational Reasoning for Trajectory Prediction

no code implementations10 Aug 2022 Jiachen Li, Chuanbo Hua, Jinkyoo Park, Hengbo Ma, Victoria Dax, Mykel J. Kochenderfer

While the modeling of pair-wise relations has been widely studied in multi-agent interacting systems, its ability to capture higher-level and larger-scale group-wise activities is limited.

Relation Relational Reasoning +1

Meta-SysId: A Meta-Learning Approach for Simultaneous Identification and Prediction

no code implementations1 Jun 2022 Junyoung Park, Federico Berto, Arec Jamgochian, Mykel J. Kochenderfer, Jinkyoo Park

In this paper, we propose Meta-SysId, a meta-learning approach to model sets of systems that have behavior governed by common but unknown laws and that differentiate themselves by their context.

Meta-Learning regression +3

Risk-Driven Design of Perception Systems

1 code implementation21 May 2022 Anthony L. Corso, Sydney M. Katz, Craig Innes, Xin Du, Subramanian Ramamoorthy, Mykel J. Kochenderfer

We formulate a risk function to quantify the effect of a given perceptual error on overall safety, and show how we can use it to design safer perception systems by including a risk-dependent term in the loss function and generating training data in risk-sensitive regions.

Model Predictive Optimized Path Integral Strategies

1 code implementation30 Mar 2022 Dylan M. Asmar, Ransalu Senanayake, Shawn Manuel, Mykel J. Kochenderfer

We generalize the derivation of model predictive path integral control (MPPI) to allow for a single joint distribution across controls in the control sequence.

How Do We Fail? Stress Testing Perception in Autonomous Vehicles

1 code implementation26 Mar 2022 Harrison Delecki, Masha Itkina, Bernard Lange, Ransalu Senanayake, Mykel J. Kochenderfer

This paper presents a method for characterizing failures of LiDAR-based perception systems for AVs in adverse weather conditions.

Autonomous Vehicles Data Augmentation +2

A Mixed Integer Programming Approach for Verifying Properties of Binarized Neural Networks

no code implementations11 Mar 2022 Christopher Lazarus, Mykel J. Kochenderfer

We compare the runtime of our approach against state-of-the-art verification algorithms for full-precision neural networks.

Collision Avoidance Model Compression

Recursive Reasoning Graph for Multi-Agent Reinforcement Learning

no code implementations6 Mar 2022 Xiaobai Ma, David Isele, Jayesh K. Gupta, Kikuo Fujimura, Mykel J. Kochenderfer

Multi-agent reinforcement learning (MARL) provides an efficient way for simultaneously learning policies for multiple agents interacting with each other.

Multi-agent Reinforcement Learning reinforcement-learning +1

Verifying Inverse Model Neural Networks

no code implementations4 Feb 2022 Chelsea Sidrane, Sydney Katz, Anthony Corso, Mykel J. Kochenderfer

When the forward model that produced the observations is nonlinear and stochastic, solving the inverse problem is very challenging.

Conditional Approximate Normalizing Flows for Joint Multi-Step Probabilistic Forecasting with Application to Electricity Demand

1 code implementation8 Jan 2022 Arec Jamgochian, Di wu, Kunal Menda, Soyeon Jung, Mykel J. Kochenderfer

In this paper, we introduce the conditional approximate normalizing flow (CANF) to make probabilistic multi-step time-series forecasts when correlations are present over long time horizons.

Decision Making Scheduling +2

Dyadic Sex Composition and Task Classification Using fNIRS Hyperscanning Data

1 code implementation7 Dec 2021 Liam A. Kruse, Allan L. Reiss, Mykel J. Kochenderfer, Stephanie Balters

Hyperscanning with functional near-infrared spectroscopy (fNIRS) is an emerging neuroimaging application that measures the nuanced neural signatures underlying social interactions.

Classification Dynamic Time Warping

Autonomous Attack Mitigation for Industrial Control Systems

no code implementations3 Nov 2021 John Mern, Kyle Hatch, Ryan Silva, Cameron Hickert, Tamim Sookoor, Mykel J. Kochenderfer

The proposed deep reinforcement learning approach outperforms a fully automated playbook method in simulation, taking less disruptive actions while also defending more nodes on the network.

Decision Making reinforcement-learning +1

Evidential Softmax for Sparse Multimodal Distributions in Deep Generative Models

1 code implementation NeurIPS 2021 Phil Chen, Masha Itkina, Ransalu Senanayake, Mykel J. Kochenderfer

We evaluate our method on a variety of generative models, including variational autoencoders and auto-regressive architectures.

Interpretable Local Tree Surrogate Policies

no code implementations16 Sep 2021 John Mern, Sidhart Krishnan, Anil Yildiz, Kyle Hatch, Mykel J. Kochenderfer

In this work, we propose a method to build predictable policy trees as surrogates for policies such as neural networks.

Multi-Agent Variational Occlusion Inference Using People as Sensors

1 code implementation5 Sep 2021 Masha Itkina, Ye-Ji Mun, Katherine Driggs-Campbell, Mykel J. Kochenderfer

We propose an occlusion inference method that characterizes observed behaviors of human agents as sensor measurements, and fuses them with those from a standard sensor suite.

Autonomous Vehicles Sensor Fusion

OVERT: An Algorithm for Safety Verification of Neural Network Control Policies for Nonlinear Systems

2 code implementations3 Aug 2021 Chelsea Sidrane, Amir Maleki, Ahmed Irfan, Mykel J. Kochenderfer

In response to this challenge, we present OVERT: a sound algorithm for safety verification of nonlinear discrete-time closed loop dynamical systems with neural network control policies.

Finding Failures in High-Fidelity Simulation using Adaptive Stress Testing and the Backward Algorithm

1 code implementation27 Jul 2021 Mark Koren, Ahmed Nassar, Mykel J. Kochenderfer

Validating the safety of autonomous systems generally requires the use of high-fidelity simulators that adequately capture the variability of real-world scenarios.

Autonomous Vehicles reinforcement-learning +1

3D Radar Velocity Maps for Uncertain Dynamic Environments

1 code implementation23 Jul 2021 Ransalu Senanayake, Kyle Beltran Hatch, Jason Zheng, Mykel J. Kochenderfer

This paper explores a Bayesian approach that captures our uncertainty in the map given training data.

Reinforcement Learning for Industrial Control Network Cyber Security Orchestration

no code implementations9 Jun 2021 John Mern, Kyle Hatch, Ryan Silva, Jeff Brush, Mykel J. Kochenderfer

Defending computer networks from cyber attack requires coordinating actions across multiple nodes based on imperfect indicators of compromise while minimizing disruptions to network operations.

reinforcement-learning Reinforcement Learning (RL)

ZoPE: A Fast Optimizer for ReLU Networks with Low-Dimensional Inputs

no code implementations9 Jun 2021 Christopher A. Strong, Sydney M. Katz, Anthony L. Corso, Mykel J. Kochenderfer

We demonstrate how to formulate and solve three types of optimization problems: (i) minimization of any convex function over the output space, (ii) minimization of a convex function over the output of two networks in series with an adversarial perturbation in the layer between them, and (iii) maximization of the difference in output between two networks.

Computational Efficiency Generative Adversarial Network

Measurable Monte Carlo Search Error Bounds

no code implementations8 Jun 2021 John Mern, Mykel J. Kochenderfer

Monte Carlo planners can often return sub-optimal actions, even if they are guaranteed to converge in the limit of infinite samples.

Verification of Image-based Neural Network Controllers Using Generative Models

no code implementations14 May 2021 Sydney M. Katz, Anthony L. Corso, Christopher A. Strong, Mykel J. Kochenderfer

For this reason, recent work has focused on combining techniques in formal methods and reachability analysis to obtain guarantees on the closed-loop performance of neural network controllers.

Generative Adversarial Network

Training Structured Mechanical Models by Minimizing Discrete Euler-Lagrange Residual

1 code implementation5 May 2021 Kunal Menda, Jayesh K. Gupta, Zachary Manchester, Mykel J. Kochenderfer

Structured Mechanical Models (SMMs) are a data-efficient black-box parameterization of mechanical systems, typically fit to data by minimizing the error between predicted and observed accelerations or next states.

Decision Making Time Series +1

Generating Probabilistic Safety Guarantees for Neural Network Controllers

1 code implementation1 Mar 2021 Sydney M. Katz, Kyle D. Julian, Christopher A. Strong, Mykel J. Kochenderfer

In this work, we develop a method to use the results from neural network verification tools to provide probabilistic safety guarantees on a neural network controller.

Collision Avoidance

Scalable Anytime Planning for Multi-Agent MDPs

1 code implementation12 Jan 2021 Shushman Choudhury, Jayesh K. Gupta, Peter Morales, Mykel J. Kochenderfer

We also introduce a multi-drone delivery domain with dynamic, i. e., state-dependent coordination graphs, and demonstrate how our approach scales to large problems on this domain that are intractable for other MCTS methods.

Transfer Learning for Efficient Iterative Safety Validation

no code implementations9 Dec 2020 Anthony Corso, Mykel J. Kochenderfer

Safety validation is important during the development of safety-critical autonomous systems but can require significant computational effort.

Autonomous Driving reinforcement-learning +2

Adaptive Stress Testing of Trajectory Predictions in Flight Management Systems

no code implementations4 Nov 2020 Robert J. Moss, Ritchie Lee, Nicholas Visser, Joachim Hochwarth, James G. Lopez, Mykel J. Kochenderfer

To find failure events and their likelihoods in flight-critical systems, we investigate the use of an advanced black-box stress testing approach called adaptive stress testing.

Decision Making Management

Out-of-Distribution Detection for Automotive Perception

no code implementations3 Nov 2020 Julia Nitsch, Masha Itkina, Ransalu Senanayake, Juan Nieto, Max Schmidt, Roland Siegwart, Mykel J. Kochenderfer, Cesar Cadena

A mechanism to detect OOD samples is important for safety-critical applications, such as automotive perception, to trigger a safe fallback mode.

Autonomous Driving Object Recognition +1

Runtime Safety Assurance Using Reinforcement Learning

no code implementations20 Oct 2020 Christopher Lazarus, James G. Lopez, Mykel J. Kochenderfer

The airworthiness and safety of a non-pedigreed autopilot must be verified, but the cost to formally do so can be prohibitive.

reinforcement-learning Reinforcement Learning (RL)

Attention Augmented ConvLSTM for Environment Prediction

1 code implementation19 Oct 2020 Bernard Lange, Masha Itkina, Mykel J. Kochenderfer

Safe and proactive planning in robotic systems generally requires accurate predictions of the environment.

Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoders

1 code implementation NeurIPS 2020 Masha Itkina, Boris Ivanovic, Ransalu Senanayake, Mykel J. Kochenderfer, Marco Pavone

Discrete latent spaces in variational autoencoders have been shown to effectively capture the data distribution for many real-world problems such as natural language understanding, human intent prediction, and visual scene representation.

Image Generation Motion Planning +1

Improved POMDP Tree Search Planning with Prioritized Action Branching

1 code implementation7 Oct 2020 John Mern, Anil Yildiz, Larry Bush, Tapan Mukerji, Mykel J. Kochenderfer

Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces.

Global Optimization of Objective Functions Represented by ReLU Networks

no code implementations7 Oct 2020 Christopher A. Strong, Haoze Wu, Aleksandar Zeljić, Kyle D. Julian, Guy Katz, Clark Barrett, Mykel J. Kochenderfer

However, individual "yes or no" questions cannot answer qualitative questions such as "what is the largest error within these bounds"; the answers to these lie in the domain of optimization.

Bayesian Optimized Monte Carlo Planning

1 code implementation7 Oct 2020 John Mern, Anil Yildiz, Zachary Sunberg, Tapan Mukerji, Mykel J. Kochenderfer

Monte Carlo tree search with progressive widening attempts to improve scaling by sampling from the action space to construct a policy search tree.

Bayesian Optimization

Provably Efficient Reward-Agnostic Navigation with Linear Value Iteration

no code implementations NeurIPS 2020 Andrea Zanette, Alessandro Lazaric, Mykel J. Kochenderfer, Emma Brunskill

There has been growing progress on theoretical analyses for provably efficient learning in MDPs with linear function approximation, but much of the existing work has made strong assumptions to enable exploration by conventional exploration frameworks.

A Maximum Independent Set Method for Scheduling Earth Observing Satellite Constellations

no code implementations15 Aug 2020 Duncan Eddy, Mykel J. Kochenderfer

This paper introduces a new approach for solving the satellite scheduling problem by generating an infeasibility-based graph representation of the problem and finding a maximal independent set of vertices for the graph.


Directional Primitives for Uncertainty-Aware Motion Estimation in Urban Environments

no code implementations1 Jul 2020 Ransalu Senanayake, Maneekwan Toyungyernsub, Mingyu Wang, Mykel J. Kochenderfer, Mac Schwager

We can use driving data collected over a long period of time to extract rich information about how vehicles behave in different areas of the roads.

Motion Estimation

Towards Recurrent Autoregressive Flow Models

no code implementations17 Jun 2020 John Mern, Peter Morales, Mykel J. Kochenderfer

The proposed method defines a conditional distribution for each variable in a sequential process by conditioning the parameters of a normalizing flow with recurrent neural connections.

Gaussian Processes

Optimal Sequential Task Assignment and Path Finding for Multi-Agent Robotic Assembly Planning

1 code implementation16 Jun 2020 Kyle Brown, Oriana Peltzer, Martin A. Sehr, Mac Schwager, Mykel J. Kochenderfer

We study the problem of sequential task assignment and collision-free routing for large teams of robots in applications with inter-task precedence constraints (e. g., task $A$ and task $B$ must both be completed before task $C$ may begin).

Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints

1 code implementation27 May 2020 Shushman Choudhury, Jayesh K. Gupta, Mykel J. Kochenderfer, Dorsa Sadigh, Jeannette Bohg

We consider the problem of dynamically allocating tasks to multiple agents under time window constraints and task completion uncertainty.

Decision Making Decision Making Under Uncertainty +1

Reinforcement Learning with Iterative Reasoning for Merging in Dense Traffic

no code implementations25 May 2020 Maxime Bouton, Alireza Nakhaei, David Isele, Kikuo Fujimura, Mykel J. Kochenderfer

This approach exposes the agent to a broad variety of behaviors during training, which promotes learning policies that are robust to model discrepancies.

Autonomous Vehicles reinforcement-learning +1

A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems

no code implementations6 May 2020 Anthony Corso, Robert J. Moss, Mark Koren, Ritchie Lee, Mykel J. Kochenderfer

Autonomous cyber-physical systems (CPS) can improve safety and efficiency for safety-critical applications, but require rigorous testing before deployment.

Autonomous Vehicles Collision Avoidance +1

Active Preference-Based Gaussian Process Regression for Reward Learning

1 code implementation6 May 2020 Erdem Biyik, Nicolas Huynh, Mykel J. Kochenderfer, Dorsa Sadigh

Our results in simulations and a user study suggest that our approach can efficiently learn expressive reward functions for robotics tasks.


Structured Mechanical Models for Robot Learning and Control

1 code implementation L4DC 2020 Jayesh K. Gupta, Kunal Menda, Zachary Manchester, Mykel J. Kochenderfer

Deep neural networks have been used to learn models of robot dynamics from data, but they suffer from data-inefficiency and the difficulty to incorporate prior knowledge.

Interpretable Safety Validation for Autonomous Vehicles

2 code implementations14 Apr 2020 Anthony Corso, Mykel J. Kochenderfer

Our methodology is demonstrated for the safety validation of an autonomous vehicle in the context of an unprotected left turn and a crosswalk with a pedestrian.

Autonomous Driving

Scalable Autonomous Vehicle Safety Validation through Dynamic Programming and Scene Decomposition

no code implementations14 Apr 2020 Anthony Corso, Ritchie Lee, Mykel J. Kochenderfer

In this work, we present a new safety validation approach that attempts to estimate the distribution over failures of an autonomous policy using approximate dynamic programming.

Autonomous Driving Open-Ended Question Answering

Adaptive Stress Testing without Domain Heuristics using Go-Explore

no code implementations8 Apr 2020 Mark Koren, Mykel J. Kochenderfer

We demonstrate that GE is able to find failures without domain-specific heuristics, such as the distance between the car and the pedestrian, on scenarios that other RL techniques are unable to solve.

Reinforcement Learning (RL)

The Adaptive Stress Testing Formulation

no code implementations8 Apr 2020 Mark Koren, Anthony Corso, Mykel J. Kochenderfer

Validation is a key challenge in the search for safe autonomy.

Adaptive Informative Path Planning with Multimodal Sensing

no code implementations21 Mar 2020 Shushman Choudhury, Nate Gruver, Mykel J. Kochenderfer

AIPPMS requires reasoning jointly about the effects of sensing and movement in terms of both energy expended and information gained.

Exchangeable Input Representations for Reinforcement Learning

no code implementations19 Mar 2020 John Mern, Dorsa Sadigh, Mykel J. Kochenderfer

We show that our proposed representation results in an input space that is a factor of $m!$ smaller for inputs of $m$ objects.

Policy Gradient Methods reinforcement-learning +1

Point-Based Methods for Model Checking in Partially Observable Markov Decision Processes

1 code implementation11 Jan 2020 Maxime Bouton, Jana Tumova, Mykel J. Kochenderfer

Autonomous systems are often required to operate in partially observable environments.

Monte-Carlo Tree Search for Policy Optimization

no code implementations23 Dec 2019 Xiaobai Ma, Katherine Driggs-Campbell, Zongzhang Zhang, Mykel J. Kochenderfer

Gradient-based methods are often used for policy optimization in deep reinforcement learning, despite being vulnerable to local optima and saddle points.

reinforcement-learning Reinforcement Learning (RL)

Guaranteeing Safety for Neural Network-Based Aircraft Collision Avoidance Systems

1 code implementation15 Dec 2019 Kyle D. Julian, Mykel J. Kochenderfer

The neural network outputs are bounded using neural network verification tools like Reluplex and Reluval, and a reachability method determines all possible ways aircraft encounters will resolve using neural network advisories and assuming bounded aircraft dynamics.

Collision Avoidance

Parameter-Conditioned Sequential Generative Modeling of Fluid Flows

no code implementations14 Dec 2019 Jeremy Morton, Freddie D. Witherden, Mykel J. Kochenderfer

The computational cost associated with simulating fluid flows can make it infeasible to run many simulations across multiple flow conditions.

Limiting Extrapolation in Linear Approximate Value Iteration

no code implementations NeurIPS 2019 Andrea Zanette, Alessandro Lazaric, Mykel J. Kochenderfer, Emma Brunskill

We prove that if the features at any state can be represented as a convex combination of features at the anchor points, then errors are propagated linearly over iterations (instead of exponentially) and our method achieves a polynomial sample complexity bound in the horizon and the number of anchor points.

Almost Horizon-Free Structure-Aware Best Policy Identification with a Generative Model

no code implementations NeurIPS 2019 Andrea Zanette, Mykel J. Kochenderfer, Emma Brunskill

This paper focuses on the problem of computing an $\epsilon$-optimal policy in a discounted Markov Decision Process (MDP) provided that we can access the reward and transition function through a generative model.

Efficient Large-Scale Multi-Drone Delivery Using Transit Networks

2 code implementations26 Sep 2019 Shushman Choudhury, Kiril Solovey, Mykel J. Kochenderfer, Marco Pavone

Our results show that the framework computes solutions typically within a few seconds on commodity hardware, and that drones travel up to $360 \%$ of their flight range with public transit.

Non-linear System Identification from Partial Observations via Iterative Smoothing and Learning

no code implementations25 Sep 2019 Kunal Menda, Jean de Becdelièvre, Jayesh K Gupta, Ilan Kroo, Mykel J. Kochenderfer, Zachary Manchester

System identification is the process of building a mathematical model of an unknown system from measurements of its inputs and outputs.

Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validation

no code implementations2 Aug 2019 Anthony Corso, Peter Du, Katherine Driggs-Campbell, Mykel J. Kochenderfer

Determining possible failure scenarios is a critical step in the evaluation of autonomous vehicle systems.

Learning an Urban Air Mobility Encounter Model from Expert Preferences

1 code implementation12 Jul 2019 Sydney M. Katz, Anne-Claire Le Bihan, Mykel J. Kochenderfer

Airspace models have played an important role in the development and evaluation of aircraft collision avoidance systems for both manned and unmanned aircraft.

Collision Avoidance

Cooperation-Aware Reinforcement Learning for Merging in Dense Traffic

1 code implementation26 Jun 2019 Maxime Bouton, Alireza Nakhaei, Kikuo Fujimura, Mykel J. Kochenderfer

In this work, we present a reinforcement learning approach to learn how to interact with drivers with different cooperation levels.

Autonomous Vehicles Decision Making +3

Hybrid Planning for Dynamic Multimodal Stochastic Shortest Paths

2 code implementations21 Jun 2019 Shushman Choudhury, Mykel J. Kochenderfer

Sequential decision problems in applications such as manipulation in warehouses, multi-step meal preparation, and routing in autonomous vehicle networks often involve reasoning about uncertainty, planning over discrete modes as well as continuous states, and reacting to dynamic updates.

Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

no code implementations6 May 2019 Carl-Johan Hoel, Katherine Driggs-Campbell, Krister Wolff, Leo Laine, Mykel J. Kochenderfer

This paper introduces a general framework for tactical decision making, which combines the concepts of planning and learning, in the form of Monte Carlo tree search and deep reinforcement learning.

Autonomous Driving Decision Making +1

Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning

1 code implementation28 Apr 2019 Masha Itkina, Katherine Driggs-Campbell, Mykel J. Kochenderfer

A key challenge for autonomous driving is safe trajectory planning in cluttered, urban environments with dynamic obstacles, such as pedestrians, bicyclists, and other vehicles.

Autonomous Driving Representation Learning

Pedestrian Collision Avoidance System for Scenarios with Occlusions

1 code implementation25 Apr 2019 Markus Schratter, Maxime Bouton, Mykel J. Kochenderfer, Daniel Watzenig

We show that combining the two approaches provides a robust autonomous braking system that reduces unnecessary braking caused by using the AEB system on its own.

Autonomous Driving Collision Avoidance

Algorithms for Verifying Deep Neural Networks

2 code implementations15 Mar 2019 Changliu Liu, Tomer Arnon, Christopher Lazarus, Clark Barrett, Mykel J. Kochenderfer

Deep neural networks are widely used for nonlinear function approximation with applications ranging from computer vision to control.

Rethinking System Health Management

no code implementations10 Mar 2019 Edward Balaban, Stephen B. Johnson, Mykel J. Kochenderfer

Health management of complex dynamic systems has traditionally evolved separately from automated control, planning, and scheduling (generally referred to in the paper as decision making).

Decision Making Management +1

Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learning

no code implementations8 Mar 2019 Xiaobai Ma, Katherine Driggs-Campbell, Mykel J. Kochenderfer

To improve efficiency and reduce failures in autonomous vehicles, research has focused on developing robust and safe learning methods that take into account disturbances in the environment.

Autonomous Driving reinforcement-learning +1

Model Primitive Hierarchical Lifelong Reinforcement Learning

1 code implementation4 Mar 2019 Bohan Wu, Jayesh K. Gupta, Mykel J. Kochenderfer

Learning interpretable and transferable subpolicies and performing task decomposition from a single, complex task is difficult.

Hierarchical Reinforcement Learning Meta-Learning +2

Deep Variational Koopman Models: Inferring Koopman Observations for Uncertainty-Aware Dynamics Modeling and Control

1 code implementation26 Feb 2019 Jeremy Morton, Freddie D. Witherden, Mykel J. Kochenderfer

We introduce the Deep Variational Koopman (DVK) model, a method for inferring distributions over observations that can be propagated linearly in time.

A General Framework for Structured Learning of Mechanical Systems

1 code implementation22 Feb 2019 Jayesh K. Gupta, Kunal Menda, Zachary Manchester, Mykel J. Kochenderfer

We address the need for a flexible, gray-box model of mechanical systems that can seamlessly incorporate prior knowledge where it is available, and train expressive function approximators where it is not.

Model-based Reinforcement Learning Reinforcement Learning (RL) +1

Dynamic Real-time Multimodal Routing with Hierarchical Hybrid Planning

1 code implementation5 Feb 2019 Shushman Choudhury, Jacob P. Knickerbocker, Mykel J. Kochenderfer

We introduce the problem of Dynamic Real-time Multimodal Routing (DREAMR), which requires planning and executing routes under uncertainty for an autonomous agent.

Decision Making Decision Making Under Uncertainty

Adaptive Stress Testing for Autonomous Vehicles

no code implementations5 Feb 2019 Mark Koren, Saud Alsaif, Ritchie Lee, Mykel J. Kochenderfer

This paper presents Monte Carlo Tree Search (MCTS) and Deep Reinforcement Learning (DRL) solutions that can scale to large environments.

Autonomous Vehicles Decision Making +2

Real-time Prediction of Automotive Collision Risk from Monocular Video

no code implementations4 Feb 2019 Derek J. Phillips, Juan Carlos Aragon, Anjali Roychowdhury, Regina Madigan, Sunil Chintakindi, Mykel J. Kochenderfer

Many automotive applications, such as Advanced Driver Assistance Systems (ADAS) for collision avoidance and warnings, require estimating the future automotive risk of a driving scene.

Collision Avoidance Object +3

Robust Super-Level Set Estimation using Gaussian Processes

no code implementations25 Nov 2018 Andrea Zanette, Junzi Zhang, Mykel J. Kochenderfer

This paper focuses on the problem of determining as large a region as possible where a function exceeds a given threshold with high probability.

Gaussian Processes

Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning

no code implementations6 Nov 2018 Ritchie Lee, Ole J. Mengshoel, Anshu Saksena, Ryan Gardner, Daniel Genin, Joshua Silbermann, Michael Owen, Mykel J. Kochenderfer

Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars.

Autonomous Driving Collision Avoidance +3

Deep Neural Network Compression for Aircraft Collision Avoidance Systems

no code implementations9 Oct 2018 Kyle D. Julian, Mykel J. Kochenderfer, Michael P. Owen

One approach to designing decision making logic for an aircraft collision avoidance system frames the problem as a Markov decision process and optimizes the system using dynamic programming.

Collision Avoidance Decision Making +1

Image-based Guidance of Autonomous Aircraft for Wildfire Surveillance and Prediction

1 code implementation4 Oct 2018 Kyle D. Julian, Mykel J. Kochenderfer

The second approach uses a particle filter to predict wildfire growth and uses observations to estimate uncertainties relating to wildfire expansion.


Using Neural Networks to Generate Information Maps for Mobile Sensors

no code implementations26 Sep 2018 Louis Dressel, Mykel J. Kochenderfer

A common method uses information maps that estimate the value of taking measurements from any point in the sensor state space.

EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning

no code implementations22 Jul 2018 Kunal Menda, Katherine Driggs-Campbell, Mykel J. Kochenderfer

While imitation learning is often used in robotics, the approach frequently suffers from data mismatch and compounding errors.

Imitation Learning

Amortized Inference Regularization

no code implementations NeurIPS 2018 Rui Shu, Hung H. Bui, Shengjia Zhao, Mykel J. Kochenderfer, Stefano Ermon

In this paper, we leverage the fact that VAEs rely on amortized inference and propose techniques for amortized inference regularization (AIR) that control the smoothness of the inference model.

Density Estimation Representation Learning +1

Deep Dynamical Modeling and Control of Unsteady Fluid Flows

1 code implementation NeurIPS 2018 Jeremy Morton, Freddie D. Witherden, Antony Jameson, Mykel J. Kochenderfer

The design of flow control systems remains a challenge due to the nonlinear nature of the equations that govern fluid flow.

Model Predictive Control

Decomposition Methods with Deep Corrections for Reinforcement Learning

1 code implementation6 Feb 2018 Maxime Bouton, Kyle Julian, Alireza Nakhaei, Kikuo Fujimura, Mykel J. Kochenderfer

In contexts where an agent interacts with multiple entities, utility decomposition can be used to separate the global objective into local tasks considering each individual entity independently.

Autonomous Driving Decision Making +5

Real-time Prediction of Intermediate-Horizon Automotive Collision Risk

no code implementations5 Feb 2018 Blake Wulfe, Sunil Chintakindi, Sou-Cheng T. Choi, Rory Hartong-Redden, Anuradha Kodali, Mykel J. Kochenderfer

Advanced collision avoidance and driver hand-off systems can benefit from the ability to accurately predict, in real time, the probability a vehicle will be involved in a collision within an intermediate horizon of 10 to 20 seconds.

Collision Avoidance Domain Adaptation

Burn-In Demonstrations for Multi-Modal Imitation Learning

no code implementations13 Oct 2017 Alex Kuefler, Mykel J. Kochenderfer

Recent work on imitation learning has generated policies that reproduce expert behavior from multi-modal data.

Autonomous Driving Imitation Learning +1

Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes

1 code implementation19 Sep 2017 Kunal Menda, Yi-Chun Chen, Justin Grana, James W. Bono, Brendan D. Tracey, Mykel J. Kochenderfer, David Wolpert

The incorporation of macro-actions (temporally extended actions) into multi-agent decision problems has the potential to address the curse of dimensionality associated with such decision problems.

reinforcement-learning Reinforcement Learning (RL)

DropoutDAgger: A Bayesian Approach to Safe Imitation Learning

no code implementations18 Sep 2017 Kunal Menda, Katherine Driggs-Campbell, Mykel J. Kochenderfer

While imitation learning is becoming common practice in robotics, this approach often suffers from data mismatch and compounding errors.

Imitation Learning

Interpretable Categorization of Heterogeneous Time Series Data

no code implementations30 Aug 2017 Ritchie Lee, Mykel J. Kochenderfer, Ole J. Mengshoel, Joshua Silbermann

In particular, when a grammar based on temporal logic is used, we show that GBDTs can be used for the interpretable classi cation of high-dimensional and heterogeneous time series data.

Clustering Collision Avoidance +2

Closed-Loop Policies for Operational Tests of Safety-Critical Systems

no code implementations25 Jul 2017 Jeremy Morton, Tim A. Wheeler, Mykel J. Kochenderfer

Manufacturers of safety-critical systems must make the case that their product is sufficiently safe for public deployment.

Scheduling Test

Simultaneous Policy Learning and Latent State Inference for Imitating Driver Behavior

2 code implementations19 Apr 2017 Jeremy Morton, Mykel J. Kochenderfer

In this work, we propose a method for learning driver models that account for variables that cannot be observed directly.

Belief State Planning for Autonomously Navigating Urban Intersections

no code implementations14 Apr 2017 Maxime Bouton, Akansel Cosgun, Mykel J. Kochenderfer

Urban intersections represent a complex environment for autonomous vehicles with many sources of uncertainty.


Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs (Extended Version)

no code implementations29 Nov 2015 Philipp Robbel, Frans A. Oliehoek, Mykel J. Kochenderfer

We present an approach to mitigate this limitation for certain types of multiagent systems, exploiting a property that can be thought of as "anonymous influence" in the factored MDP.

Predicting the behavior of interacting humans by fusing data from multiple sources

no code implementations9 Aug 2014 Erik J. Schlicht, Ritchie Lee, David H. Wolpert, Mykel J. Kochenderfer, Brendan Tracey

Multi-fidelity methods combine inexpensive low-fidelity simulations with costly but highfidelity simulations to produce an accurate model of a system of interest at minimal cost.

A Comparison of Monte Carlo Tree Search and Mathematical Optimization for Large Scale Dynamic Resource Allocation

no code implementations21 May 2014 Dimitris Bertsimas, J. Daniel Griffith, Vishal Gupta, Mykel J. Kochenderfer, Velibor V. Mišić, Robert Moss

In this paper, we adapt both MCTS and MO to a problem inspired by tactical wildfire and management and undertake an extensive computational study comparing the two methods on large scale instances in terms of both the state and the action spaces.

Management Stochastic Optimization

Cannot find the paper you are looking for? You can Submit a new open access paper.