Search Results for author:
Found 46 papers, 16 papers with code
Robust Markov Decision Processes: A Place Where AI and Formal Methods Meet
We also discuss how RMDPs relate to other models and how they are used in several contexts, including reinforcement learning and abstraction techniques.
Pessimistic Iterative Planning for Robust POMDPs
To compute such robust memory-based policies, we propose the pessimistic iterative planning (PIP) framework, which alternates between two main steps: (1) selecting a pessimistic (non-robust) POMDP via worst-case probability instances from the uncertainty sets; and (2) computing a finite-state controller (FSC) for this pessimistic POMDP.
Maintenance Strategies for Sewer Pipes with Multi-State Degradation and Deep Reinforcement Learning
Large-scale infrastructure systems are crucial for societal welfare, and their effective management requires strategic forecasting and intervention methods that account for various complexities.
Learning-Based Verification of Stochastic Dynamical Systems with Neural Network Policies
As our main contribution, we significantly accelerate algorithmic approaches for verifying that a neural network is indeed a RASM.
Approximate Dec-POMDP Solving Using Multi-Agent A*
Our experiments demonstrate the efficacy and scalability of the approach.
Imprecise Probabilities Meet Partial Observability: Game Semantics for Robust POMDPs
Partially observable Markov decision processes (POMDPs) rely on the key assumption that probability distributions are precisely known.
A Stability-Based Abstraction Framework for Reach-Avoid Control of Stochastic Dynamical Systems with Unknown Noise Distributions
To address this issue, we propose a novel abstraction scheme for stochastic linear systems that exploits the system's stability to obtain significantly smaller abstract models.
Factored Online Planning in Many-Agent POMDPs
However, the challenges of value estimation and belief estimation have only been tackled individually, which prevents existing methods from scaling to settings with many agents.
Robust Active Measuring under Model Uncertainty
Partial observability and uncertainty are common problems in sequential decision-making that particularly impede the use of formal models such as Markov decision processes (MDPs).
Correct-by-Construction Control for Stochastic and Uncertain Dynamical Models via Formal Abstractions
Such autonomous systems are naturally modeled as stochastic dynamical models.
Reinforcement Learning by Guided Safe Exploration
Drawing from transfer learning, we also regularize a target policy (the student) towards the guide while the student is unreliable and gradually eliminate the influence of the guide as training progresses.
More for Less: Safe Policy Improvement With Stronger Performance Guarantees
In an offline reinforcement learning setting, the safe policy improvement (SPI) problem aims to improve the performance of a behavior policy according to which sample data has been generated.
Efficient Sensitivity Analysis for Parametric Robust Markov Chains
As our main contribution, we present an efficient method to compute these partial derivatives.
Act-Then-Measure: Reinforcement Learning for Partially Observable Environments with Active Measuring
In these models, actions consist of two components: a control action that affects the environment, and a measurement action that affects what the agent can observe.
Decision-Making Under Uncertainty: Beyond Probabilities
In this paper, the focus is on the uncertainty that goes beyond this classical interpretation, particularly by employing a clear distinction between aleatoric and epistemic uncertainty.
Safe Policy Improvement for POMDPs via Finite-State Controllers
In our novel approach to the SPI problem for POMDPs, we assume that a finite-state controller (FSC) represents the behavior policy and that finite memory is sufficient to derive optimal policies.
Robust Control for Dynamical Systems With Non-Gaussian Noise via Formal Abstractions
This iMDP is, with a user-specified confidence probability, robust against uncertainty in the transition probabilities, and the tightness of the probability intervals can be controlled through the number of samples.
Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checking
We use this metric to craft optimal adversarial attacks.
Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty
Stochastic noise causes aleatoric uncertainty, whereas imprecise knowledge of model parameters leads to epistemic uncertainty.
Safe Reinforcement Learning From Pixels Using a Stochastic Latent Representation
We address the problem of safe reinforcement learning from pixel observations.
COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checking
This paper presents COOL-MC, a tool that integrates state-of-the-art reinforcement learning (RL) and model checking.
A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning
We approach the problem of rehabilitation planning in an online and offline DRL setting.
Robust Anytime Learning of Markov Decision Processes
Markov decision processes (MDPs) are formal models commonly used in sequential decision-making.
Safe Reinforcement Learning via Shielding under Partial Observability
Safe exploration is a common problem in reinforcement learning (RL) that aims to prevent agents from making disastrous decisions while exploring their environment.
Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise
We use state-of-the-art verification techniques to provide guarantees on the iMDP, and compute a controller for which these guarantees carry over to the autonomous system.
Convex Optimization for Parameter Synthesis in MDPs
The parameter synthesis problem is to compute an instantiation of these unspecified parameters such that the resulting MDP satisfies the temporal logic specification.
Correct-by-construction reach-avoid control of partially observable linear stochastic systems
By construction, any policy on the abstraction can be refined into a piecewise linear feedback controller for the LTI system.
Damage detection using in-domain and cross-domain transfer learning
We, therefore, evaluate a combination of in-domain and cross-domain transfer learning strategies for damage detection in bridges.
Balancing Wind and Batteries: Towards Predictive Verification of Smart Grids
We study a smart grid with wind power and battery storage.
Robust Finite-State Controllers for Uncertain POMDPs
(3) We linearize this dual problem and (4) solve the resulting finite linear program to obtain locally optimal solutions to the original problem.
Adversarial Patch Camouflage against Aerial Detection
The traditional way of hiding military assets from sight is camouflage, for example by using camouflage nets.
Strengthening Deterministic Policies for POMDPs
Second, based on the results of the original MILP, we employ a preprocessing of the POMDP to encompass memory-based decisions.
Enforcing Almost-Sure Reachability in POMDPs
Partially-Observable Markov Decision Processes (POMDPs) are a well-known stochastic model for sequential decision making under limited information.
Robustness Verification for Classifier Ensembles
The robustness-checking problem consists of assessing, given a set of classifiers and a labelled data set, whether there exists a randomized attack that induces a certain expected loss against all classifiers.
Verifiable RNN-Based Policies for POMDPs Under Temporal Logic Constraints
Recurrent neural networks (RNNs) have emerged as an effective representation of control policies in sequential decision-making problems.
Neural Simplex Architecture
NSA not only provides safety assurances in the presence of a possibly unsafe neural controller, but can also improve the safety of such a controller in an online setting via retraining, without overly degrading its performance.
Synthesis of Provably Correct Autonomy Protocols for Shared Control
We design the autonomy protocol to ensure that the resulting robot behavior satisfies given safety and performance specifications in probabilistic temporal logic.
Counterexample-Guided Strategy Improvement for POMDPs Using Recurrent Neural Networks
The particular problem is to determine strategies that provably adhere to (probabilistic) temporal logic constraints.
Shepherding Hordes of Markov Chains
This paper considers large families of Markov chains (MCs) that are defined over a set of parameters with finite discrete domains.
The Partially Observable Games We Play for Cyber Deception
Then, the deception problem is to compute a strategy for the deceiver that minimizes the expected cost of deception against all strategies of the infiltrator.
Safe Reinforcement Learning via Probabilistic Shields
This paper targets the efficient construction of a safety shield for decision making in scenarios that incorporate uncertainty.
Synthesis in pMDPs: A Tale of 1001 Parameters
This paper considers parametric Markov decision processes (pMDPs) whose transitions are equipped with affine functions over a finite set of parameters.
Human-in-the-Loop Synthesis for Partially Observable Markov Decision Processes
The efficient verification of this MC gives quantitative insights into the quality of the inferred human strategy by proving or disproving given system specifications.
Strategy Synthesis in POMDPs via Game-Based Abstractions
We study synthesis problems with constraints in partially observable Markov decision processes (POMDPs), where the objective is to compute a strategy for an agent that is guaranteed to satisfy certain safety and performance specifications.
Probabilistic Model Checking for Complex Cognitive Tasks -- A case study in human-robot interaction
Probabilistic model checking is used to predict the human's behavior.
Synthesis of Shared Control Protocols with Provable Safety and Performance Guarantees
We formalize synthesis of shared control protocols with correctness guarantees for temporal logic specifications.
