Search Results for author:
Found 58 papers, 21 papers with code
Data-driven Interval MDP for Robust Control Synthesis
The abstraction of dynamical systems is a powerful tool that enables the design of feedback controllers using a correct-by-design framework.
A Stability-Based Abstraction Framework for Reach-Avoid Control of Stochastic Dynamical Systems with Unknown Noise Distributions
To address this issue, we propose a novel abstraction scheme for stochastic linear systems that exploits the system's stability to obtain significantly smaller abstract models.
Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification
In addition to this, we also characterise the conditions under which a behavioural model is robust to small perturbations of the observed policy, and we analyse how robust many behavioural models are to misspecification of their parameter values (such as e. g.\ the discount rate).
Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers
Many machine learning applications require operating on a spatially distributed dataset.
On the Limitations of Markovian Rewards to Express Multi-Objective, Risk-Sensitive, and Modal Tasks
Moreover, we find that scalar, Markovian rewards are unable to express most of the instances in each of these three classes.
Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis
This paper addresses the problem of maintaining safety during training in Reinforcement Learning (RL), such that the safety constraint violations are bounded at any point during learning.
Learning Robust Policies for Uncertain Parametric Markov Decision Processes
Synthesising verifiably correct controllers for dynamical systems is crucial for safety-critical problems.
Fossil 2.0: Formal Certificate Synthesis for the Verification and Control of Dynamical Models
This paper presents Fossil 2. 0, a new major release of a software tool for the synthesis of certificates (e. g., Lyapunov and barrier functions) for dynamical systems modelled as ordinary differential and difference equations.
Correct-by-Construction Control for Stochastic and Uncertain Dynamical Models via Formal Abstractions
Such autonomous systems are naturally modeled as stochastic dynamical models.
Probabilistic Reach-Avoid for Bayesian Neural Networks
Such computed lower bounds provide safety certification for the given policy and BNN model.
STARC: A General Framework For Quantifying Differences Between Reward Functions
This means that reward learning algorithms generally must be evaluated empirically, which is expensive, and that their failure modes are difficult to anticipate in advance.
A General Verification Framework for Dynamical and Control Models via Certificate Synthesis
An emerging branch of control theory specialises in certificate learning, concerning the specification of a desired (possibly complex) system behaviour for an autonomous or control model, which is then analytically verified by means of a function-based proof.
On the Trade-off Between Efficiency and Precision of Neural Abstraction
Neural abstractions have been recently introduced as formal approximations of complex, nonlinear dynamical models.
On Imperfect Recall in Multi-Agent Influence Diagrams
Multi-agent influence diagrams (MAIDs) are a popular game-theoretic model based on Bayesian networks.
An Exact Characterisation of Flexibility in Populations of Electric Vehicles
The flexibility of an individual EV can be quantified as a convex polytope and the flexibility of a population of EVs is the Minkowski sum of these polytopes.
Networked Communication for Decentralised Agents in Mean-Field Games
We introduce networked communication to the mean-field game framework, in particular to oracle-free settings where $N$ decentralised agents learn along a single, non-episodic evolution path of the empirical system.
Abstracting Linear Stochastic Systems via Knowledge Filtering
In this paper, we propose a new model reduction technique for linear stochastic systems that builds upon knowledge filtering and utilizes optimal Kalman filtering techniques.
Inner approximations of stochastic programs for data-driven stochastic barrier function design
In particular, we show that the stochastic program to synthesize a SBF can be relaxed into a chance-constrained optimisation problem on which scenario approach theory applies.
Distributionally Robust Optimal and Safe Control of Stochastic Systems via Kernel Conditional Mean Embedding
We present a novel distributionally robust framework for dynamic programming that uses kernel methods to design feedback control policies.
Data-driven abstractions via adaptive refinements and a Kantorovich metric [extended version]
In order to learn the optimal structure, we define a Kantorovich-inspired metric between Markov chains, and we use it as a loss function.
Policy Evaluation in Distributional LQR
Distributional reinforcement learning (DRL) enhances the understanding of the effects of the randomness in the environment by letting agents learn the distribution of a random return, rather than its expected value as in standard RL.
Neural Abstractions
We present a novel method for the safety verification of nonlinear dynamical models that uses neural networks to represent abstractions of their dynamics.
Reasoning about Causality in Games
Regarding question iii), we describe correspondences between causal games and other formalisms, and explain how causal games can be used to answer queries that other causal or game-theoretic models do not support.
Robust Control for Dynamical Systems With Non-Gaussian Noise via Formal Abstractions
This iMDP is, with a user-specified confidence probability, robust against uncertainty in the transition probabilities, and the tightness of the probability intervals can be controlled through the number of samples.
Lexicographic Multi-Objective Reinforcement Learning
In this work we introduce reinforcement learning techniques for solving lexicographic multi-objective problems.
Multi-Objective Reinforcement Learning
reinforcement-learning
Misspecification in Inverse Reinforcement Learning
In this paper, we provide a mathematical analysis of how robust different IRL models are to misspecification, and answer precisely how the demonstrator policy may differ from each of the standard models before that model leads to faulty inferences about the reward function $R$.
Data-driven memory-dependent abstractions of dynamical systems
We propose a sample-based, sequential method to abstract a (potentially black-box) dynamical system with a sequence of memory-dependent Markov chains of increasing size.
Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics
We consider the cases where the transition probabilities of this MDP are either known up to an interval or completely unknown.
Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty
Stochastic noise causes aleatoric uncertainty, whereas imprecise knowledge of model parameters leads to epistemic uncertainty.
Bounded Robustness in Reinforcement Learning via Lexicographic Objectives
Policy robustness in Reinforcement Learning may not be desirable at any cost: the alterations caused by robustness requirements from otherwise optimal policies should be explainable, quantifiable and formally verifiable.
LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning
LCRL is a software tool that implements model-free Reinforcement Learning (RL) algorithms over unknown Markov Decision Processes (MDPs), synthesising policies that satisfy a given linear temporal specification with maximal probability.
Learning Task Automata for Reinforcement Learning using Hidden Markov Models
Second, we propose a novel method for distilling the task automaton (assumed to be a deterministic finite automaton) from the learnt product MDP.
Low Emission Building Control with Zero-Shot Reinforcement Learning
Heating and cooling systems in buildings account for 31\% of global energy use, much of which are regulated by Rule Based Controllers (RBCs) that neither maximise energy efficiency nor minimise emissions by interacting optimally with the grid.
Low Emission Building Control with Zero-Shot Reinforcement Learning
Heating and cooling systems in buildings account for 31% of global energy use, much of which are regulated by Rule Based Controllers (RBCs) that neither maximise energy efficiency nor minimise emissions by interacting optimally with the grid.
Invariance in Policy Optimisation and Partial Identifiability in Reward Learning
It is often very challenging to manually design reward functions for complex, real-world tasks.
Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise
We use state-of-the-art verification techniques to provide guarantees on the iMDP, and compute a controller for which these guarantees carry over to the autonomous system.
Certification of Iterative Predictions in Bayesian Neural Networks
We consider the problem of computing reach-avoid probabilities for iterative predictions made with Bayesian neural network (BNN) models.
Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic
This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) with unknown transition probabilities over continuous state and action spaces.
Equilibrium Refinements for Multi-Agent Influence Diagrams: Theory and Practice
Multi-agent influence diagrams (MAIDs) are a popular form of graphical model that, for certain classes of games, have been shown to offer key complexity and explainability advantages over traditional extensive form game (EFG) representations.
Multi-Agent Reinforcement Learning with Temporal Logic Specifications
In this paper, we study the problem of learning to satisfy temporal logic specifications with a group of agents in an unknown environment, which may exhibit probabilistic behaviour.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
SafePILCO: a software tool for safe and data-efficient policy synthesis
SafePILCO is a software tool for safe and data-efficient policy search with reinforcement learning.
Automated and Sound Synthesis of Lyapunov Functions with SMT Solvers
In this paper we employ SMT solvers to soundly synthesise Lyapunov functions that assert the stability of a given dynamical model.
Automated and Formal Synthesis of Neural Barrier Certificates for Dynamical Models
We introduce an automated, formal, counterexample-based approach to synthesise Barrier Certificates (BC) for the safety verification of continuous and hybrid dynamical models.
Jump Operator Planning: Goal-Conditioned Policy Ensembles and Zero-Shot Transfer
In Hierarchical Control, compositionality, abstraction, and task-transfer are crucial for designing versatile algorithms which can solve a variety of problems with maximal representational reuse.
A Randomized Algorithm to Reduce the Support of Discrete Measures
Given a discrete probability measure supported on $N$ atoms and a set of $n$ real-valued functions, there exists a probability measure that is supported on a subset of $n+1$ of the original $N$ atoms and has the same mean when integrated against each of the $n$ functions.
Carathéodory Sampling for Stochastic Gradient Descent
Various flavours of Stochastic Gradient Descent (SGD) replace the expensive summation that computes the full gradient by approximating it with a small sum over a randomly selected subsample of the data set that in turn suffers from a high variance.
Formal Synthesis of Lyapunov Neural Networks
We employ a counterexample-guided approach where a numerical learner and a symbolic verifier interact to construct provably correct Lyapunov neural networks (LNNs).
Cautious Reinforcement Learning with Logical Constraints
This paper presents the concept of an adaptive safe padding that forces Reinforcement Learning (RL) to synthesise optimal control policies while ensuring safety during the learning process.
Safety Guarantees for Planning Based on Iterative Gaussian Processes
Gaussian Processes (GPs) are widely employed in control and learning because of their principled treatment of uncertainty.
DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning
This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives.
Modular Deep Reinforcement Learning with Temporal Logic Specifications
We propose an actor-critic, model-free, and online Reinforcement Learning (RL) framework for continuous-state continuous-action Markov Decision Processes (MDPs) when the reward is highly sparse but encompasses a high-level temporal structure.
Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees
Reinforcement Learning (RL) has emerged as an efficient method of choice for solving complex sequential decision making problems in automatic control, computer science, economics, and biology.
Certified Reinforcement Learning with Logic Guidance
Reinforcement Learning (RL) is a widely employed machine learning architecture that has been applied to a variety of control problems.
Logically-Constrained Neural Fitted Q-Iteration
We propose a method for efficient training of Q-functions for continuous-state Markov Decision Processes (MDPs) such that the traces of the resulting policies satisfy a given Linear Temporal Logic (LTL) property.
Logically-Constrained Reinforcement Learning
With this reward function, the policy synthesis procedure is "constrained" by the given specification.
Safe Policy Search with Gaussian Process Models
We propose a method to optimise the parameters of a policy which will be used to safely perform a given task in a data-efficient manner.
Automated Experiment Design for Data-Efficient Verification of Parametric Markov Decision Processes
We present a new method for statistical verification of quantitative properties over a partially unknown system with actions, utilising a parameterised model (in this work, a parametric Markov decision process) and data collected from experiments performed on the underlying system.
Sampling-based Approximations with Quantitative Performance for the Probabilistic Reach-Avoid Problem over General Markov Processes
This article deals with stochastic processes endowed with the Markov (memoryless) property and evolving over general (uncountable) state spaces.
