Search Results for author:
Found 16 papers, 3 papers with code
Active Learning of Mealy Machines with Timers
However, whereas Waga needs exponentially many concrete queries to implement a single symbolic query, we only need a polynomial number.
Synthesis of Hierarchical Controllers Based on Deep Reinforcement Learning Policies
We propose a novel approach to the problem of controller design for environments modeled as Markov decision processes (MDPs).
Formally-Sharp DAgger for MCTS: Lower-Latency Monte Carlo Tree Search using Data Aggregation with Formal Methods
In particular, we use model-checking techniques to guide the MCTS algorithm in order to generate offline samples of high-quality decisions on a representative set of states of the MDP.
Graph-Based Reductions for Parametric and Weighted MDPs
In terms of computational complexity, we establish that determining whether p is never worse than q is coETR-complete.
Wasserstein Auto-encoded MDPs: Formal Verification of Efficiently Distilled RL Policies with Many-sided Guarantees
Our approach yields bisimulation guarantees while learning the distilled policy, allowing concrete optimization of the abstraction and representation model quality.
The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Models
Maintaining a probability distribution that models the belief over what the true state is can be used as a sufficient statistic of the history, but its computation requires access to the model of the environment and is often intractable.
Distillation of RL Policies with Formal Guarantees via Variational Abstraction of Markov Decision Processes (Technical Report)
Finally, we show how one can use a policy obtained via state-of-the-art RL to efficiently train a variational autoencoder that yields a discrete latent model with provably approximately correct bisimulation guarantees.
Continuous One-Counter Automata
Our three main results are as follows: (1) We prove that the reachability problem for COCA with global upper and lower bound tests is in NC2; (2) that, in general, the problem is decidable in polynomial time; and (3) that it is decidable in the polynomial hierarchy for COCA with parametric counter updates and bound tests.
Formal Languages and Automata Theory Logic in Computer Science
Safe Learning for Near Optimal Scheduling
In this paper, we investigate the combination of synthesis, model-based learning, and online sampling techniques to obtain safe and near-optimal schedulers for a preemptible task scheduling problem.
Robustness Verification for Classifier Ensembles
The robustness-checking problem consists of assessing, given a set of classifiers and a labelled data set, whether there exists a randomized attack that induces a certain expected loss against all classifiers.
Revisiting Parameter Synthesis for One-Counter Automata
We study the (parameter) synthesis problem for one-counter automata with parameters.
Logic in Computer Science Formal Languages and Automata Theory
Let's Agree to Degree: Comparing Graph Convolutional Networks in the Message-Passing Framework
In this paper we cast neural networks defined on graphs as message-passing neural networks (MPNNs) in order to study the distinguishing power of different classes of such models.
The Impatient May Use Limited Optimism to Minimize Regret
Discounted-sum games provide a formal model for the study of reinforcement learning, where the agent is enticed to get rewards early since later rewards are discounted.
On the Complexity of Value Iteration
We show that, given a horizon $n$ in binary and an MDP, computing an optimal policy is EXP-complete, thus resolving an open problem that goes back to the seminal 1987 paper on the complexity of MDPs by Papadimitriou and Tsitsiklis.
Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints
Assuming the support of the unknown transition function and a lower bound on the minimal transition probability are known in advance, we show that in MDPs consisting of a single end component, two combinations of guarantees on the parity and mean-payoff objectives can be achieved depending on how much memory one is willing to use.
Optimizing Expectation with Guarantees in POMDPs (Technical Report)
In this work we go beyond both the "expectation" and "threshold" approaches and consider a "guaranteed payoff optimization (GPO)" problem for POMDPs, where we are given a threshold $t$ and the objective is to find a policy $\sigma$ such that a) each possible outcome of $\sigma$ yields a discounted-sum payoff of at least $t$, and b) the expected discounted-sum payoff of $\sigma$ is optimal (or near-optimal) among all policies satisfying a).
