Search Results for author:
Found 18 papers, 2 papers with code
Partial Identifiability in Inverse Reinforcement Learning For Agents With Non-Exponential Discounting
However, work so far has only considered agents that discount future reward exponentially: this is a serious limitation, especially given that extensive work in the behavioural sciences suggests that humans are better modelled as discounting hyperbolically.
Partial Identifiability and Misspecification in Inverse Reinforcement Learning
We also provide necessary and sufficient conditions that describe precisely how the observed demonstrator policy may differ from each of the standard behavioural models before that model leads to faulty inferences about the reward function $R$.
The Perils of Optimizing Learned Reward Functions: Low Training Error Does Not Guarantee Low Regret
We say that such a reward model has an error-regret mismatch.
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts.
Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification
In addition to this, we also characterise the conditions under which a behavioural model is robust to small perturbations of the observed policy, and we analyse how robust many behavioural models are to misspecification of their parameter values (such as e. g.\ the discount rate).
On the Limitations of Markovian Rewards to Express Multi-Objective, Risk-Sensitive, and Modal Tasks
Moreover, we find that scalar, Markovian rewards are unable to express most of the instances in each of these three classes.
On The Expressivity of Objective-Specification Formalisms in Reinforcement Learning
However, it is well-known that certain tasks cannot be expressed by means of an objective in the Markov rewards formalism, motivating the study of alternative objective-specification formalisms in RL such as Linear Temporal Logic and Multi-Objective Reinforcement Learning.
Multi-Objective Reinforcement Learning
reinforcement-learning
+1
Goodhart's Law in Reinforcement Learning
First, we propose a way to quantify the magnitude of this effect and show empirically that optimising an imperfect proxy reward often leads to the behaviour predicted by Goodhart's law for a wide range of environments and reward functions.
STARC: A General Framework For Quantifying Differences Between Reward Functions
This means that reward learning algorithms generally must be evaluated empirically, which is expensive, and that their failure modes are difficult to anticipate in advance.
Lexicographic Multi-Objective Reinforcement Learning
In this work we introduce reinforcement learning techniques for solving lexicographic multi-objective problems.
Multi-Objective Reinforcement Learning
reinforcement-learning
+1
Misspecification in Inverse Reinforcement Learning
In this paper, we provide a mathematical analysis of how robust different IRL models are to misspecification, and answer precisely how the demonstrator policy may differ from each of the standard models before that model leads to faulty inferences about the reward function $R$.
Defining and Characterizing Reward Hacking
We provide the first formal definition of reward hacking, a phenomenon where optimizing an imperfect proxy reward function, $\mathcal{\tilde{R}}$, leads to poor performance according to the true reward function, $\mathcal{R}$.
Invariance in Policy Optimisation and Partial Identifiability in Reward Learning
It is often very challenging to manually design reward functions for complex, real-world tasks.
Reinforcement Learning in Newcomblike Environments
This gives us a powerful tool for reasoning about the limit behaviour of agents -- for example, it lets us show that there are Newcomblike environments in which a reinforcement learning agent cannot converge to any optimal policy.
A General Counterexample to Any Decision Theory and Some Responses
In this paper I present an argument and a general schema which can be used to construct a problem case for any decision theory, in a way that could be taken to show that one cannot formulate a decision theory that is never outperformed by any other decision theory.
Is SGD a Bayesian sampler? Well, almost
Our main findings are that $P_{SGD}(f\mid S)$ correlates remarkably well with $P_B(f\mid S)$ and that $P_B(f\mid S)$ is strongly biased towards low-error and low complexity functions.
Neural networks are a priori biased towards Boolean functions with low entropy
Understanding the inductive bias of neural networks is critical to explaining their ability to generalise.
Risks from Learned Optimization in Advanced Machine Learning Systems
We analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer - a situation we refer to as mesa-optimization, a neologism we introduce in this paper.
