Search Results for author:
Found 12 papers, 0 papers with code
CoinRun: Solving Goal Misgeneralisation
Goal misgeneralisation is a key challenge in AI alignment -- the task of getting powerful Artificial Intelligences to align their goals with human intentions and human morality.
Concept Extrapolation: A Conceptual Primer
This article is a primer on concept extrapolation - the ability to take a concept, a feature, or a goal that is defined in one context and extrapolate it safely to a more general context.
Recognising the importance of preference change: A call for a coordinated multidisciplinary research effort in the age of AI
We operationalize preference to incorporate concepts from various disciplines, outlining the importance of meta-preferences and preference-change preferences, and proposing a preliminary framework for how preferences change.
The dangers in algorithms learning humans' values and irrationalities
For an artificial intelligence (AI) to be aligned with human values (or human preferences), it must first learn those values.
Chess as a Testing Grounds for the Oracle Approach to AI Safety
To reduce the danger of powerful super-intelligent AIs, we might make the first such AIs oracles that can only send and receive messages.
Pitfalls of learning a reward function online
We formally introduce two desirable properties: the first is `unriggability', which prevents the agent from steering the learning process in the direction of a reward function that is easier to optimise.
Counterfactual equivalence for POMDPs, and underlying deterministic environments
Partially Observable Markov Decision Processes (POMDPs) are rich environments often used in machine learning.
'Indifference' methods for managing agent rewards
`Indifference' refers to a class of methods used to control reward based agents.
Occam's razor is insufficient to infer the preferences of irrational agents
Inverse reinforcement learning (IRL) attempts to infer human rewards or preferences from observed behavior.
Good and safe uses of AI Oracles
It is possible that powerful and potentially dangerous artificial intelligence (AI) might be developed in the future.
Low Impact Artificial Intelligences
This paper looks at an alternative approach: defining a general concept of `low impact'.
Anthropic decision theory
This paper sets out to resolve how agents ought to act in the Sleeping Beauty problem and various related anthropic (self-locating belief) problems, not through the calculation of anthropic probabilities, but through finding the correct decision to make.
