Search Results for author:
Found 16 papers, 2 papers with code
A Proof Strategy Language and Proof Script Generation for Isabelle/HOL
We introduce a language, PSL, designed to capture high level proof strategies in Isabelle/HOL.
Logic in Computer Science
TacticToe: Learning to Prove with Tactics
We implement a automated tactical prover TacticToe on top of the HOL4 interactive theorem prover.
Penalizing side effects using stepwise relative reachability
How can we design safe reinforcement learning agents that avoid unnecessary disruptions to their environment?
Modeling AGI Safety Frameworks with Causal Influence Diagrams
Proposals for safe AGI systems are typically made at the level of frameworks, specifying how the components of the proposed system should be trained and interact with each other.
Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspective
Can humans get arbitrarily capable reinforcement learning (RL) agents to do their bidding?
Scalable Neural Learning for Verifiable Consistency with Temporal Specifications
Formal verification of machine learning models has attracted attention recently, and significant progress has been made on proving simple properties like robustness to small perturbations of the input features.
Avoiding Tampering Incentives in Deep RL via Decoupled Approval
How can we design agents that pursue a given objective when all feedback mechanisms are influenceable by the agent?
REALab: An Embedded Perspective on Tampering
Standard Markov Decision Process (MDP) formulations of RL and simulated environments mirroring the MDP structure assume secure access to feedback (e. g., rewards).
Formal Methods for the Informal Engineer: Workshop Recommendations
Formal Methods for the Informal Engineer (FMIE) was a workshop held at the Broad Institute of MIT and Harvard in 2021 to explore the potential role of verified software in the biomedical software ecosystem.
Safe Deep RL in 3D Environments using Human Feedback
In this paper we answer this question in the affirmative, using ReQueST to train an agent to perform a 3D first-person object collection task using data entirely from human contractors.
Discovering Agents
Causal models of agents have been used to analyse the safety aspects of machine learning systems.
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
However, an AI system may pursue an undesired goal even when the specification is correct, in the case of goal misgeneralization.
Solving math word problems with process- and outcome-based feedback
Recent work has shown that asking language models to generate reasoning steps improves performance on many reasoning tasks.
Ranked #30 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Scaling Goal-based Exploration via Pruning Proto-goals
One of the gnarliest challenges in reinforcement learning (RL) is exploration that scales to vast domains, where novelty-, or coverage-seeking behaviour falls short.
Explaining grokking through circuit efficiency
One of the most surprising puzzles in neural network generalisation is grokking: a network with perfect training accuracy but poor generalisation will, upon further training, transition to perfect generalisation.
Evaluating Frontier Models for Dangerous Capabilities
To understand the risks posed by a new AI system, we must understand what it can and cannot do.
