Search Results for author:
Found 14 papers, 4 papers with code
Interpreting Emergent Planning in Model-Free Reinforcement Learning
We present the first mechanistic evidence that model-free reinforcement learning agents can learn to plan.
The Reality of AI and Biorisk
To accurately and confidently answer the question 'could an AI model or system increase biorisk', it is necessary to have both a sound theoretical threat model for how AI models or systems could increase biorisk and a robust method for testing that threat model.
Learning to Forget using Hypernetworks
Machine unlearning is gaining increasing attention as a way to remove adversarial data poisoning attacks from already trained models and to comply with privacy and AI regulations.
Comparing Bottom-Up and Top-Down Steering Approaches on In-Context Learning Tasks
A key objective of interpretability research on large language models (LLMs) is to develop methods for robustly steering models toward desired behaviors.
Noisy Zero-Shot Coordination: Breaking The Common Knowledge Assumption In Zero-Shot Coordination Games
We show that with NZSC training, RL agents can be trained to coordinate well with novel partners even when the (exact) problem setting of the coordination is not common knowledge.
Adversarial Robustness of In-Context Learning in Transformers for Linear Regression
This work investigates the vulnerability of in-context learning in transformers to \textit{hijacking attacks} focusing on the setting of linear regression tasks.
IDs for AI Systems
AI systems are increasingly pervasive, yet information needed to decide whether and how to engage with them may not exist or be accessible.
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs).
Reward Model Ensembles Help Mitigate Overoptimization
Gao et al. (2023) studied this phenomenon in a synthetic human feedback setup with a significantly larger "gold" reward model acting as the true reward (instead of humans) and showed that overoptimization remains a persistent problem regardless of the size of the proxy reward model and training data used.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Domain Generalization for Robust Model-Based Offline Reinforcement Learning
Existing offline reinforcement learning (RL) algorithms typically assume that training data is either: 1) generated by a known policy, or 2) of entirely unknown origin.
Constrained Reinforcement Learning With Learned Constraints
In this work, given a reward function and a set of demonstrations from an expert that maximizes this reward function while respecting \textit{unknown} constraints, we propose a framework to learn the most likely constraints that the expert respects.
Inverse Constrained Reinforcement Learning
However, for the real world deployment of reinforcement learning (RL), it is critical that RL agents are aware of these constraints, so that they can act safely.
Learning To Solve Differential Equations Across Initial Conditions
Recently, there has been a lot of interest in using neural networks for solving partial differential equations.
