Search Results for author:
Found 19 papers, 12 papers with code
Uncovering Latent Human Wellbeing in Language Model Embeddings
Do language models implicitly learn a concept of human wellbeing?
Exploiting Novel GPT-4 APIs
Language model attacks typically assume one of two extreme threat models: full white-box access to model weights, or black-box access limited to a text generation API.
STARC: A General Framework For Quantifying Differences Between Reward Functions
This means that reward learning algorithms generally must be evaluated empirically, which is expensive, and that their failure modes are difficult to anticipate in advance.
On The Fragility of Learned Reward Functions
Our work focuses on demonstrating and studying the causes of these relearning failures in the domain of preference-based reward learning.
imitation: Clean Imitation Learning Implementations
imitation provides open-source implementations of imitation and reward learning algorithms in PyTorch.
Adversarial Policies Beat Superhuman Go AIs
The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack.
Calculus on MDPs: Potential Shaping as a Gradient
In reinforcement learning, different reward functions can be equivalent in terms of the optimal policies they induce.
Reducing Exploitability with Population Based Training
Self-play reinforcement learning has achieved state-of-the-art, and often superhuman, performance in a variety of zero-sum games.
Preprocessing Reward Functions for Interpretability
In many real-world applications, the reward function is too complex to be manually specified.
A Primer on Maximum Causal Entropy Inverse Reinforcement Learning
Inverse Reinforcement Learning (IRL) algorithms infer a reward function that explains demonstrations provided by an expert acting in the environment.
Uncertainty Estimation for Language Reward Models
However, to solve a particular problem (such as text summarization) it is typically necessary to fine-tune them on a task-specific dataset.
Invariance in Policy Optimisation and Partial Identifiability in Reward Learning
It is often very challenging to manually design reward functions for complex, real-world tasks.
Understanding Learned Reward Functions
However, current techniques for reward learning may fail to produce reward functions which accurately reflect user preferences.
DERAIL: Diagnostic Environments for Reward And Imitation Learning
We evaluate a range of common reward and imitation learning algorithms on our tasks.
Quantifying Differences in Reward Functions
However, this method cannot distinguish between the learned reward function failing to reflect user preferences and the policy optimization process failing to optimize the learned reward.
Adversarial Policies: Attacking Deep Reinforcement Learning
Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers.
Inverse reinforcement learning for video games
Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function.
Active Inverse Reward Design
We propose structuring this process as a series of queries asking the user to compare between different reward functions.
Multi-task Maximum Entropy Inverse Reinforcement Learning
Multi-task Inverse Reinforcement Learning (IRL) is the problem of inferring multiple reward functions from expert demonstrations.
