Search Results for author:
Found 16 papers, 10 papers with code
Model evaluation for extreme risks
Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities.
Formalizing the presumption of independence
Mathematical proof aims to deliver confident conclusions, but a very similar process of deduction can be used to make uncertain estimates that are open to revision.
Training language models to follow instructions with human feedback
In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback.
Recursively Summarizing Books with Human Feedback
Our human labelers are able to supervise and evaluate the models quickly, despite not having read the entire books themselves.
Learning to summarize from human feedback
We collect a large, high-quality dataset of human comparisons between summaries, train a model to predict the human-preferred summary, and use that model as a reward function to fine-tune a summarization policy using reinforcement learning.
Fine-Tuning Language Models from Human Preferences
Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and we believe reward learning for language is a key to making RL practical and safe for real-world tasks.
Supervising strong learners by amplifying weak experts
Many real world learning tasks involve complex or hard-to-specify objectives, and using an easier-to-specify proxy can lead to poor performance or misaligned behavior.
Unrestricted Adversarial Examples
We introduce a two-player contest for evaluating the safety and robustness of machine learning systems, with a large prize pool.
AI safety via debate
To make AI systems broadly useful for challenging real-world tasks, we need them to learn complex human goals and preferences.
Deep reinforcement learning from human preferences
For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems.
A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
In particular, we demonstrate an equivalence between a sample-based algorithm for maximum entropy IRL and a GAN in which the generator's density can be evaluated and is provided as an additional input to the discriminator.
Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
Nevertheless, often the overall gist of what the policy does in simulation remains valid in the real world.
Concrete Problems in AI Safety
Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Collaborative prediction with expert advice
A case in point is the foundational learning problem of prediction with expert advice.
Online Local Learning via Semidefinite Programming
In many online learning problems we are interested in predicting local information about some universe of items.
