Search Results for author:
Found 18 papers, 11 papers with code
Training language models to follow instructions with human feedback
In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Fine-Tuning Language Models from Human Preferences
Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and we believe reward learning for language is a key to making RL practical and safe for real-world tasks.
Learning to summarize from human feedback
We collect a large, high-quality dataset of human comparisons between summaries, train a model to predict the human-preferred summary, and use that model as a reward function to fine-tune a summarization policy using reinforcement learning.
Unrestricted Adversarial Examples
We introduce a two-player contest for evaluating the safety and robustness of machine learning systems, with a large prize pool.
Deep reinforcement learning from human preferences
For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems.
A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
In particular, we demonstrate an equivalence between a sample-based algorithm for maximum entropy IRL and a GAN in which the generator's density can be evaluated and is provided as an additional input to the discriminator.
AI safety via debate
To make AI systems broadly useful for challenging real-world tasks, we need them to learn complex human goals and preferences.
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).
Supervising strong learners by amplifying weak experts
Many real world learning tasks involve complex or hard-to-specify objectives, and using an easier-to-specify proxy can lead to poor performance or misaligned behavior.
Concrete Problems in AI Safety
Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society.
Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
Nevertheless, often the overall gist of what the policy does in simulation remains valid in the real world.
Collaborative prediction with expert advice
A case in point is the foundational learning problem of prediction with expert advice.
Online Local Learning via Semidefinite Programming
In many online learning problems we are interested in predicting local information about some universe of items.
Recursively Summarizing Books with Human Feedback
Our human labelers are able to supervise and evaluate the models quickly, despite not having read the entire books themselves.
Formalizing the presumption of independence
Mathematical proof aims to deliver confident conclusions, but a very similar process of deduction can be used to make uncertain estimates that are open to revision.
Model evaluation for extreme risks
Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities.
Evaluating Language-Model Agents on Realistic Autonomous Tasks
We find that these language model agents can only complete the easiest tasks from this list, although they make some progress on the more challenging tasks.
