Search Results for author:
Found 22 papers, 7 papers with code
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).
Towards Measuring the Representation of Subjective Global Opinions in Language Models
We first build a dataset, GlobalOpinionQA, comprised of questions and answers from cross-national surveys designed to capture diverse opinions on global issues across different countries.
Model evaluation for extreme risks
Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities.
Regulatory Markets: The Future of AI Governance
Appropriately regulating artificial intelligence is an increasingly urgent policy challenge.
The Capacity for Moral Self-Correction in Large Language Models
We believe that at this level of scale, language models obtain two capabilities that they can use for moral self-correction: (1) they can follow instructions and (2) they can learn complex normative concepts of harm like stereotyping, bias, and discrimination.
Discovering Language Model Behaviors with Model-Written Evaluations
We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse.
In-context Learning and Induction Heads
In this work, we present preliminary and indirect evidence for a hypothesis that induction heads might constitute the mechanism for the majority of all "in-context learning" in large transformer models (i. e. decreasing loss at increasing token indices).
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
We provide our own analysis of the data and find a variety of harmful outputs, which range from offensive language to more subtly harmful non-violent unethical outputs.
Language Models (Mostly) Know What They Know
We study whether language models can evaluate the validity of their own claims and predict which questions they will be able to answer correctly.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
We apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models to act as helpful and harmless assistants.
A General Language Assistant as a Laboratory for Alignment
We find that the benefits from modest interventions increase with model size, generalize to a variety of alignment evaluations, and do not compromise the performance of large models.
Why and How Governments Should Monitor AI Development
In this paper we outline a proposal for improving the governance of artificial intelligence (AI) by investing in government capacity to systematically measure and monitor the capabilities and impacts of AI systems.
Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications
Recently, there have been breakthroughs in computer vision ("CV") models that are more generalizable with the advent of models such as CLIP and ALIGN.
Learning Transferable Visual Models From Natural Language Supervision
State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories.
Ranked #1 on
Zero-Shot Learning
on COCO-MLT
(using extra training data)
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
On October 14th, 2020, researchers from OpenAI, the Stanford Institute for Human-Centered Artificial Intelligence, and other universities convened to discuss open research questions surrounding GPT-3, the largest publicly-disclosed dense language model at the time.
Language Models are Few-Shot Learners
By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
Ranked #1 on
Question Answering
on CoQA
(Overall metric)
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development.
Computers and Society
Release Strategies and the Social Impacts of Language Models
Large language models have a range of beneficial uses: they can assist in prose, poetry, and programming; analyze dataset biases; and more.
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats.
