Search Results for author:
Found 65 papers, 41 papers with code
MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models
In this paper, we present the first unified platform, MMDT (Multimodal DecodingTrust), designed to provide a comprehensive safety and trustworthiness evaluation for MMFMs.
Superintelligence Strategy: Expert Version
Rapid advances in AI are beginning to reshape national security.
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems
Across a diverse set of LLMs, we find that while larger models obtain higher accuracy on our benchmark, they do not become more honest.
Beyond Release: Access Considerations for Generative AI Systems
Generative AI release decisions determine whether system components are made available, but release does not address many other elements that change how users and stakeholders are able to engage with a system.
EnigmaEval: A Benchmark of Long Multimodal Reasoning Challenges
As language models master existing reasoning benchmarks, we need new challenges to evaluate their cognitive frontiers.
Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs
To study these emergent value systems, we propose utility engineering as a research agenda, comprising both the analysis and control of AI utilities.
Humanity's Last Exam
However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities.
Ranked #4 on
Humanity's Last Exam
on Humanity's Last Exam
Introduction to AI Safety, Ethics, and Society
Artificial Intelligence is rapidly embedding itself within militaries, economies, and societies, reshaping their very foundations.
AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
The robustness of LLMs to jailbreak attacks, where users design prompts to circumvent safety measures and misuse model capabilities, has been studied primarily for LLMs acting as simple chatbots.
LLM-PBE: Assessing Data Privacy in Large Language Models
Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis.
Tamper-Resistant Safeguards for Open-Weight LLMs
Rapid advances in the capabilities of large language models (LLMs) have raised widespread concerns regarding their potential for malicious use.
Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
As artificial intelligence systems grow more powerful, there has been increasing interest in "AI safety" research to address emerging and future risks.
Improving Alignment and Robustness with Circuit Breakers
Existing techniques aimed at improving alignment, such as refusal training, are often bypassed.
Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression
While state-of-the-art (SoTA) compression methods boast impressive advancements in preserving benign task performance, the potential risks of compression in terms of safety and trustworthiness have been largely neglected.
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
To measure these risks of malicious use, government institutions and major AI labs are developing evaluations for hazardous capabilities in LLMs.
Uncovering Latent Human Wellbeing in Language Model Embeddings
Do language models implicitly learn a concept of human wellbeing?
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods.
Can LLMs Follow Simple Rules?
As Large Language Models (LLMs) are deployed with increasing real-world responsibilities, it is important to be able to specify and constrain the behavior of these systems in a reliable manner.
Representation Engineering: A Top-Down Approach to AI Transparency
In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience.
Ranked #3 on
Question Answering
on TruthfulQA
Identifying and Mitigating the Security Risks of Generative AI
However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks.
AI Deception: A Survey of Examples, Risks, and Potential Solutions
This paper argues that a range of current AI systems have learned how to deceive humans.
An Overview of Catastrophic AI Risks
Rapid advancements in artificial intelligence (AI) have sparked growing concerns among experts, policymakers, and world leaders regarding the potential for increasingly advanced AI systems to pose catastrophic risks.
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications such as healthcare and finance -- where mistakes can be costly.
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
And how do we measure these behaviors in general-purpose models such as GPT-4?
Natural Selection Favors AIs over Humans
Evolution endowed humans with high intelligence, which allowed us to become one of the most successful species on the planet.
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding
Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets.
How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios
In experiments, we show how video models that are primarily trained to recognize actions and find contours of objects can be repurposed to understand human preferences and the emotional content of videos.
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature.
Forecasting Future World Events with Neural Networks
We test language models on our forecasting task and find that performance is far below a human expert baseline.
Actionable Guidance for High-Consequence AI Risk Management: Towards Standards Addressing AI Catastrophic Risks
In this document, we provide detailed actionable-guidance recommendations focused on identifying and managing risks of events with very high or catastrophic consequences, intended as a risk management practices resource for NIST for AI RMF version 1. 0 (released in January 2023), or for AI RMF users, or for other AI risk management guidance and standards as appropriate.
X-Risk Analysis for AI Research
Artificial intelligence (AI) has the potential to greatly improve society, but as with any powerful technology, it comes with heightened risks and responsibilities.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy.
Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselines
To alleviate this issue, we propose a novel data augmentation scheme, FourierMix, that produces augmentations to improve the spectral coverage of the training data.
A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
To date, several research domains tackle the problem of detecting unfamiliar samples, including anomaly detection, novelty detection, one-class learning, open set recognition, and out-of-distribution detection.
What Would Jiminy Cricket Do? Towards Agents That Behave Morally
When making everyday decisions, people are guided by their conscience, an internal sense of right and wrong.
Improving and Assessing Anomaly Detectors for Large-Scale Settings
We conduct extensive experiments in these more realistic settings for out-of-distribution detection and find that a surprisingly simple detector based on the maximum logit outperforms prior methods in all the large-scale multi-class, multi-label, and segmentation tasks, establishing a simple new baseline for future work.
Unsolved Problems in ML Safety
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings.
VisDA-2021 Competition Universal Domain Adaptation to Improve Performance on Out-of-Distribution Data
Progress in machine learning is typically measured by training and testing a model on the same distribution of data, i. e., the same domain.
Measuring Coding Challenge Competence With APPS
Recent models such as GPT-Neo can pass approximately 20% of the test cases of introductory problems, so we find that machine learning models are now beginning to learn how to code.
Ranked #14 on
Code Generation
on APPS
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
We address this bottleneck within the legal domain by introducing the Contract Understanding Atticus Dataset (CUAD), a new dataset for legal contract review.
Measuring Mathematical Problem Solving With the MATH Dataset
To facilitate future research and increase accuracy on MATH, we also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics.
Ranked #121 on
Math Word Problem Solving
on MATH
A Rigorous Evaluation of Real-World Distribution Shifts
Motivated by this, we introduce a new data augmentation method which advances the state-of-the-art and outperforms models pretrained with 1000x more labeled data.
Towards Machine Ethics with Language Models
We show how to assess a language model’s knowledge of basic concepts of morality.
How Multipurpose Are Language Models?
By comprehensively evaluating the breadth and depth of a model's academic and professional understanding, our test can be used to analyze models across many tasks and to identify important shortcomings.
Measuring Massive Multitask Language Understanding
By comprehensively evaluating the breadth and depth of a model's academic and professional understanding, our test can be used to analyze models across many tasks and to identify important shortcomings.
Ranked #32 on
Multi-task Language Understanding
on MMLU
Aligning AI With Shared Human Values
We show how to assess a language model's knowledge of basic concepts of morality.
Ranked #1 on
Average
on hendrycks2020ethics
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
We find that using larger models and artificial data augmentations can improve robustness on real-world distribution shifts, contrary to claims in prior work.
Ranked #29 on
Domain Generalization
on ImageNet-R
Pretrained Transformers Improve Out-of-Distribution Robustness
Although pretrained Transformers such as BERT achieve high accuracy on in-distribution examples, do they generalize to new distributions?
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
We propose AugMix, a data processing technique that is simple to implement, adds limited computational overhead, and helps models withstand unforeseen corruptions.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Scaling Out-of-Distribution Detection for Real-World Settings
We conduct extensive experiments in these more realistic settings for out-of-distribution detection and find that a surprisingly simple detector based on the maximum logit outperforms prior methods in all the large-scale multi-class, multi-label, and segmentation tasks, establishing a simple new baseline for future work.
Testing Robustness Against Unforeseen Adversaries
To narrow in on this discrepancy between research and reality we introduce ImageNet-UA, a framework for evaluating model robustness against a range of unforeseen adversaries, including eighteen new non-L_p attacks.
Natural Adversarial Examples
We also curate an adversarial out-of-distribution detection dataset called ImageNet-O, which is the first out-of-distribution detection dataset created for ImageNet models.
Ranked #39 on
Domain Generalization
on ImageNet-A
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
Self-supervision provides effective representations for downstream tasks without requiring labels.
Transfer of Adversarial Robustness Between Perturbation Types
We study the transfer of adversarial robustness of deep neural networks between different perturbation types.
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations.
Ranked #41 on
Domain Generalization
on ImageNet-C
Using Pre-Training Can Improve Model Robustness and Uncertainty
He et al. (2018) have called into question the utility of pre-training by showing that training from scratch can often yield similar performance to pre-training.
Deep Anomaly Detection with Outlier Exposure
We also analyze the flexibility and robustness of Outlier Exposure, and identify characteristics of the auxiliary dataset that improve performance.
Ranked #3 on
Out-of-Distribution Detection
on CIFAR-100
(using extra training data)
Open Category Detection with PAC Guarantees
Further, while there are algorithms for open category detection, there are few empirical results that directly report alien detection rates.
Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
Then we propose a new dataset called Icons-50 which opens research on a new kind of robustness, surface variation robustness.
Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
We utilize trusted data by proposing a loss correction technique that utilizes trusted examples in a data-efficient manner to mitigate the effects of label noise on deep neural network classifiers.
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
We consider the two related problems of detecting if an example is misclassified or out-of-distribution.
Ranked #13 on
Out-of-Distribution Detection
on CIFAR-10 vs CIFAR-100
Early Methods for Detecting Adversarial Images
Many machine learning classifiers are vulnerable to adversarial perturbations.
Adjusting for Dropout Variance in Batch Normalization and Weight Initialization
We show how to adjust for the variance introduced by dropout with corrections to weight initialization and Batch Normalization, yielding higher accuracy.
Gaussian Error Linear Units (GELUs)
We propose the Gaussian Error Linear Unit (GELU), a high-performing neural network activation function.
