Search Results for author:
Found 13 papers, 3 papers with code
Bridging the Data Provenance Gap Across Text, Speech and Video
In this work we conduct the largest and first-of-its-kind longitudinal audit across modalities--popular text, speech, and video datasets--from their detailed sourcing trends and use restrictions to their geographical and linguistic representation.
Language model developers should report train-test overlap
Overall, we take the position that language model developers should publish train-test overlap statistics and/or training data whenever they report evaluation results on public test sets.
Acceptable Use Policies for Foundation Models
This paper identifies acceptable use policies from 30 foundation model developers, analyzes the use restrictions they contain, and argues that acceptable use policies are an important lens for understanding the regulation of foundation models.
Consent in Crisis: The Rapid Decline of the AI Data Commons
To our knowledge, we conduct the first, large-scale, longitudinal audit of the consent protocols for the web domains underlying AI training corpora.
The Foundation Model Transparency Index v1.1: May 2024
To characterize the status quo, the Foundation Model Transparency Index was launched in October 2023 to measure the transparency of leading foundation model developers.
AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies
However, existing public benchmarks often define safety categories based on previous literature, intuitions, or common sense, leading to disjointed sets of categories for risks specified in recent regulations and policies, which makes it challenging to evaluate and compare FMs across these benchmarks.
AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies
We present a comprehensive AI risk taxonomy derived from eight government policies from the European Union, United States, and China and 16 company policies worldwide, making a significant step towards establishing a unified language for generative AI safety evaluation.
The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources
Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications.
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
A Safe Harbor for AI Evaluation and Red Teaming
Independent evaluation and red teaming are critical for identifying the risks posed by generative AI systems.
On the Societal Impact of Open Foundation Models
To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk.
Foundation Model Transparency Reports
Foundation models are critical digital technologies with sweeping societal impact that necessitates transparency.
The Foundation Model Transparency Index
We score 10 major foundation model developers (e. g. OpenAI, Google, Meta) against the 100 indicators to assess their transparency.
