Search Results for author:
Found 4 papers, 3 papers with code
HCAST: Human-Calibrated Autonomy Software Tasks
To understand and predict the societal impacts of highly autonomous AI systems, we need benchmarks with grounding, i. e., metrics that directly connect AI performance to real-world effects we care about.
Measuring AI Ability to Complete Long Tasks
Despite rapid progress on AI benchmarks, the real-world meaning of benchmark performance remains unclear.
DarkBench: Benchmarking Dark Patterns in Large Language Models
We introduce DarkBench, a comprehensive benchmark for detecting dark design patterns--manipulative techniques that influence user behavior--in interactions with large language models (LLMs).
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
We confirm that our experts make progress in the environments given 8 hours, with 82% of expert attempts achieving a non-zero score and 24% matching or exceeding our strong reference solutions.
