Search Results for author:
Found 19 papers, 10 papers with code
An Approach to Technical AGI Safety and Security
Second, system-level security measures such as monitoring and access control can mitigate harm even if the model is misaligned.
Observation Interference in Partially Observable Assistance Games
We show that this incentive for interference goes away if the human is playing optimally, or if we introduce a communication channel for the human to communicate their preferences to the assistant.
Obfuscated Activations Bypass LLM Latent-Space Defenses
Recent latent-space monitoring techniques have shown promise as defenses against LLM attacks.
The Partially Observable Off-Switch Game
Unlike when the human has full observability, we find that in optimal play, even AI agents assisting perfectly rational humans sometimes avoid shutdown.
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
These results stand in stark contrast to existing evidence of universal and transferable text jailbreaks against language models and transferable adversarial attacks against image classifiers, suggesting that VLMs may be more robust to gradient-based transfer attacks.
When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback
Past analyses of reinforcement learning from human feedback (RLHF) assume that the human evaluators fully observe the environment.
Uncovering Latent Human Wellbeing in Language Model Embeddings
Do language models implicitly learn a concept of human wellbeing?
A StrongREJECT for Empty Jailbreaks
To create a benchmark, researchers must choose a dataset of forbidden prompts to which a victim model will respond, along with an evaluation method that scores the harmfulness of the victim model's responses.
ALMANACS: A Simulatability Benchmark for Language Model Explainability
ALMANACS scores explainability methods on simulatability, i. e., how well the explanations improve behavior prediction on new inputs.
Image Hijacks: Adversarial Images can Control Generative Models at Runtime
In this work, we focus on the image input to a vision-language model (VLM).
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
And how do we measure these behaviors in general-purpose models such as GPT-4?
imitation: Clean Imitation Learning Implementations
imitation provides open-source implementations of imitation and reward learning algorithms in PyTorch.
For Learning in Symmetric Teams, Local Optima are Global Nash Equilibria
In this work, we show that any locally optimal symmetric strategy profile is also a (global) Nash equilibrium.
An Empirical Investigation of Representation Learning for Imitation
We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation across several environment suites.
RvS: What is Essential for Offline RL via Supervised Learning?
Recent work has shown that supervised learning alone, without temporal difference (TD) learning, can be remarkably effective for offline RL.
The Essential Elements of Offline RL via Supervised Learning
These methods, which we collectively refer to as reinforcement learning via supervised learning (RvS), involve a number of design decisions, such as policy architectures and how the conditioning variable is constructed.
Sparse Graphical Memory for Robust Planning
To operate effectively in the real world, agents should be able to act from high-dimensional raw sensory input such as images and achieve diverse goals across long time-horizons.
A Map Equation with Metadata: Varying the Role of Attributes in Community Detection
In this work, we introduce a tuning parameter to the content map equation that allows users of the Infomap community detection algorithm to control the metadata's relative importance for identifying network structure.
Post-processing partitions to identify domains of modularity optimization
We introduce the Convex Hull of Admissible Modularity Partitions (CHAMP) algorithm to prune and prioritize different network community structures identified across multiple runs of possibly various computational heuristics.
Social and Information Networks Physics and Society
