Search Results for author:
Found 59 papers, 25 papers with code
Image Hijacks: Adversarial Images can Control Generative Models at Runtime
In this work, we focus on the image input to a vision-language model (VLM).
Who Needs to Know? Minimal Knowledge for Optimal Coordination
We apply this algorithm to analyze the strategically relevant information for tasks in both a standard and a partially observable version of the Overcooked environment.
TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI
While several recent works have identified societal-scale and extinction-level risks to humanity arising from artificial intelligence, few have attempted an {\em exhaustive taxonomy} of such risks.
Bridging RL Theory and Practice with the Effective Horizon
We find that prior bounds do not correlate well with when deep RL succeeds vs. fails, but discover a surprising property that does.
Active Reward Learning from Multiple Teachers
Reward learning algorithms utilize human feedback to infer a reward function, which is then used to train an AI system.
imitation: Clean Imitation Learning Implementations
imitation provides open-source implementations of imitation and reward learning algorithms in PyTorch.
Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
Offline reinforcement learning (RL), which refers to decision-making from a previously-collected dataset of interactions, has received significant attention over the past years.
Adversarial Policies Beat Superhuman Go AIs
The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack.
For Learning in Symmetric Teams, Local Optima are Global Nash Equilibria
In this work, we show that any locally optimal symmetric strategy profile is also a (global) Nash equilibrium.
An Empirical Investigation of Representation Learning for Imitation
We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation across several environment suites.
Estimating and Penalizing Induced Preference Shifts in Recommender Systems
These steps involve two challenging ingredients: estimation requires anticipating how hypothetical algorithms would influence user preferences if deployed - we do this by using historical user interaction data to train a predictive user model which implicitly contains their preference dynamics; evaluation and optimization additionally require metrics to assess whether such influences are manipulative or otherwise unwanted - we use the notion of "safe shifts", that define a trust region within which behavior is safe: for instance, the natural way in which users would shift without interference from the system could be deemed "safe".
Invariance in Policy Optimisation and Partial Identifiability in Reward Learning
It is often very challenging to manually design reward functions for complex, real-world tasks.
Quantifying Local Specialization in Deep Neural Networks
These results suggest that graph-based partitioning can reveal local specialization and that statistical methods can be used to automatedly screen for sets of neurons that can be understood abstractly.
Cross-Domain Imitation Learning via Optimal Transport
Cross-domain imitation learning studies how to leverage expert demonstrations of one agent to train an imitation agent with a different embodiment or morphology.
Scalable Online Planning via Reinforcement Learning Fine-Tuning
Lookahead search has been a critical component of recent AI successes, such as in the games of chess, go, and poker.
Detecting Modularity in Deep Neural Networks
These results suggest that graph-based partitioning can reveal modularity and help us understand how deep neural networks function.
Explore and Control with Adversarial Surprise
Unsupervised reinforcement learning (RL) studies how to leverage environment statistics to learn useful behaviors without the cost of reward engineering.
The MineRL BASALT Competition on Learning from Human Feedback
Rather than training AI systems using a predefined reward function or using a labeled dataset with a predefined set of categories, we instead train the AI system using a learning signal derived from some form of human feedback, which can evolve over time as the understanding of the task changes, or as the capabilities of the AI system improve.
Uncertain Decisions Facilitate Better Preference Learning
We give the first statistical analysis of IDT, providing conditions necessary to identify these preferences and characterizing the sample complexity -- the number of decisions that must be observed to learn the tradeoff the human is making to a desired precision.
MADE: Exploration via Maximizing Deviation from Explored Regions
As a proof of concept, we evaluate the new intrinsic reward on tabular examples across a variety of model-based and model-free algorithms, showing improvements over count-only exploration strategies.
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism
Based on the composition of the offline dataset, two main categories of methods are used: imitation learning which is suitable for expert datasets and vanilla offline RL which often requires uniform coverage datasets.
Clusterability in Neural Networks
We also exhibit novel methods to promote clusterability in neural network training, and find that in multi-layer perceptrons they lead to more clusterable networks with little reduction in accuracy.
Accumulating Risk Capital Through Investing in Cooperation
Recent work on promoting cooperation in multi-agent learning has resulted in many methods which successfully promote cooperation at the cost of becoming more vulnerable to exploitation by malicious actors.
Discrete Predictive Representation for Long-horizon Planning
Discrete representations have been key in enabling robots to plan at more abstract levels and solve temporally-extended tasks more efficiently for decades.
Benefits of Assistance over Reward Learning
By merging reward learning and control, assistive agents can reason about the impact of control actions on reward learning, leading to several advantages over agents based on reward learning.
Importance and Coherence: Methods for Evaluating Modularity in Neural Networks
We apply these methods on partitionings generated by a spectral clustering algorithm which uses a graph representation of the network's neurons and weights.
Multi-Principal Assistance Games: Definition and Collegial Mechanisms
We introduce the concept of a multi-principal assistance game (MPAG), and circumvent an obstacle in social choice theory, Gibbard's theorem, by using a sufficiently collegial preference inference mechanism.
Understanding Learned Reward Functions
However, current techniques for reward learning may fail to produce reward functions which accurately reflect user preferences.
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
We call our technique Protagonist Antagonist Induced Regret Environment Design (PAIRED).
DERAIL: Diagnostic Environments for Reward And Imitation Learning
We evaluate a range of common reward and imitation learning algorithms on our tasks.
Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Open-universe probabilistic models enable Bayesian inference about how many objects underlie data, and how they are related.
The MAGICAL Benchmark for Robust Imitation
This rewards precise reproduction of demonstrations in one particular environment, but provides little information about how robustly an algorithm can generalise the demonstrator's intent to substantially different deployment settings.
SLIP: Learning to Predict in Unknown Dynamical Systems with Long-Term Memory
Our theoretical and experimental results shed light on the conditions required for efficient probably approximately correct (PAC) learning of the Kalman filter from partially observed data.
Multi-Principal Assistance Games
Assistance games (also known as cooperative inverse reinforcement learning games) have been proposed as a model for beneficial AI, wherein a robotic agent must act on behalf of a human principal but is initially uncertain about the humans payoff function.
Quantifying Differences in Reward Functions
However, this method cannot distinguish between the learned reward function failing to reflect user preferences and the policy optimization process failing to optimize the learned reward.
Pruned Neural Networks are Surprisingly Modular
To discern structure in these weights, we introduce a measurable notion of modularity for multi-layer perceptrons (MLPs), and investigate the modular structure of MLPs trained on datasets of small images.
Bayesian Relational Memory for Semantic Visual Navigation
We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards.
Adversarial Policies: Attacking Deep Reinforcement Learning
Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers.
Inverse reinforcement learning for video games
Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function.
Learning and Planning with a Semantic Model
Building deep reinforcement learning agents that can generalize and adapt to unseen environments remains a fundamental challenge for AI.
Learning Plannable Representations with Causal InfoGAN
Finally, to generate a visual plan, we project the current and goal observations onto their respective states in the planning model, plan a trajectory, and then use the generative model to transform the trajectory to a sequence of observations.
An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning
We apply this update to a variety of POMDP solvers and find that it enables us to scale CIRL to non-trivial problems, with larger reward parameter spaces, and larger action spaces for both robot and human.
Discrete-Continuous Mixtures in Probabilistic Programming: Generalized Semantics and Inference Algorithms
Despite the recent successes of probabilistic programming languages (PPLs) in AI applications, PPLs offer only limited support for random variables whose distributions combine discrete and continuous elements.
Inverse Reward Design
When designing the reward, we might think of some specific training scenarios, and make sure that the reward will lead to the right behavior in those scenarios.
Servant of Many Masters: Shifting priorities in Pareto-optimal sequential decision-making
It is often argued that an agent making decisions on behalf of two or more principals who have different utility functions should adopt a {\em Pareto-optimal} policy, i. e., a policy that cannot be improved upon for one agent without making sacrifices for another.
Adversarial Training for Relation Extraction
Adversarial training is a mean of regularizing classification algorithms by generating adversarial noise to the training data.
Should Robots be Obedient?
We show that when a human is not perfectly rational then a robot that tries to infer and act according to the human's underlying preferences can always perform better than a robot that simply follows the human's literal order.
The Off-Switch Game
We analyze a simple game between a human H and a robot R, where H can press R's off switch but R can disable the off switch.
Swift: Compiled Inference for Probabilistic Programming Languages
A probabilistic program defines a probability measure over its semantic structures.
Cooperative Inverse Reinforcement Learning
For an autonomous system to be helpful to humans and to pose no unwarranted risks, it needs to align its values with those of the humans in its environment in such a way that its actions contribute to the maximization of value for the humans.
Towards Practical Bayesian Parameter and State Estimation
Joint state and parameter estimation is a core problem for dynamic Bayesian networks.
Research Priorities for Robust and Beneficial Artificial Intelligence
Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to investigate how to maximize these benefits while avoiding potential pitfalls.
Probabilistic Model-Based Approach for Heart Beat Detection
One of the biggest flaws in the medical system is perhaps an unexpected one: the patient alarm system.
Selecting Computations: Theory and Applications
Sequential decision problems are often approximately solvable by simulating possible future action sequences.
Fine-Grained Decision-Theoretic Search Control
This formula is used to estimate the value of expanding further successors, using a general formula for the value of a computation in game-playing developed in earlier work.
Automated Construction of Sparse Bayesian Networks from Unstructured Probabilistic Models and Domain Information
An algorithm for automated construction of a sparse Bayesian network given an unstructured probabilistic model and causal domain information from an expert has been developed and implemented.
Variational MCMC
One of these algorithms is a mixture of two MCMC kernels: a random walk Metropolis kernel and a blockMetropolis-Hastings (MH) kernel with a variational approximation as proposaldistribution.
Markov Chain Monte Carlo Data Association for Multiple-Target Tracking
This paper presents Markov chain Monte Carlo data association (MCMCDA) for solving data association problems arising in multiple-target tracking in a cluttered environment.
