Search Results for author:
Found 35 papers, 10 papers with code
Recommending to Strategic Users
We model this user behavior as a two-stage noisy signalling game between the recommendation system and users: the recommendation system initially commits to a recommendation policy, presents content to the users during a cold start phase which the users choose to strategically consume in order to affect the types of content they will be recommended in a recommendation phase.
Benchmarking Interpretability Tools for Deep Neural Networks
Interpreting deep neural networks is the topic of much current research in AI.
Diagnostics for Deep Neural Networks with Automated Copy/Paste Attacks
Third, we compare this approach with other interpretability tools by attempting to rediscover trojans.
White-Box Adversarial Policies in Deep Reinforcement Learning
Third, we show that training against white-box adversarial policies can be used to make learners in single-agent environments more robust to domain shifts.
Get It in Writing: Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL
It is an open problem in MARL to replicate such cooperative behaviors in selfish agents.
Towards Psychologically-Grounded Dynamic Preference Models
Second, we discuss implications of dynamic preference models for recommendation systems evaluation and design.
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
The last decade of machine learning has seen drastic increases in scale and capabilities.
Building Human Values into Recommender Systems: An Interdisciplinary Synthesis
We collect a set of values that seem most relevant to recommender systems operating across different domains, then examine them from the perspectives of current industry practice, measurement, product design, and policy approaches.
How to talk so AI will learn: Instructions, descriptions, and autonomy
We study two distinct types of language: $\textit{instructions}$, which provide information about the desired policy, and $\textit{descriptions}$, which provide information about the reward function.
Estimating and Penalizing Induced Preference Shifts in Recommender Systems
These steps involve two challenging ingredients: estimation requires anticipating how hypothetical algorithms would influence user preferences if deployed - we do this by using historical user interaction data to train a predictive user model which implicitly contains their preference dynamics; evaluation and optimization additionally require metrics to assess whether such influences are manipulative or otherwise unwanted - we use the notion of "safe shifts", that define a trust region within which behavior is safe: for instance, the natural way in which users would shift without interference from the system could be deemed "safe".
Linguistic communication as (inverse) reward design
We then define a pragmatic listener which performs inverse reward design by jointly inferring the speaker's latent horizon and rewards.
Guided Imitation of Task and Motion Planning
While modern policy optimization methods can do complex manipulation from sensory data, they struggle on problems with extended time horizons and multiple sub-goals.
Robust Feature-Level Adversaries are Interpretability Tools
We demonstrate that they can be used to produce targeted, universal, disguised, physically-realizable, and black-box attacks at the ImageNet scale.
What are you optimizing for? Aligning Recommender Systems with Human Values
We describe cases where real recommender systems were modified in the service of various human values such as diversity, fairness, well-being, time well spent, and factual accuracy.
Consequences of Misaligned AI
We consider the cost of this incompleteness by analyzing a model of a principal and an agent in a resource constrained world where the $L$ attributes of the state correspond to different sources of utility for the principal.
Multi-Principal Assistance Games: Definition and Collegial Mechanisms
We introduce the concept of a multi-principal assistance game (MPAG), and circumvent an obstacle in social choice theory, Gibbard's theorem, by using a sufficiently collegial preference inference mechanism.
Multi-Principal Assistance Games
Assistance games (also known as cooperative inverse reinforcement learning games) have been proposed as a model for beneficial AI, wherein a robotic agent must act on behalf of a human principal but is initially uncertain about the humans payoff function.
Silly rules improve the capacity of agents to learn stable enforcement and compliance behaviors
How can societies learn to enforce and comply with social norms?
An Extensible Interactive Interface for Agent Design
In deep reinforcement learning, for example, directly specifying a reward as a function of a high-dimensional observation is challenging.
Adversarial Training with Voronoi Constraints
We show that adversarial training with Voronoi constraints produces robust models which significantly improve over the state-of-the-art on MNIST and are competitive on CIFAR-10.
Conservative Agency via Attainable Utility Preservation
Reward functions are easy to misspecify; although designers can make corrections after observing mistakes, an agent pursuing a misspecified reward function can irreversibly change the state of its environment.
The Assistive Multi-Armed Bandit
Learning preferences implicit in the choices humans make is a well studied problem in both economics and computer science.
On the Utility of Model Learning in HRI
Fundamental to robotics is the debate between model-based and model-free learning: should the robot build an explicit model of the world, or learn a policy directly?
Human-AI Learning Performance in Multi-Armed Bandits
People frequently face challenging decision-making problems in which outcomes are uncertain or unknown.
Legible Normativity for AI Alignment: The Value of Silly Rules
It has become commonplace to assert that autonomous agents will have to be built to follow human rules of behavior--social norms and laws.
On the Geometry of Adversarial Examples
Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models.
Active Inverse Reward Design
We propose structuring this process as a series of queries asking the user to compare between different reward functions.
An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning
We apply this update to a variety of POMDP solvers and find that it enables us to scale CIRL to non-trivial problems, with larger reward parameter spaces, and larger action spaces for both robot and human.
Simplifying Reward Design through Divide-and-Conquer
Designing a good reward function is essential to robot planning and reinforcement learning, but it can also be challenging and frustrating.
Incomplete Contracting and AI Alignment
We suggest that the analysis of incomplete contracting developed by law and economics researchers can provide a useful framework for understanding the AI alignment problem and help to generate a systematic approach to finding solutions.
Inverse Reward Design
When designing the reward, we might think of some specific training scenarios, and make sure that the reward will lead to the right behavior in those scenarios.
Pragmatic-Pedagogic Value Alignment
In robotics, value alignment is key to the design of collaborative robots that can integrate into human workflows, successfully inferring and adapting to their users' objectives as they go.
Should Robots be Obedient?
We show that when a human is not perfectly rational then a robot that tries to infer and act according to the human's underlying preferences can always perform better than a robot that simply follows the human's literal order.
The Off-Switch Game
We analyze a simple game between a human H and a robot R, where H can press R's off switch but R can disable the off switch.
Cooperative Inverse Reinforcement Learning
For an autonomous system to be helpful to humans and to pose no unwarranted risks, it needs to align its values with those of the humans in its environment in such a way that its actions contribute to the maximization of value for the humans.
