Search Results for author: Lillian J. Ratliff

Found 19 papers, 4 papers with code

Multi-learner risk reduction under endogenous participation dynamics

no code implementations6 Jun 2022 Sarah Dean, Mihaela Curmei, Lillian J. Ratliff, Jamie Morgenstern, Maryam Fazel

Previous work on the single learner case shows that myopic risk minimization can result in high overall loss~\citep{perdomo2020performative, miller2021outside} and representation disparity~\citep{hashimoto2018fairness, zhang2019group}.

General sum stochastic games with networked information flows

no code implementations5 May 2022 Sarah H. Q. Li, Lillian J. Ratliff, Peeyush Kumar

Inspired by applications such as supply chain management, epidemics, and social networks, we formulate a stochastic game model that addresses three key features common across these domains: 1) network-structured player interactions, 2) pair-wise mixed cooperation and competition among players, and 3) limited global information toward individual decision-making.

Decision Making Management +2

Decision-Dependent Risk Minimization in Geometrically Decaying Dynamic Environments

no code implementations8 Apr 2022 Mitas Ray, Dmitriy Drusvyatskiy, Maryam Fazel, Lillian J. Ratliff

This paper studies the problem of expected loss minimization given a data distribution that is dependent on the decision-maker's action and evolves dynamically in time according to a geometric decay process.

Multiplayer Performative Prediction: Learning in Decision-Dependent Games

no code implementations10 Jan 2022 Adhyyan Narang, Evan Faulkner, Dmitriy Drusvyatskiy, Maryam Fazel, Lillian J. Ratliff

We show that under mild assumptions, the performatively stable equilibria can be found efficiently by a variety of algorithms, including repeated retraining and the repeated (stochastic) gradient method.

Stackelberg Actor-Critic: Game-Theoretic Reinforcement Learning Algorithms

1 code implementation25 Sep 2021 Liyuan Zheng, Tanner Fiez, Zane Alumbaugh, Benjamin Chasnov, Lillian J. Ratliff

The hierarchical interaction between the actor and critic in actor-critic based reinforcement learning algorithms naturally lends itself to a game-theoretic interpretation.

OpenAI Gym

Approximate Regions of Attraction in Learning with Decision-Dependent Distributions

no code implementations30 Jun 2021 Roy Dong, Lillian J. Ratliff

As data-driven methods are deployed in real-world settings, the processes that generate the observed data will often react to the decisions of the learner.

Zeroth-Order Methods for Convex-Concave Minmax Problems: Applications to Decision-Dependent Risk Minimization

no code implementations16 Jun 2021 Chinmay Maheshwari, Chih-Yuan Chiu, Eric Mazumdar, S. Shankar Sastry, Lillian J. Ratliff

Min-max optimization is emerging as a key framework for analyzing problems of robustness to strategically and adversarially generated data.

Minimax Optimization with Smooth Algorithmic Adversaries

1 code implementation ICLR 2022 Tanner Fiez, Chi Jin, Praneeth Netrapalli, Lillian J. Ratliff

This paper considers minimax optimization $\min_x \max_y f(x, y)$ in the challenging setting where $f$ can be both nonconvex in $x$ and nonconcave in $y$.

Function Design for Improved Competitive Ratio in Online Resource Allocation with Procurement Costs

no code implementations23 Dec 2020 Mitas Ray, Omid Sadeghi, Lillian J. Ratliff, Maryam Fazel

We study the problem of online resource allocation, where multiple customers arrive sequentially and the seller must irrevocably allocate resources to each incoming customer while also facing a procurement cost for the total allocation.

Safe Reinforcement Learning of Control-Affine Systems with Vertex Networks

1 code implementation20 Mar 2020 Liyuan Zheng, Yuanyuan Shi, Lillian J. Ratliff, Baosen Zhang

This paper focuses on finding reinforcement learning policies for control systems with hard state and action constraints.

reinforcement-learning Safe Reinforcement Learning

Constrained Upper Confidence Reinforcement Learning

no code implementations26 Jan 2020 Liyuan Zheng, Lillian J. Ratliff

Constrained Markov Decision Processes are a class of stochastic decision problems in which the decision maker must select a policy that satisfies auxiliary cost constraints.

reinforcement-learning

Policy-Gradient Algorithms Have No Guarantees of Convergence in Linear Quadratic Games

no code implementations8 Jul 2019 Eric Mazumdar, Lillian J. Ratliff, Michael. I. Jordan, S. Shankar Sastry

In such games the state and action spaces are continuous and global Nash equilibria can be found be solving coupled Ricatti equations.

reinforcement-learning

Convergence of Learning Dynamics in Stackelberg Games

1 code implementation4 Jun 2019 Tanner Fiez, Benjamin Chasnov, Lillian J. Ratliff

Using this insight, we develop a gradient-based update for the leader while the follower employs a best response strategy for which each stable critical point is guaranteed to be a Stackelberg equilibrium in zero-sum games.

Convergence Analysis of Gradient-Based Learning with Non-Uniform Learning Rates in Non-Cooperative Multi-Agent Settings

no code implementations30 May 2019 Benjamin Chasnov, Lillian J. Ratliff, Eric Mazumdar, Samuel A. Burden

Considering a class of gradient-based multi-agent learning algorithms in non-cooperative settings, we provide local convergence guarantees to a neighborhood of a stable local Nash equilibrium.

Competitive Statistical Estimation with Strategic Data Sources

no code implementations29 Apr 2019 Tyler Westenbroek, Roy Dong, Lillian J. Ratliff, S. Shankar Sastry

Recent work has explored mechanisms to ensure that the data sources share high quality data with a single data aggregator, addressing the issue of moral hazard.

Combinatorial Bandits for Incentivizing Agents with Dynamic Preferences

no code implementations6 Jul 2018 Tanner Fiez, Shreyas Sekar, Liyuan Zheng, Lillian J. Ratliff

The design of personalized incentives or recommendations to improve user engagement is gaining prominence as digital platform providers continually emerge.

On Gradient-Based Learning in Continuous Games

no code implementations16 Apr 2018 Eric Mazumdar, Lillian J. Ratliff, S. Shankar Sastry

We formulate a general framework for competitive gradient-based learning that encompasses a wide breadth of multi-agent learning algorithms, and analyze the limiting behavior of competitive gradient-based learning algorithms using dynamical systems theory.

Multi-agent Reinforcement Learning

Multi-Armed Bandits for Correlated Markovian Environments with Smoothed Reward Feedback

no code implementations11 Mar 2018 Tanner Fiez, Shreyas Sekar, Lillian J. Ratliff

We analyze these algorithms under two types of smoothed reward feedback at the end of each epoch: a reward that is the discount-average of the discounted rewards within an epoch, and a reward that is the time-average of the rewards within an epoch.

Multi-Armed Bandits Q-Learning

Inverse Risk-Sensitive Reinforcement Learning

no code implementations29 Mar 2017 Lillian J. Ratliff, Eric Mazumdar

We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive.

Decision Making reinforcement-learning

Cannot find the paper you are looking for? You can Submit a new open access paper.