no code implementations • 6 Nov 2024 • Jianyi Yang, Pengfei Li, Adam Wierman, Shaolei Ren
In this paper, we remove the FLM assumption and tackle the open problem of OBM with general bids.
1 code implementation • 6 Nov 2024 • Chengrui Qu, Laixi Shi, Kishan Panaganti, Pengcheng You, Adam Wierman
However, with prior information on the degree of the dynamics shift, we design HySRL, a transfer algorithm that achieves problem-dependent sample complexity and outperforms pure online RL.
no code implementations • 30 Sep 2024 • Laixi Shi, Jingchu Gai, Eric Mazumdar, Yuejie Chi, Adam Wierman
A notorious yet open challenge is if RMGs can escape the curse of multiagency, where the sample complexity scales exponentially with the number of agents.
1 code implementation • 30 Sep 2024 • Christopher Yeh, Nicolas Christianson, Alan Wu, Adam Wierman, Yisong Yue
However, ensuring robustness guarantees requires well-calibrated uncertainty estimates, which can be difficult to achieve in high-capacity prediction models such as deep neural networks.
no code implementations • 2 Sep 2024 • Zaiwei Chen, Kaiqing Zhang, Eric Mazumdar, Asuman Ozdaglar, Adam Wierman
In this paper, we consider two-player zero-sum matrix and stochastic games and develop learning dynamics that are payoff-based, convergent, rational, and symmetric between the two players.
no code implementations • 14 Aug 2024 • Adam Lechowicz, Nicolas Christianson, Bo Sun, Noman Bashir, Mohammad Hajiesmaili, Adam Wierman, Prashant Shenoy
We formalize this as an online problem called spatiotemporal online allocation with deadline constraints ($\mathsf{SOAD}$), in which an online player completes a workload (e. g., a batch compute job) by moving and scheduling the workload across a network subject to a deadline $T$.
no code implementations • 22 Jun 2024 • Zhengfei Zhang, Kishan Panaganti, Laixi Shi, Yanan Sui, Adam Wierman, Yisong Yue
We study the problem of Distributionally Robust Constrained RL (DRC-RL), where the goal is to maximize the expected reward subject to environmental distribution shifts and constraints.
no code implementations • 31 May 2024 • Shangding Gu, Laixi Shi, Yuhao Ding, Alois Knoll, Costas Spanos, Adam Wierman, Ming Jin
Safe reinforcement learning (RL) is crucial for deploying RL agents in real-world applications, as it aims to maximize long-term rewards while satisfying safety constraints.
no code implementations • 30 May 2024 • Ruiyang Jin, Zaiwei Chen, Yiheng Lin, Jie Song, Adam Wierman
Independent learning (IL), despite being a popular approach in practice to achieve scalability in large-scale multi-agent systems, usually lacks global convergence guarantees.
no code implementations • 22 May 2024 • Benjamin C. Lee, David Brooks, Arthur van Benthem, Udit Gupta, Gage Hills, Vincent Liu, Benjamin Pierce, Christopher Stewart, Emma Strubell, Gu-Yeon Wei, Adam Wierman, Yuan YAO, Minlan Yu
For embodied carbon, we must re-think conventional design strategies -- over-provisioned monolithic servers, frequent hardware refresh cycles, custom silicon -- and adopt life-cycle design strategies that more effectively reduce, reuse and recycle hardware at scale.
no code implementations • 8 May 2024 • Kishan Panaganti, Adam Wierman, Eric Mazumdar
To the best of our knowledge, we provide the first improved out-of-data-distribution assumption in large-scale problems with general function approximation under the hybrid robust $\phi$-regularized reinforcement learning framework.
no code implementations • 29 Apr 2024 • Laixi Shi, Eric Mazumdar, Yuejie Chi, Adam Wierman
To overcome the sim-to-real gap in reinforcement learning (RL), learned policies must maintain robustness against environmental uncertainties.
Multi-agent Reinforcement Learning Reinforcement Learning (RL)
no code implementations • 21 Feb 2024 • Adam Lechowicz, Nicolas Christianson, Bo Sun, Noman Bashir, Mohammad Hajiesmaili, Adam Wierman, Prashant Shenoy
We introduce and study a family of online metric problems with long-term constraints.
no code implementations • 8 Dec 2023 • Zaiwei Chen, Kaiqing Zhang, Eric Mazumdar, Asuman Ozdaglar, Adam Wierman
Specifically, through a change of variable, we show that the update equation of the slow-timescale iterates resembles the classical smoothed best-response dynamics, where the regularized Nash gap serves as a valid Lyapunov function.
no code implementations • 3 Nov 2023 • Jinhang Zuo, Zhiyao Zhang, Xuchuang Wang, Cheng Chen, Shuai Li, John C. S. Lui, Mohammad Hajiesmaili, Adam Wierman
Cooperative multi-agent multi-armed bandits (CMA2B) consider the collaborative efforts of multiple agents in a shared multi-armed bandit game.
no code implementations • 31 Oct 2023 • Neelkamal Bhuyan, Debankur Mukherjee, Adam Wierman
We provide the online optimal algorithm when the minimizers of the hitting cost function evolve as a general stochastic process, which, for the case of martingale process, takes the form of a distribution-agnostic dynamic interpolation algorithm (LAI).
1 code implementation • 31 Oct 2023 • Adam Lechowicz, Nicolas Christianson, Bo Sun, Noman Bashir, Mohammad Hajiesmaili, Adam Wierman, Prashant Shenoy
We introduce competitive (robust) threshold-based algorithms for both the minimization and maximization variants of this problem, and show they are optimal among deterministic online algorithms.
no code implementations • 17 Oct 2023 • Bo Sun, Jerry Huang, Nicolas Christianson, Mohammad Hajiesmaili, Adam Wierman, Raouf Boutaba
The burgeoning field of algorithms with predictions studies the problem of using possibly imperfect machine learning predictions to improve online algorithm performance.
1 code implementation • 26 Sep 2023 • YingYing Li, Jing Yu, Lauren Conger, Taylan Kargin, Adam Wierman
This paper studies uncertainty set estimation for unknown linear systems.
1 code implementation • 29 Jun 2023 • Christopher Yeh, Jing Yu, Yuanyuan Shi, Adam Wierman
In this work, we combine a nested convex body chasing algorithm with a robust predictive controller to achieve provably finite-time convergence to safe voltage limits in the online setting where there is uncertainty in both the network topology as well as load and generation variations.
1 code implementation • 20 Jun 2023 • Pengfei Li, Jianyi Yang, Adam Wierman, Shaolei Ren
The results demonstrate that existing GLB approaches may amplify environmental inequity while our proposed equity-aware GLB can significantly reduce the regional disparity in terms of carbon and water footprints.
no code implementations • 16 Jun 2023 • Pengfei Li, Jianyi Yang, Adam Wierman, Shaolei Ren
This paper studies decentralized online convex optimization in a networked multi-agent system and proposes a novel algorithm, Learning-Augmented Decentralized Online optimization (LADO), for individual agents to select actions only based on local online information.
no code implementations • 30 Mar 2023 • Xutong Liu, Jinhang Zuo, Siwei Wang, John C. S. Lui, Mohammad Hajiesmaili, Adam Wierman, Wei Chen
We study contextual combinatorial bandits with probabilistically triggered arms (C$^2$MAB-T) under a variety of smoothness conditions that capture a wide range of applications, such as contextual cascading bandits and contextual influence maximization bandits.
1 code implementation • 8 Mar 2023 • Zhaoyi Zhou, Zaiwei Chen, Yiheng Lin, Adam Wierman
The algorithm is scalable since each agent uses only local information and does not need access to the global state.
no code implementations • 30 Nov 2022 • Yizhou Zhang, Guannan Qu, Pan Xu, Yiheng Lin, Zaiwei Chen, Adam Wierman
In particular, we show that, despite restricting each agent's attention to only its $\kappa$-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in $\kappa$.
Multi-agent Reinforcement Learning reinforcement-learning +1
1 code implementation • 16 Sep 2022 • Jie Feng, Yuanyuan Shi, Guannan Qu, Steven H. Low, Anima Anandkumar, Adam Wierman
In this paper, we propose a stability-constrained reinforcement learning (RL) method for real-time voltage control, that guarantees system stability both during policy learning and deployment of the learned policy.
1 code implementation • 29 Jun 2022 • Christopher Yeh, Jing Yu, Yuanyuan Shi, Adam Wierman
Voltage control generally requires accurate information about the grid's topology in order to guarantee network stability.
no code implementations • 23 Jun 2022 • Nicolas Christianson, Tinashe Handina, Adam Wierman
We consider the problem of convex function chasing with black-box advice, where an online decision-maker aims to minimize the total cost of making and switching between decisions in a normed vector space, aided by black-box advice such as the decisions of a machine-learned algorithm.
no code implementations • 3 Jun 2022 • Sahin Lale, Yuanyuan Shi, Guannan Qu, Kamyar Azizzadenesheli, Adam Wierman, Anima Anandkumar
However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems.
no code implementations • 2 Jun 2022 • Tongxin Li, Ruixiao Yang, Guannan Qu, Yiheng Lin, Steven Low, Adam Wierman
Machine-learned black-box policies are ubiquitous for nonlinear control problems.
no code implementations • 12 May 2022 • Chen Liang, Alessandro Zocca, Steven H. Low, Adam Wierman
Transmission power systems usually consist of interconnected sub-grids that are operated relatively independently.
1 code implementation • 12 Apr 2022 • Sungho Shin, Yiheng Lin, Guannan Qu, Adam Wierman, Mihai Anitescu
This paper studies the trade-off between the degree of decentralization and the performance of a distributed controller in a linear-quadratic control setting.
no code implementations • 5 Mar 2022 • Jing Yu, Dimitar Ho, Adam Wierman
We investigate the problem of stabilizing an unknown networked linear system under communication constraints and adversarial disturbances.
no code implementations • 7 Feb 2022 • Daan Rutten, Nico Christianson, Debankur Mukherjee, Adam Wierman
The goal of the decision maker is to exploit the predictions if they are accurate, while guaranteeing performance that is not much worse than the hindsight optimal sequence of decisions, even when predictions are inaccurate.
no code implementations • 29 Oct 2021 • Weici Pan, Guanya Shi, Yiheng Lin, Adam Wierman
We study a variant of online optimization in which the learner receives $k$-round $\textit{delayed feedback}$ about hitting cost and there is a multi-step nonlinear switching cost, i. e., costs depend on multiple previous actions in a nonlinear manner.
no code implementations • 30 Sep 2021 • Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, Adam Wierman
Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems.
no code implementations • NeurIPS 2021 • Bo Sun, Russell Lee, Mohammad Hajiesmaili, Adam Wierman, Danny H. K. Tsang
This paper leverages machine-learned predictions to design competitive algorithms for online conversion problems with the goal of improving the competitive ratio when predictions are accurate (i. e., consistency), while also guaranteeing a worst-case competitive ratio regardless of the prediction quality (i. e., robustness).
no code implementations • NeurIPS 2021 • Tongxin Li, Ruixiao Yang, Guannan Qu, Guanya Shi, Chenkai Yu, Adam Wierman, Steven H. Low
Motivated by online learning methods, we design a self-tuning policy that adaptively learns the trust parameter $\lambda$ with a competitive ratio that depends on $\varepsilon$ and the variation of system perturbations and predictions.
no code implementations • 29 Apr 2021 • Guannan Qu, Yuanyuan Shi, Sahin Lale, Anima Anandkumar, Adam Wierman
In this work, we propose an efficient online control algorithm, COvariance Constrained Online Linear Quadratic (COCO-LQ) control, that guarantees input-to-state stability for a large class of LTV systems while also minimizing the control cost.
no code implementations • 21 Dec 2020 • Tongxin Li, Bo Sun, Yue Chen, Zixin Ye, Steven H. Low, Adam Wierman
To be used effectively, an aggregator must be able to communicate the available flexibility of the loads they control, as known as the aggregate flexibility to a system operator.
Optimization and Control Systems and Control Systems and Control
no code implementations • NeurIPS 2020 • Chenkai Yu, Guanya Shi, Soon-Jo Chung, Yisong Yue, Adam Wierman
We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics.
no code implementations • 12 Jun 2020 • Guannan Qu, Chenkai Yu, Steven Low, Adam Wierman
Model-free learning-based control methods have seen great success recently.
1 code implementation • NeurIPS 2021 • Yiheng Lin, Guannan Qu, Longbo Huang, Adam Wierman
We study multi-agent reinforcement learning (MARL) in a stochastic network of agents.
Multi-agent Reinforcement Learning reinforcement-learning +2
no code implementations • NeurIPS 2020 • Guannan Qu, Yiheng Lin, Adam Wierman, Na Li
It has long been recognized that multi-agent reinforcement learning (MARL) faces significant scalability issues due to the fact that the size of the state and action spaces are exponentially large in the number of agents.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • L4DC 2020 • Guannan Qu, Adam Wierman, Na Li
We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized.
no code implementations • 22 May 2020 • Linqi Guo, Chen Liang, Alessandro Zocca, Steven H. Low, Adam Wierman
Transmission line failure in power systems prop-agate non-locally, making the control of the resulting outages extremely difficult.
no code implementations • 22 May 2020 • Chen Liang, Linqi Guo, Alessandro Zocca, Steven H. Low, Adam Wierman
Transmission line failures in power systems propagate and cascade non-locally.
no code implementations • 20 May 2020 • Linqi Guo, Chen Liang, Alessandro Zocca, Steven H. Low, Adam Wierman
Transmission line failures in power systems propagate non-locally, making the control of the resulting outages extremely difficult.
1 code implementation • NeurIPS 2020 • Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, Adam Wierman
This paper presents competitive algorithms for a novel class of online optimization problems with memory.
no code implementations • 1 Feb 2020 • Guannan Qu, Adam Wierman
We consider a general asynchronous Stochastic Approximation (SA) scheme featuring a weighted infinity-norm contractive operator, and prove a bound on its finite-time convergence rate on a single trajectory.
no code implementations • 5 Dec 2019 • Guannan Qu, Adam Wierman, Na Li
We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized.
no code implementations • 10 Nov 2019 • Yiheng Lin, Gautam Goel, Adam Wierman
In this work, we give two general sufficient conditions that specify a relationship between the hitting and movement costs which guarantees that a new algorithm, Synchronized Fixed Horizon Control (SFHC), provides a $1+O(1/w)$ competitive ratio, where $w$ is the number of predictions available to the learner.
no code implementations • NeurIPS 2019 • Gautam Goel, Yiheng Lin, Haoyuan Sun, Adam Wierman
We prove a new lower bound on the competitive ratio of any online algorithm in the setting where the costs are $m$-strongly convex and the movement costs are the squared $\ell_2$ norm.
no code implementations • 11 Mar 2019 • John Pang, Weixuan Lin, Hu Fu, Jack Kleeman, Eilyan Bitar, Adam Wierman
In this paper, we analyze the worst case efficiency loss of online platform designs under a networked Cournot competition model.
Computer Science and Game Theory
no code implementations • 23 Oct 2018 • Gautam Goel, Adam Wierman
We consider Online Convex Optimization (OCO) in the setting where the costs are $m$-strongly convex and the online learner pays a switching cost for changing decisions between rounds.
no code implementations • 3 Oct 2018 • Riley Murray, Venkat Chandrasekaran, Adam Wierman
When specialized to the context of polynomials, we obtain analysis and computational tools that only depend on the particular monomials that constitute a sparse polynomial.
Optimization and Control
no code implementations • 28 Mar 2018 • Niangjun Chen, Gautam Goel, Adam Wierman
We demonstrate the generality of the OBD framework by showing how, with different choices of "balance," OBD can improve upon state-of-the-art performance guarantees for both competitive ratio and regret, in particular, OBD is the first algorithm to achieve a dimension-free competitive ratio, $3 + O(1/\alpha)$, for locally polyhedral costs, where $\alpha$ measures the "steepness" of the costs.
no code implementations • 17 Nov 2017 • Palma London, Shai Vardi, Adam Wierman, Hanling Yi
This paper presents an acceleration framework for packing linear programming problems where the amount of data available is limited, i. e., where the number of constraints m is small compared to the variable dimension n. The framework can be used as a black box to speed up linear programming solvers dramatically, by two orders of magnitude in our experiments.
no code implementations • 25 Apr 2015 • Niangjun Chen, Anish Agarwal, Adam Wierman, Siddharth Barman, Lachlan L. H. Andrew
Making use of predictions is a crucial, but under-explored, area of online algorithms.
no code implementations • 6 Jul 2013 • Wei Chen, Dayu Huang, Ankur A. Kulkarni, Jayakrishnan Unnikrishnan, Quanyan Zhu, Prashant Mehta, Sean Meyn, Adam Wierman
Neuro-dynamic programming is a class of powerful techniques for approximating the solution to dynamic programming equations.