Search Results for author:
Found 48 papers, 8 papers with code
Chasing Convex Functions with Long-term Constraints
We introduce and study a family of online metric problems with long-term constraints.
Two-Timescale Q-Learning with Function Approximation in Zero-Sum Stochastic Games
Specifically, through a change of variable, we show that the update equation of the slow-timescale iterates resembles the classical smoothed best-response dynamics, where the regularized Nash gap serves as a valid Lyapunov function.
Adversarial Attacks on Cooperative Multi-agent Bandits
Cooperative multi-agent multi-armed bandits (CMA2B) consider the collaborative efforts of multiple agents in a shared multi-armed bandit game.
Best of Both Worlds Guarantees for Smoothed Online Quadratic Optimization
We provide the online optimal algorithm when the minimizers of the hitting cost function evolve as a general stochastic process, which, for the case of martingale process, takes the form of a distribution-agnostic dynamic interpolation algorithm (LAI).
Online Conversion with Switching Costs: Robust and Learning-Augmented Algorithms
We introduce competitive (robust) threshold-based algorithms for both the minimization and maximization variants of this problem, and show they are optimal among deterministic online algorithms.
Online Algorithms with Uncertainty-Quantified Predictions
In particular, we consider predictions augmented with uncertainty quantification describing the likelihood of the ground truth falling in a certain range, designing online algorithms with these probabilistic predictions for two classic online problems: ski rental and online search.
Learning the Uncertainty Sets for Control Dynamics via Set Membership: A Non-Asymptotic Analysis
Further, this result is applied to robust adaptive model predictive control with uncertainty sets updated by set membership.
Online learning for robust voltage control under uncertain grid topology
In this work, we combine a nested convex body chasing algorithm with a robust predictive controller to achieve provably finite-time convergence to safe voltage limits in the online setting where there is uncertainty in both the network topology as well as load and generation variations.
Towards Environmentally Equitable AI via Geographical Load Balancing
The results demonstrate that existing GLB approaches may amplify environmental inequity while our proposed equity-aware GLB can significantly reduce the regional disparity in terms of carbon and water footprints.
Learning-Augmented Decentralized Online Convex Optimization in Networks
This paper studies decentralized online convex optimization in a networked multi-agent system and proposes a novel algorithm, Learning-Augmented Decentralized Online optimization (LADO), for individual agents to select actions only based on local online information.
Contextual Combinatorial Bandits with Probabilistically Triggered Arms
We study contextual combinatorial bandits with probabilistically triggered arms (C$^2$MAB-T) under a variety of smoothness conditions that capture a wide range of applications, such as contextual cascading bandits and contextual influence maximization bandits.
Convergence Rates for Localized Actor-Critic in Networked Markov Potential Games
The algorithm is scalable since each agent uses only local information and does not need access to the global state.
Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learning
In particular, we show that, despite restricting each agent's attention to only its $\kappa$-hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in $\kappa$.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control
In this paper, we propose a stability-constrained reinforcement learning (RL) method for real-time voltage control, that guarantees system stability both during policy learning and deployment of the learned policy.
Robust Online Voltage Control with an Unknown Grid Topology
Voltage control generally requires accurate information about the grid's topology in order to guarantee network stability.
Chasing Convex Bodies and Functions with Black-Box Advice
We consider the problem of convex function chasing with black-box advice, where an online decision-maker aims to minimize the total cost of making and switching between decisions in a normed vector space, aided by black-box advice such as the decisions of a machine-learned algorithm.
KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems
However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems.
Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Control
Machine-learned black-box policies are ubiquitous for nonlinear control problems.
Interface Networks for Failure Localization in Power Systems
Transmission power systems usually consist of interconnected sub-grids that are operated relatively independently.
Near-Optimal Distributed Linear-Quadratic Regulator for Networked Systems
This paper studies the trade-off between the degree of decentralization and the performance of a distributed controller in a linear-quadratic control setting.
Online Adversarial Stabilization of Unknown Networked Systems
We investigate the problem of stabilizing an unknown networked linear system under communication constraints and adversarial disturbances.
Smoothed Online Optimization with Unreliable Predictions
The goal of the decision maker is to exploit the predictions if they are accurate, while guaranteeing performance that is not much worse than the hindsight optimal sequence of decisions, even when predictions are inaccurate.
Online Optimization with Feedback Delay and Nonlinear Switching Cost
We study a variant of online optimization in which the learner receives $k$-round $\textit{delayed feedback}$ about hitting cost and there is a multi-step nonlinear switching cost, i. e., costs depend on multiple previous actions in a nonlinear manner.
Stability Constrained Reinforcement Learning for Real-Time Voltage Control
Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems.
Pareto-Optimal Learning-Augmented Algorithms for Online Conversion Problems
This paper leverages machine-learned predictions to design competitive algorithms for online conversion problems with the goal of improving the competitive ratio when predictions are accurate (i. e., consistency), while also guaranteeing a worst-case competitive ratio regardless of the prediction quality (i. e., robustness).
Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions
Motivated by online learning methods, we design a self-tuning policy that adaptively learns the trust parameter $\lambda$ with a competitive ratio that depends on $\varepsilon$ and the variation of system perturbations and predictions.
Stable Online Control of Linear Time-Varying Systems
In this work, we propose an efficient online control algorithm, COvariance Constrained Online Linear Quadratic (COCO-LQ) control, that guarantees input-to-state stability for a large class of LTV systems while also minimizing the control cost.
Learning-Based Predictive Control via Real-Time Aggregate Flexibility
To be used effectively, an aggregator must be able to communicate the available flexibility of the loads they control, as known as the aggregate flexibility to a system operator.
Optimization and Control Systems and Control Systems and Control
The Power of Predictions in Online Control
We study the impact of predictions in online Linear Quadratic Regulator control with both stochastic and adversarial disturbances in the dynamics.
Combining Model-Based and Model-Free Methods for Nonlinear Control: A Provably Convergent Policy Gradient Approach
Model-free learning-based control methods have seen great success recently.
Multi-Agent Reinforcement Learning in Stochastic Networked Systems
We study multi-agent reinforcement learning (MARL) in a stochastic network of agents.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward
It has long been recognized that multi-agent reinforcement learning (MARL) faces significant scalability issues due to the fact that the size of the state and action spaces are exponentially large in the number of agents.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Scalable Reinforcement Learning of Localized Policies for Multi-Agent Networked Systems
We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized.
Adaptive Network Response to Line Failures in Power Systems
Transmission line failures in power systems propagate and cascade non-locally.
Line Failure Localization of Power Networks Part II: Cut Set Outages
Transmission line failure in power systems prop-agate non-locally, making the control of the resulting outages extremely difficult.
Line Failure Localization of Power Networks Part I: Non-cut Outages
Transmission line failures in power systems propagate non-locally, making the control of the resulting outages extremely difficult.
Online Optimization with Memory and Competitive Control
This paper presents competitive algorithms for a novel class of online optimization problems with memory.
Finite-Time Analysis of Asynchronous Stochastic Approximation and $Q$-Learning
We consider a general asynchronous Stochastic Approximation (SA) scheme featuring a weighted infinity-norm contractive operator, and prove a bound on its finite-time convergence rate on a single trajectory.
Scalable Reinforcement Learning for Multi-Agent Networked Systems
We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized.
Online Optimization with Predictions and Non-convex Losses
In this work, we give two general sufficient conditions that specify a relationship between the hitting and movement costs which guarantees that a new algorithm, Synchronized Fixed Horizon Control (SFHC), provides a $1+O(1/w)$ competitive ratio, where $w$ is the number of predictions available to the learner.
Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization
We prove a new lower bound on the competitive ratio of any online algorithm in the setting where the costs are $m$-strongly convex and the movement costs are the squared $\ell_2$ norm.
Transparency and Control in Platforms for Networked Markets
In this paper, we analyze the worst case efficiency loss of online platform designs under a networked Cournot competition model.
Computer Science and Game Theory
Smoothed Online Optimization for Regression and Control
We consider Online Convex Optimization (OCO) in the setting where the costs are $m$-strongly convex and the online learner pays a switching cost for changing decisions between rounds.
Newton Polytopes and Relative Entropy Optimization
When specialized to the context of polynomials, we obtain analysis and computational tools that only depend on the particular monomials that constitute a sparse polynomial.
Optimization and Control
Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent
We demonstrate the generality of the OBD framework by showing how, with different choices of "balance," OBD can improve upon state-of-the-art performance guarantees for both competitive ratio and regret, in particular, OBD is the first algorithm to achieve a dimension-free competitive ratio, $3 + O(1/\alpha)$, for locally polyhedral costs, where $\alpha$ measures the "steepness" of the costs.
A Parallelizable Acceleration Framework for Packing Linear Programs
This paper presents an acceleration framework for packing linear programming problems where the amount of data available is limited, i. e., where the number of constraints m is small compared to the variable dimension n. The framework can be used as a black box to speed up linear programming solvers dramatically, by two orders of magnitude in our experiments.
Online Convex Optimization Using Predictions
Making use of predictions is a crucial, but under-explored, area of online algorithms.
Approximate dynamic programming using fluid and diffusion approximations with applications to power management
Neuro-dynamic programming is a class of powerful techniques for approximating the solution to dynamic programming equations.
