no code implementations • 17 Apr 2024 • Erkan Bayram, Shenyu Liu, Mohamed-Ali Belabbas, Tamer Başar
Given a training set in the form of a paired $(\mathcal{X},\mathcal{Y})$, we say that the control system $\dot{x} = f(x, u)$ has learned the paired set via the control $u^*$ if the system steers each point of $\mathcal{X}$ to its corresponding target in $\mathcal{Y}$.
2 code implementations • 12 Apr 2024 • Haoran Qiu, Weichao Mao, Archit Patke, Shengkun Cui, Saurabh Jha, Chen Wang, Hubertus Franke, Zbigniew T. Kalbarczyk, Tamer Başar, Ravishankar K. Iyer
Large language models (LLMs) have been driving a new wave of interactive AI applications across numerous domains.
no code implementations • 3 Apr 2024 • Xiangyuan Zhang, Weichao Mao, Haoran Qiu, Tamer Başar
Closed-loop control of nonlinear dynamical systems with partial-state observability demands expert knowledge of a diverse, less standardized set of theoretical tools.
no code implementations • 25 Mar 2024 • Muhammad Aneeq uz Zaman, Shubham Aggarwal, Melih Bastopcu, Tamer Başar
In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies.
no code implementations • 17 Mar 2024 • Muhammad Aneeq uz Zaman, Alec Koppel, Mathieu Laurière, Tamer Başar
This MFTG NE is then shown to be $\mathcal{O}(1/M)$-NE for the finite population game where $M$ is a lower bound on the number of agents in each team.
no code implementations • 13 Mar 2024 • Raj Kiriti Velicheti, Melih Bastopcu, S. Rasoul Etesami, Tamer Başar
In this work, we consider an online version of information design where a sender interacts with a receiver of an unknown type who is adversarially chosen at each round.
no code implementations • 1 Mar 2024 • Xiangyuan Zhang, Saviz Mowlavi, Mouhacine Benosman, Tamer Başar
The PO step fine-tunes the model-based controller to compensate for the modeling error from dimensionality reduction.
no code implementations • 2 Feb 2024 • Weichao Mao, Haoran Qiu, Chen Wang, Hubertus Franke, Zbigniew Kalbarczyk, Tamer Başar
No-regret learning has a long history of being closely connected to game theory.
1 code implementation • 30 Nov 2023 • Xiangyuan Zhang, Weichao Mao, Saviz Mowlavi, Mouhacine Benosman, Tamer Başar
This project serves the learning for dynamics & control (L4DC) community, aiming to explore key questions: the convergence of RL algorithms in learning control policies; the stability and robustness issues of learning-based controllers; and the scalability of RL algorithms to high- and potentially infinite-dimensional systems.
1 code implementation • 9 Sep 2023 • Xiangyuan Zhang, Saviz Mowlavi, Mouhacine Benosman, Tamer Başar
We introduce the receding-horizon policy gradient (RHPG) algorithm, the first PG algorithm with provable global convergence in learning the optimal linear estimator designs, i. e., the Kalman filter (KF).
no code implementations • 16 Mar 2023 • Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar
A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit $\mathcal{C} < N$ on the number of transmission attempts at each instant.
no code implementations • 25 Feb 2023 • Xiangyuan Zhang, Tamer Başar
We revisit in this paper the discrete-time linear quadratic regulator (LQR) problem from the perspective of receding-horizon policy gradient (RHPG), a newly developed model-free learning framework for control applications.
no code implementations • 30 Jan 2023 • Xiangyuan Zhang, Bin Hu, Tamer Başar
We develop the first end-to-end sample complexity of model-free policy gradient (PG) methods in discrete-time infinite-horizon Kalman filtering.
no code implementations • 5 Dec 2022 • Leilei Cui, Tamer Başar, Zhong-Ping Jiang
This paper proposes a novel robust reinforcement learning framework for discrete-time linear systems with model mismatch that may arise from the sim-to-real gap.
no code implementations • NeurIPS 2020 • Yanli Liu, Kaiqing Zhang, Tamer Başar, Wotao Yin
In this paper, we revisit and improve the convergence of policy gradient (PG), natural PG (NPG) methods, and their variance-reduced variants, under general smooth policy parametrizations.
no code implementations • 10 Oct 2022 • Bin Hu, Kaiqing Zhang, Na Li, Mehran Mesbahi, Maryam Fazel, Tamer Başar
Gradient-based methods have been widely used for system design and optimization in diverse application domains.
no code implementations • 26 Sep 2022 • Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar
Due to a hard bandwidth constraint on the number of transmissions through the network, at most $R_d < N$ agents can concurrently access their state information through the network.
no code implementations • 24 Aug 2022 • Muhammad Aneeq uz Zaman, Alec Koppel, Sujay Bhatt, Tamer Başar
Given that the underlying Markov Decision Process (MDP) of the agent is communicating, we provide finite sample convergence guarantees in terms of convergence of the mean-field and control policy to the mean-field equilibrium.
no code implementations • 23 Jul 2022 • Sebin Gracy, Philip E. Paré, Ji Liu, Henrik Sandberg, Carolyn L. Beck, Karl Henrik Johansson, Tamer Başar
We establish a sufficient condition and multiple necessary conditions for local exponential convergence to the boundary equilibrium (i. e., one virus persists, the other one dies out) of each virus.
no code implementations • 6 Jun 2022 • Dongsheng Ding, Kaiqing Zhang, Jiali Duan, Tamer Başar, Mihailo R. Jovanović
We study sequential decision making problems aimed at maximizing the expected total reward while satisfying a constraint on the expected total utility.
no code implementations • 11 Mar 2022 • Shubham Aggarwal, Muhammad Aneeq uz Zaman, Tamer Başar
Since the complexity of solving the game increases with the number of agents, we use the Mean-Field Game paradigm to solve it.
no code implementations • 15 Dec 2021 • Zuguang Gao, Qianqian Ma, Tamer Başar, John R. Birge
With linear function approximation, the results are for convergence to a linear approximated equilibrium - a new notion of equilibrium that we propose - which describes that each agent's policy is a best reply (to other agents) within a linear space.
no code implementations • 12 Oct 2021 • Weichao Mao, Lin F. Yang, Kaiqing Zhang, Tamer Başar
Multi-agent reinforcement learning (MARL) algorithms often suffer from an exponential sample complexity dependence on the number of agents, a phenomenon known as \emph{the curse of multiagents}.
no code implementations • 12 Oct 2021 • Weichao Mao, Tamer Başar
We show that the agents can find an $\epsilon$-approximate CCE in at most $\widetilde{O}( H^6S A /\epsilon^2)$ episodes, where $S$ is the number of states, $A$ is the size of the largest individual action space, and $H$ is the length of an episode.
no code implementations • 29 Sep 2021 • Muhammad Aneeq uz Zaman, Sujay Bhatt, Tamer Başar
In this paper, we propose a game between an exogenous adversary and a network of agents connected via a multigraph.
no code implementations • 18 Jan 2021 • Sadegh Arefizadeh, Sadjaad Ozgoli, Sadegh Bolouki, Tamer Başar
A principal is tasked to optimize the network's performance by controlling the information available to each agent with regard to other agents' latest actions.
Dynamical Systems Systems and Control Systems and Control Optimization and Control
no code implementations • NeurIPS 2021 • Kaiqing Zhang, Xiangyuan Zhang, Bin Hu, Tamer Başar
Direct policy search serves as one of the workhorses in modern reinforcement learning (RL), and its applications in continuous control tasks have recently attracted increasing attention.
no code implementations • 7 Oct 2020 • Weichao Mao, Kaiqing Zhang, Ruihao Zhu, David Simchi-Levi, Tamer Başar
We consider model-free reinforcement learning (RL) in non-stationary Markov decision processes.
no code implementations • 9 Sep 2020 • Muhammad Aneeq uz Zaman, Kaiqing Zhang, Erik Miehling, Tamer Başar
We propose an actor-critic algorithm to iteratively compute the mean-field equilibrium (MFE) of the LQ-MFG.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • NeurIPS 2020 • Kaiqing Zhang, Sham M. Kakade, Tamer Başar, Lin F. Yang
This is in contrast to the usual reward-aware setting, with a $\tilde\Omega(|S|(|A|+|B|)(1-\gamma)^{-3}\epsilon^{-2})$ lower bound, where this model-based approach is near-optimal with only a gap on the $|A|,|B|$ dependence.
Model-based Reinforcement Learning Reinforcement Learning (RL)
no code implementations • NeurIPS 2020 • Weichao Mao, Kaiqing Zhang, Qiaomin Xie, Tamer Başar
Monte-Carlo planning, as exemplified by Monte-Carlo Tree Search (MCTS), has demonstrated remarkable performance in applications with finite spaces.
1 code implementation • 2 Apr 2020 • Weichao Mao, Kaiqing Zhang, Erik Miehling, Tamer Başar
To enable the development of tractable algorithms, we introduce the concept of an information state embedding that serves to compress agents' histories.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • 1 Mar 2020 • Xingyu Sha, Jia-Qi Zhang, Keyou You, Kaiqing Zhang, Tamer Başar
This paper proposes a \emph{fully asynchronous} scheme for the policy evaluation problem of distributed reinforcement learning (DisRL) over directed peer-to-peer networks.
no code implementations • 9 Dec 2019 • Kaiqing Zhang, Zhuoran Yang, Tamer Başar
Multi-agent reinforcement learning (MARL) has long been a significant and everlasting research topic in both machine learning and control.
no code implementations • 24 Nov 2019 • Kaiqing Zhang, Zhuoran Yang, Tamer Başar
Orthogonal to the existing reviews on MARL, we highlight several new angles and taxonomies of MARL theory, including learning in extensive-form games, decentralized MARL with networked agents, MARL in the mean-field regime, (non-)convergence of policy-based methods for learning in games, etc.
no code implementations • NeurIPS 2019 • Xiangyuan Zhang, Kaiqing Zhang, Erik Miehling, Tamer Başar
Through interacting with the more informed player, the less informed player attempts to both infer, and act according to, the true objective function.
no code implementations • 21 Oct 2019 • Kaiqing Zhang, Bin Hu, Tamer Başar
In this paper, we study the convergence theory of PO for $\mathcal{H}_2$ linear control with $\mathcal{H}_\infty$-norm robustness guarantee.
no code implementations • 6 Aug 2019 • Kaiqing Zhang, Erik Miehling, Tamer Başar
To demonstrate the applicability of the model, we propose a novel collaborative intrusion response model, where multiple agents (defenders) possessing asymmetric information aim to collaboratively defend a computer network.
no code implementations • 6 Jul 2019 • Yixuan Lin, Kaiqing Zhang, Zhuoran Yang, Zhaoran Wang, Tamer Başar, Romeil Sandhu, Ji Liu
This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors.
no code implementations • 19 Jun 2019 • Kaiqing Zhang, Alec Koppel, Hao Zhu, Tamer Başar
Under a further strict saddle points assumption, this result establishes convergence to essentially locally-optimal policies of the underlying problem, and thus bridges the gap in existing literature on the convergence of PG methods.
no code implementations • NeurIPS 2019 • Kaiqing Zhang, Zhuoran Yang, Tamer Başar
To the best of our knowledge, this work appears to be the first one to investigate the optimization landscape of LQ games, and provably show the convergence of policy optimization methods to the Nash equilibria.
no code implementations • 7 Dec 2018 • Tianyi Chen, Kaiqing Zhang, Georgios B. Giannakis, Tamer Başar
This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners.
no code implementations • 6 Dec 2018 • Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer Başar
This work appears to be the first finite-sample analysis for batch MARL, a step towards rigorous theoretical understanding of general MARL algorithms in the finite-sample regime.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • 19 Nov 2018 • Kaiqing Zhang, Yang Liu, Ji Liu, Mingyan Liu, Tamer Başar
This paper addresses the problem of distributed learning of average belief with sequential observations, in which a network of $n>1$ agents aim to reach a consensus on the average value of their beliefs, by exchanging information only with their neighbors.
5 code implementations • ICML 2018 • Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer Başar
To this end, we propose two decentralized actor-critic algorithms with function approximation, which are applicable to large-scale MARL problems where both the number of states and the number of agents are massively large.
Multi-agent Reinforcement Learning reinforcement-learning +1
no code implementations • 22 Feb 2018 • Muhammed O. Sayin, Chung-Wei Lin, Shinichi Shiraishi, Tamer Başar
We propose a reliable intersection control mechanism for strategic autonomous and connected vehicles (agents) in non-cooperative environments.
no code implementations • 3 Oct 2016 • Muhammed O. Sayin, Suleyman S. Kozat, Tamer Başar
Finally, in the numerical examples, we demonstrate the superior performance of the introduced algorithms in the finite-horizon MSE sense due to optimal estimation.