no code implementations • 17 Apr 2024 • Erkan Bayram, Shenyu Liu, Mohamed-Ali Belabbas, Tamer Başar

Given a training set in the form of a paired $(\mathcal{X},\mathcal{Y})$, we say that the control system $\dot x = f(x, u)$ has learned the paired set via the control $u^*$ if the system steers each point of $\mathcal{X}$ to its corresponding target in $\mathcal{Y}$.

1 code implementation • 12 Apr 2024 • Haoran Qiu, Weichao Mao, Archit Patke, Shengkun Cui, Saurabh Jha, Chen Wang, Hubertus Franke, Zbigniew T. Kalbarczyk, Tamer Başar, Ravishankar K. Iyer

Large language models (LLMs) have been driving a new wave of interactive AI applications across numerous domains.

no code implementations • 3 Apr 2024 • Xiangyuan Zhang, Weichao Mao, Haoran Qiu, Tamer Başar

Closed-loop control of nonlinear dynamical systems with partial-state observability demands expert knowledge of a diverse, less standardized set of theoretical tools.

no code implementations • 25 Mar 2024 • Muhammad Aneeq uz Zaman, Shubham Aggarwal, Melih Bastopcu, Tamer Başar

In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies.

no code implementations • 17 Mar 2024 • Muhammad Aneeq uz Zaman, Alec Koppel, Mathieu Laurière, Tamer Başar

This MFTG NE is then shown to be $\mathcal{O}(1/M)$-NE for the finite population game where $M$ is a lower bound on the number of agents in each team.

no code implementations • 13 Mar 2024 • Raj Kiriti Velicheti, Melih Bastopcu, S. Rasoul Etesami, Tamer Başar

In this work, we consider an online version of information design where a sender interacts with a receiver of an unknown type who is adversarially chosen at each round.

no code implementations • 1 Mar 2024 • Xiangyuan Zhang, Saviz Mowlavi, Mouhacine Benosman, Tamer Başar

The PO step fine-tunes the model-based controller to compensate for the modeling error from dimensionality reduction.

no code implementations • 2 Feb 2024 • Weichao Mao, Haoran Qiu, Chen Wang, Hubertus Franke, Zbigniew Kalbarczyk, Tamer Başar

No-regret learning has a long history of being closely connected to game theory.

1 code implementation • 30 Nov 2023 • Xiangyuan Zhang, Weichao Mao, Saviz Mowlavi, Mouhacine Benosman, Tamer Başar

This project serves the learning for dynamics & control (L4DC) community, aiming to explore key questions: the convergence of RL algorithms in learning control policies; the stability and robustness issues of learning-based controllers; and the scalability of RL algorithms to high- and potentially infinite-dimensional systems.

1 code implementation • 9 Sep 2023 • Xiangyuan Zhang, Saviz Mowlavi, Mouhacine Benosman, Tamer Başar

We introduce the receding-horizon policy gradient (RHPG) algorithm, the first PG algorithm with provable global convergence in learning the optimal linear estimator designs, i. e., the Kalman filter (KF).

no code implementations • 16 Mar 2023 • Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit $\mathcal{C} < N$ on the number of transmission attempts at each instant.

no code implementations • 25 Feb 2023 • Xiangyuan Zhang, Tamer Başar

We revisit in this paper the discrete-time linear quadratic regulator (LQR) problem from the perspective of receding-horizon policy gradient (RHPG), a newly developed model-free learning framework for control applications.

no code implementations • 30 Jan 2023 • Xiangyuan Zhang, Bin Hu, Tamer Başar

We develop the first end-to-end sample complexity of model-free policy gradient (PG) methods in discrete-time infinite-horizon Kalman filtering.

no code implementations • 5 Dec 2022 • Leilei Cui, Tamer Başar, Zhong-Ping Jiang

This paper proposes a novel robust reinforcement learning framework for discrete-time linear systems with model mismatch that may arise from the sim-to-real gap.

no code implementations • NeurIPS 2020 • Yanli Liu, Kaiqing Zhang, Tamer Başar, Wotao Yin

In this paper, we revisit and improve the convergence of policy gradient (PG), natural PG (NPG) methods, and their variance-reduced variants, under general smooth policy parametrizations.

no code implementations • 10 Oct 2022 • Bin Hu, Kaiqing Zhang, Na Li, Mehran Mesbahi, Maryam Fazel, Tamer Başar

Gradient-based methods have been widely used for system design and optimization in diverse application domains.

no code implementations • 26 Sep 2022 • Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

Due to a hard bandwidth constraint on the number of transmissions through the network, at most $R_d < N$ agents can concurrently access their state information through the network.

no code implementations • 24 Aug 2022 • Muhammad Aneeq uz Zaman, Alec Koppel, Sujay Bhatt, Tamer Başar

Given that the underlying Markov Decision Process (MDP) of the agent is communicating, we provide finite sample convergence guarantees in terms of convergence of the mean-field and control policy to the mean-field equilibrium.

no code implementations • 23 Jul 2022 • Sebin Gracy, Philip E. Paré, Ji Liu, Henrik Sandberg, Carolyn L. Beck, Karl Henrik Johansson, Tamer Başar

We establish a sufficient condition and multiple necessary conditions for local exponential convergence to the boundary equilibrium (i. e., one virus persists, the other one dies out) of each virus.

no code implementations • 6 Jun 2022 • Dongsheng Ding, Kaiqing Zhang, Jiali Duan, Tamer Başar, Mihailo R. Jovanović

We study sequential decision making problems aimed at maximizing the expected total reward while satisfying a constraint on the expected total utility.

no code implementations • 11 Mar 2022 • Shubham Aggarwal, Muhammad Aneeq uz Zaman, Tamer Başar

Since the complexity of solving the game increases with the number of agents, we use the Mean-Field Game paradigm to solve it.

no code implementations • 15 Dec 2021 • Zuguang Gao, Qianqian Ma, Tamer Başar, John R. Birge

With linear function approximation, the results are for convergence to a linear approximated equilibrium - a new notion of equilibrium that we propose - which describes that each agent's policy is a best reply (to other agents) within a linear space.

no code implementations • 12 Oct 2021 • Weichao Mao, Lin F. Yang, Kaiqing Zhang, Tamer Başar

Multi-agent reinforcement learning (MARL) algorithms often suffer from an exponential sample complexity dependence on the number of agents, a phenomenon known as \emph{the curse of multiagents}.

no code implementations • 12 Oct 2021 • Weichao Mao, Tamer Başar

We show that the agents can find an $\epsilon$-approximate CCE in at most $\widetilde{O}( H^6S A /\epsilon^2)$ episodes, where $S$ is the number of states, $A$ is the size of the largest individual action space, and $H$ is the length of an episode.

no code implementations • 29 Sep 2021 • Muhammad Aneeq uz Zaman, Sujay Bhatt, Tamer Başar

In this paper, we propose a game between an exogenous adversary and a network of agents connected via a multigraph.

no code implementations • 18 Jan 2021 • Sadegh Arefizadeh, Sadjaad Ozgoli, Sadegh Bolouki, Tamer Başar

A principal is tasked to optimize the network's performance by controlling the information available to each agent with regard to other agents' latest actions.

Dynamical Systems Systems and Control Systems and Control Optimization and Control

no code implementations • NeurIPS 2021 • Kaiqing Zhang, Xiangyuan Zhang, Bin Hu, Tamer Başar

Direct policy search serves as one of the workhorses in modern reinforcement learning (RL), and its applications in continuous control tasks have recently attracted increasing attention.

no code implementations • 7 Oct 2020 • Weichao Mao, Kaiqing Zhang, Ruihao Zhu, David Simchi-Levi, Tamer Başar

We consider model-free reinforcement learning (RL) in non-stationary Markov decision processes.

no code implementations • 9 Sep 2020 • Muhammad Aneeq uz Zaman, Kaiqing Zhang, Erik Miehling, Tamer Başar

We propose an actor-critic algorithm to iteratively compute the mean-field equilibrium (MFE) of the LQ-MFG.

Multi-agent Reinforcement Learning
reinforcement-learning
**+1**

no code implementations • NeurIPS 2020 • Kaiqing Zhang, Sham M. Kakade, Tamer Başar, Lin F. Yang

This is in contrast to the usual reward-aware setting, with a $\tilde\Omega(|S|(|A|+|B|)(1-\gamma)^{-3}\epsilon^{-2})$ lower bound, where this model-based approach is near-optimal with only a gap on the $|A|,|B|$ dependence.

Model-based Reinforcement Learning Reinforcement Learning (RL)

no code implementations • NeurIPS 2020 • Weichao Mao, Kaiqing Zhang, Qiaomin Xie, Tamer Başar

Monte-Carlo planning, as exemplified by Monte-Carlo Tree Search (MCTS), has demonstrated remarkable performance in applications with finite spaces.

1 code implementation • 2 Apr 2020 • Weichao Mao, Kaiqing Zhang, Erik Miehling, Tamer Başar

To enable the development of tractable algorithms, we introduce the concept of an information state embedding that serves to compress agents' histories.

Multi-agent Reinforcement Learning
reinforcement-learning
**+1**

no code implementations • 1 Mar 2020 • Xingyu Sha, Jia-Qi Zhang, Keyou You, Kaiqing Zhang, Tamer Başar

This paper proposes a \emph{fully asynchronous} scheme for the policy evaluation problem of distributed reinforcement learning (DisRL) over directed peer-to-peer networks.

no code implementations • 9 Dec 2019 • Kaiqing Zhang, Zhuoran Yang, Tamer Başar

Multi-agent reinforcement learning (MARL) has long been a significant and everlasting research topic in both machine learning and control.

no code implementations • 24 Nov 2019 • Kaiqing Zhang, Zhuoran Yang, Tamer Başar

Orthogonal to the existing reviews on MARL, we highlight several new angles and taxonomies of MARL theory, including learning in extensive-form games, decentralized MARL with networked agents, MARL in the mean-field regime, (non-)convergence of policy-based methods for learning in games, etc.

no code implementations • NeurIPS 2019 • Xiangyuan Zhang, Kaiqing Zhang, Erik Miehling, Tamer Başar

Through interacting with the more informed player, the less informed player attempts to both infer, and act according to, the true objective function.

no code implementations • 21 Oct 2019 • Kaiqing Zhang, Bin Hu, Tamer Başar

In this paper, we study the convergence theory of PO for $\mathcal{H}_2$ linear control with $\mathcal{H}_\infty$-norm robustness guarantee.

no code implementations • 6 Aug 2019 • Kaiqing Zhang, Erik Miehling, Tamer Başar

To demonstrate the applicability of the model, we propose a novel collaborative intrusion response model, where multiple agents (defenders) possessing asymmetric information aim to collaboratively defend a computer network.

no code implementations • 6 Jul 2019 • Yixuan Lin, Kaiqing Zhang, Zhuoran Yang, Zhaoran Wang, Tamer Başar, Romeil Sandhu, Ji Liu

This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors.

no code implementations • 19 Jun 2019 • Kaiqing Zhang, Alec Koppel, Hao Zhu, Tamer Başar

Under a further strict saddle points assumption, this result establishes convergence to essentially locally-optimal policies of the underlying problem, and thus bridges the gap in existing literature on the convergence of PG methods.

no code implementations • NeurIPS 2019 • Kaiqing Zhang, Zhuoran Yang, Tamer Başar

To the best of our knowledge, this work appears to be the first one to investigate the optimization landscape of LQ games, and provably show the convergence of policy optimization methods to the Nash equilibria.

no code implementations • 7 Dec 2018 • Tianyi Chen, Kaiqing Zhang, Georgios B. Giannakis, Tamer Başar

This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners.

no code implementations • 6 Dec 2018 • Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer Başar

This work appears to be the first finite-sample analysis for batch MARL, a step towards rigorous theoretical understanding of general MARL algorithms in the finite-sample regime.

Multi-agent Reinforcement Learning
reinforcement-learning
**+1**

no code implementations • 19 Nov 2018 • Kaiqing Zhang, Yang Liu, Ji Liu, Mingyan Liu, Tamer Başar

This paper addresses the problem of distributed learning of average belief with sequential observations, in which a network of $n>1$ agents aim to reach a consensus on the average value of their beliefs, by exchanging information only with their neighbors.

5 code implementations • ICML 2018 • Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, Tamer Başar

To this end, we propose two decentralized actor-critic algorithms with function approximation, which are applicable to large-scale MARL problems where both the number of states and the number of agents are massively large.

Multi-agent Reinforcement Learning
reinforcement-learning
**+1**

no code implementations • 22 Feb 2018 • Muhammed O. Sayin, Chung-Wei Lin, Shinichi Shiraishi, Tamer Başar

We propose a reliable intersection control mechanism for strategic autonomous and connected vehicles (agents) in non-cooperative environments.

no code implementations • 3 Oct 2016 • Muhammed O. Sayin, Suleyman S. Kozat, Tamer Başar

Finally, in the numerical examples, we demonstrate the superior performance of the introduced algorithms in the finite-horizon MSE sense due to optimal estimation.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.