no code implementations • 11 Sep 2019 • Jingliang Duan, Jie Li, Qiang Ge, Shengbo Eben Li, Monimoy Bujarbaruah, Fei Ma, Dezhao Zhang
The warm-up phase minimizes the square of the Hamiltonian to achieve admissibility, while the generalized policy iteration phase relaxes the update termination conditions for faster convergence.
no code implementations • 26 Nov 2019 • Jingliang Duan, Zhengyu Liu, Shengbo Eben Li, Qi Sun, Zhenzhong Jia, Bo Cheng
CADP linearizes the constrained optimization problem locally into a quadratically constrained linear programming problem, and then obtains the optimal update of the policy network by solving its dual problem.
no code implementations • 23 Dec 2019 • Yang Guan, Shengbo Eben Li, Jingliang Duan, Jie Li, Yangang Ren, Qi Sun, Bo Cheng
Reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks.
3 code implementations • 9 Jan 2020 • Jingliang Duan, Yang Guan, Shengbo Eben Li, Yangang Ren, Bo Cheng
In reinforcement learning (RL), function approximation errors are known to easily lead to the Q-value overestimations, thus greatly reducing policy performance.
no code implementations • 13 Feb 2020 • Yangang Ren, Jingliang Duan, Shengbo Eben Li, Yang Guan, Qi Sun
In this paper, we introduce the minimax formulation and distributional framework to improve the generalization ability of RL algorithms and develop the Minimax Distributional Soft Actor-Critic (Minimax DSAC) algorithm.
no code implementations • 3 Mar 2020 • Lu Wen, Jingliang Duan, Shengbo Eben Li, Shaobing Xu, Huei Peng
The simulations of two scenarios for autonomous vehicles confirm we can ensure safety while achieving fast learning.
no code implementations • 14 Jul 2020 • Jie Li, Shengbo Eben Li, Yang Guan, Jingliang Duan, Wenyu Li, Yuming Yin
The simulation results show that the TPI algorithm can converge to the optimal solution for the linear plant, and has high resistance to disturbances for the nonlinear plant.
no code implementations • 17 Feb 2021 • Baiyu Peng, Yao Mu, Jingliang Duan, Yang Guan, Shengbo Eben Li, Jianyu Chen
Taking a control perspective, we first interpret the penalty method and the Lagrangian method as proportional feedback and integral feedback control, respectively.
no code implementations • 20 Feb 2021 • Zhengyu Liu, Jingliang Duan, Wenxuan Wang, Shengbo Eben Li, Yuming Yin, Ziyu Lin, Bo Cheng
This paper proposes an offline control algorithm, called Recurrent Model Predictive Control (RMPC), to solve large-scale nonlinear finite-horizon optimal control problems.
2 code implementations • 23 Feb 2021 • Yang Guan, Jingliang Duan, Shengbo Eben Li, Jie Li, Jianyu Chen, Bo Cheng
Formally, MPG is constructed as a weighted average of the data-driven and model-driven PGs, where the former is the derivative of the learned Q-value function, and the latter is that of the model-predictive return.
no code implementations • 23 Feb 2021 • Zhengyu Liu, Jingliang Duan, Wenxuan Wang, Shengbo Eben Li, Yuming Yin, Ziyu Lin, Qi Sun, Bo Cheng
This paper proposes an off-line algorithm, called Recurrent Model Predictive Control (RMPC), to solve general nonlinear finite-horizon optimal control problems.
no code implementations • 8 Mar 2021 • Yiting Kong, Yang Guan, Jingliang Duan, Shengbo Eben Li, Qi Sun, Bingbing Nie
In this paper, we propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called the Shielded Distributional Soft Actor-critic (SDSAC).
no code implementations • 9 Mar 2021 • Kaiming Tang, Shengbo Eben Li, Yuming Yin, Yang Guan, Jingliang Duan, Wenhan Cao, Jie Li
The equivalence holds given certain conditions about initial state distributions and policy formats, in which the system state is the estimation error, control input is the filter gain, and control objective function is the accumulated estimation error.
2 code implementations • 18 Mar 2021 • Yang Guan, Yangang Ren, Qi Sun, Shengbo Eben Li, Haitong Ma, Jingliang Duan, Yifan Dai, Bo Cheng
In this paper, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically.
no code implementations • 26 Aug 2021 • Baiyu Peng, Jingliang Duan, Jianyu Chen, Shengbo Eben Li, Genjin Xie, Congsheng Zhang, Yang Guan, Yao Mu, Enxin Sun
Based on this, the penalty method is formulated as a proportional controller, and the Lagrangian method is formulated as an integral controller.
no code implementations • 24 Oct 2021 • Yangang Ren, Jianhua Jiang, Dongjie Yu, Shengbo Eben Li, Jingliang Duan, Chen Chen, Keqiang Li
This paper develops the dynamic permutation state representation in the framework of integrated decision and control (IDC) to handle signalized intersections with mixed traffic flows.
no code implementations • 6 Apr 2022 • Wenhan Cao, Jingliang Duan, Shengbo Eben Li, Chen Chen, Chang Liu, Yu Wang
Both the primal and dual estimators are learned from data using supervised learning techniques, and the explicit sample size is provided, which enables us to guarantee the quality of each learned estimator in terms of feasibility and optimality.
no code implementations • 18 Aug 2022 • Xuyang Chen, Jingliang Duan, Yingbin Liang, Lin Zhao
To our knowledge, this is the first finite-time convergence analysis for the single sample two-timescale AC for solving LQR with global optimality.
no code implementations • 12 Sep 2022 • Jingliang Duan, Wenhan Cao, Yang Zheng, Lin Zhao
At the core of our results is the uniqueness of the stationary point of dLQR when it is observable, which is in a concise form of an observer-based controller with the optimal similarity transformation.
1 code implementation • 14 Oct 2022 • Dongjie Yu, Wenjun Zou, Yujie Yang, Haitong Ma, Shengbo Eben Li, Jingliang Duan, Jianyu Chen
Furthermore, we build a safe RL framework to resolve constraints required by the DRC and its corresponding shield policy.
Model-based Reinforcement Learning reinforcement-learning +2
no code implementations • 3 Dec 2022 • Yangang Ren, Yao Lyu, Wenxuan Wang, Shengbo Eben Li, Zeyang Li, Jingliang Duan
In this paper, we propose the smoothing policy iteration (SPI) algorithm to solve the zero-sum MGs approximately, where the maximum operator is replaced by the weighted LogSumExp (WLSE) function to obtain the nearly optimal equilibrium policies.
no code implementations • 18 Apr 2023 • Yujie Yang, Zhilong Zheng, Shengbo Eben Li, Jingliang Duan, Jingjing Liu, Xianyuan Zhan, Ya-Qin Zhang
To address this challenge, we propose an indirect safe RL framework called feasible policy iteration, which guarantees that the feasible region monotonically expands and converges to the maximum one, and the state-value function monotonically improves and converges to the optimal one.
1 code implementation • 9 Oct 2023 • Jingliang Duan, Wenxuan Wang, Liming Xiao, Jiaxin Gao, Shengbo Eben Li
Reinforcement learning (RL) has proven to be highly effective in tackling complex decision-making and control tasks.
no code implementations • 29 Oct 2023 • Jingliang Duan, Jie Li, Xuyang Chen, Kai Zhao, Shengbo Eben Li, Lin Zhao
Despite the absence of convexity, we leverage these properties to derive novel findings regarding convergence (and nearly dimension-free rate) to stationary points for three policy gradient methods, including the vanilla policy gradient method, the natural policy gradient method, and the Gauss-Newton method.
no code implementations • 6 Nov 2023 • Xujie Song, Tong Liu, Shengbo Eben Li, Jingliang Duan, Wenxuan Wang, Keqiang Li
This paper proposes an Ising learning algorithm to train quantized neural network (QNN), by incorporating two essential techinques, namely binary representation of topological network and order reduction of loss function.
no code implementations • 4 Dec 2023 • Haoqi Yan, Haoyuan Xu, Hongbo Gao, Fei Ma, Shengbo Eben Li, Jingliang Duan
To tackle these challenges, this study proposes an integrated drill boom control method based on Reinforcement Learning (RL).
1 code implementation • 19 Mar 2024 • Wenjun Zou, Yao Lyu, Jie Li, Yujie Yang, Shengbo Eben Li, Jingliang Duan, Xianyuan Zhan, Jingjing Liu, Yaqin Zhang, Keqiang Li
Safe reinforcement learning (RL) offers advanced solutions to constrained optimal control problems.