Search Results for author: Shengbo Eben Li

Found 53 papers, 14 papers with code

Canonical Form of Datatic Description in Control Systems

no code implementations4 Mar 2024 Guojian Zhan, Ziang Zheng, Shengbo Eben Li

This paper for the first time introduces the concept of canonical data form for the purpose of achieving more effective design of datatic controllers.

Attribute

A Cognitive-Based Trajectory Prediction Approach for Autonomous Driving

1 code implementation29 Feb 2024 Haicheng Liao, Yongkang Li, Zhenning Li, Chengyue Wang, Zhiyong Cui, Shengbo Eben Li, Chengzhong Xu

In autonomous vehicle (AV) technology, the ability to accurately predict the movements of surrounding vehicles is paramount for ensuring safety and operational efficiency.

Autonomous Driving Decision Making +2

On the Stability of Datatic Control Systems

no code implementations30 Jan 2024 Yujie Yang, Zhilong Zheng, Shengbo Eben Li

This information restricts the time derivative of any unknown state to the intersection of a set of closed balls.

Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model

1 code implementation19 Jan 2024 Yinan Zheng, Jianxiong Li, Dongjie Yu, Yujie Yang, Shengbo Eben Li, Xianyuan Zhan, Jingjing Liu

Interestingly, we discover that via reachability analysis of safe-control theory, the hard safety constraint can be equivalently translated to identifying the largest feasible region given the offline dataset.

Offline RL reinforcement-learning

BAT: Behavior-Aware Human-Like Trajectory Prediction for Autonomous Driving

1 code implementation11 Dec 2023 Haicheng Liao, Zhenning Li, Huanming Shen, Wenxuan Zeng, Dongping Liao, Guofa Li, Shengbo Eben Li, Chengzhong Xu

The ability to accurately predict the trajectory of surrounding vehicles is a critical hurdle to overcome on the journey to fully autonomous vehicles.

Autonomous Driving Decision Making +1

Integrated Drill Boom Hole-Seeking Control via Reinforcement Learning

no code implementations4 Dec 2023 Haoqi Yan, Haoyuan Xu, Hongbo Gao, Fei Ma, Shengbo Eben Li, Jingliang Duan

To tackle these challenges, this study proposes an integrated drill boom control method based on Reinforcement Learning (RL).

reinforcement-learning Reinforcement Learning (RL)

Training Multi-layer Neural Networks on Ising Machine

no code implementations6 Nov 2023 Xujie Song, Tong Liu, Shengbo Eben Li, Jingliang Duan, Wenxuan Wang, Keqiang Li

This paper proposes an Ising learning algorithm to train quantized neural network (QNN), by incorporating two essential techinques, namely binary representation of topological network and order reduction of loss function.

Optimization Landscape of Policy Gradient Methods for Discrete-time Static Output Feedback

no code implementations29 Oct 2023 Jingliang Duan, Jie Li, Xuyang Chen, Kai Zhao, Shengbo Eben Li, Lin Zhao

Despite the absence of convexity, we leverage these properties to derive novel findings regarding convergence (and nearly dimension-free rate) to stationary points for three policy gradient methods, including the vanilla policy gradient method, the natural policy gradient method, and the Gauss-Newton method.

Policy Gradient Methods

Robust Safe Reinforcement Learning under Adversarial Disturbances

no code implementations11 Oct 2023 Zeyang Li, Chuxiong Hu, Shengbo Eben Li, Jia Cheng, Yunan Wang

To address this challenge, this paper proposes a robust safe reinforcement learning framework that tackles worst-case disturbances.

reinforcement-learning Safe Reinforcement Learning

Bridging the Gap between Newton-Raphson Method and Regularized Policy Iteration

no code implementations11 Oct 2023 Zeyang Li, Chuxiong Hu, Yunan Wang, Guojian Zhan, Jie Li, Shengbo Eben Li

We also show that a modified version of regularized policy iteration, i. e., with finite-step policy evaluation, is equivalent to inexact Newton method where the Newton iteration formula is solved with truncated iterations.

DSAC-T: Distributional Soft Actor-Critic with Three Refinements

1 code implementation9 Oct 2023 Jingliang Duan, Wenxuan Wang, Liming Xiao, Jiaxin Gao, Shengbo Eben Li

Reinforcement learning (RL) has proven to be highly effective in tackling complex decision-making and control tasks.

Decision Making Reinforcement Learning (RL)

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

no code implementations4 Oct 2023 Hao Sha, Yao Mu, YuXuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, Mingyu Ding

Existing learning-based autonomous driving (AD) systems face challenges in comprehending high-level information, generalizing to rare events, and providing interpretability.

Autonomous Driving Decision Making

Learning Optimal Robust Control of Connected Vehicles in Mixed Traffic Flow

no code implementations18 Sep 2023 Jie Li, Jiawei Wang, Shengbo Eben Li, Keqiang Li

Connected and automated vehicles (CAVs) technologies promise to attenuate undesired traffic disturbances.

Safe Reinforcement Learning with Dual Robustness

no code implementations13 Sep 2023 Zeyang Li, Chuxiong Hu, Yunan Wang, Yujie Yang, Shengbo Eben Li

To address this issue, we propose a systematic framework to unify safe RL and robust RL, including problem formulation, iteration scheme, convergence analysis and practical algorithm design.

reinforcement-learning Reinforcement Learning (RL) +2

Feasible Policy Iteration

no code implementations18 Apr 2023 Yujie Yang, Zhilong Zheng, Shengbo Eben Li, Jingliang Duan, Jingjing Liu, Xianyuan Zhan, Ya-Qin Zhang

To address this challenge, we propose an indirect safe RL framework called feasible policy iteration, which guarantees that the feasible region monotonically expands and converges to the maximum one, and the state-value function monotonically improves and converges to the optimal one.

Reinforcement Learning (RL) Safe Reinforcement Learning

Smoothing Policy Iteration for Zero-sum Markov Games

no code implementations3 Dec 2022 Yangang Ren, Yao Lyu, Wenxuan Wang, Shengbo Eben Li, Zeyang Li, Jingliang Duan

In this paper, we propose the smoothing policy iteration (SPI) algorithm to solve the zero-sum MGs approximately, where the maximum operator is replaced by the weighted LogSumExp (WLSE) function to obtain the nearly optimal equilibrium policies.

Adversarial Robustness

Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model

no code implementations8 Oct 2022 Zeyu Gao, Yao Mu, Ruoyan Shen, Chen Chen, Yangang Ren, Jianyu Chen, Shengbo Eben Li, Ping Luo, YanFeng Lu

End-to-end autonomous driving provides a feasible way to automatically maximize overall driving system performance by directly mapping the raw pixels from a front-facing camera to control signals.

Autonomous Driving

Robust Bayesian Inference for Moving Horizon Estimation

no code implementations5 Oct 2022 Wenhan Cao, Chang Liu, Zhiqian Lan, Shengbo Eben Li, Wei Pan, Angelo Alessandri

The accuracy of moving horizon estimation (MHE) suffers significantly in the presence of measurement outliers.

Bayesian Inference Combinatorial Optimization

Performance-Driven Controller Tuning via Derivative-Free Reinforcement Learning

no code implementations11 Sep 2022 YuHeng Lei, Jianyu Chen, Shengbo Eben Li, Sifa Zheng

Choosing an appropriate parameter set for the designed controller is critical for the final performance but usually requires a tedious and careful tuning process, which implies a strong need for automatic tuning methods.

Autonomous Driving reinforcement-learning +1

Reachability Constrained Reinforcement Learning

2 code implementations16 May 2022 Dongjie Yu, Haitong Ma, Shengbo Eben Li, Jianyu Chen

Recent studies incorporate feasible sets into CRL with energy-based methods such as control barrier function (CBF), safety index (SI), and leverage prior conservative estimations of feasible sets, which harms the performance of the learned policy.

reinforcement-learning Reinforcement Learning (RL) +1

Primal-dual Estimator Learning: an Offline Constrained Moving Horizon Estimation Method with Feasibility and Near-optimality Guarantees

no code implementations6 Apr 2022 Wenhan Cao, Jingliang Duan, Shengbo Eben Li, Chen Chen, Chang Liu, Yu Wang

Both the primal and dual estimators are learned from data using supervised learning techniques, and the explicit sample size is provided, which enables us to guarantee the quality of each learned estimator in terms of feasibility and optimality.

Zeroth-Order Actor-Critic

no code implementations29 Jan 2022 YuHeng Lei, Jianyu Chen, Shengbo Eben Li, Sifa Zheng

The recent advanced evolution-based zeroth-order optimization methods and the policy gradient-based first-order methods are two promising alternatives to solve reinforcement learning (RL) problems with complementary advantages.

Continuous Control Reinforcement Learning (RL)

Learn Zero-Constraint-Violation Policy in Model-Free Constrained Reinforcement Learning

1 code implementation25 Nov 2021 Haitong Ma, Changliu Liu, Shengbo Eben Li, Sifa Zheng, Wenchao Sun, Jianyu Chen

Existing methods mostly use the posterior penalty for dangerous actions, which means that the agent is not penalized until experiencing danger.

reinforcement-learning Reinforcement Learning (RL)

Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learning

1 code implementation15 Nov 2021 Haitong Ma, Changliu Liu, Shengbo Eben Li, Sifa Zheng, Jianyu Chen

This paper proposes a novel approach that simultaneously synthesizes the energy-function-based safety certificate and learns the safe control policy with CRL.

reinforcement-learning Reinforcement Learning (RL) +1

Self-learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections

no code implementations24 Oct 2021 Yangang Ren, Jianhua Jiang, Dongjie Yu, Shengbo Eben Li, Jingliang Duan, Chen Chen, Keqiang Li

This paper develops the dynamic permutation state representation in the framework of integrated decision and control (IDC) to handle signalized intersections with mixed traffic flows.

Autonomous Driving Decision Making

Integrated Decision and Control at Multi-Lane Intersections with Mixed Traffic Flow

no code implementations30 Aug 2021 Jianhua Jiang, Yangang Ren, Yang Guan, Shengbo Eben Li, Yuming Yin, Xiaoping Jin

Autonomous driving at intersections is one of the most complicated and accident-prone traffic scenarios, especially with mixed traffic participants such as vehicles, bicycles and pedestrians.

Autonomous Driving Model Predictive Control +1

Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence

2 code implementations18 Mar 2021 Yang Guan, Yangang Ren, Qi Sun, Shengbo Eben Li, Haitong Ma, Jingliang Duan, Yifan Dai, Bo Cheng

In this paper, we present an interpretable and computationally efficient framework called integrated decision and control (IDC) for automated vehicles, which decomposes the driving task into static path planning and dynamic optimal tracking that are structured hierarchically.

Autonomous Driving Model-based Reinforcement Learning +2

Approximate Optimal Filter for Linear Gaussian Time-invariant Systems

no code implementations9 Mar 2021 Kaiming Tang, Shengbo Eben Li, Yuming Yin, Yang Guan, Jingliang Duan, Wenhan Cao, Jie Li

The equivalence holds given certain conditions about initial state distributions and policy formats, in which the system state is the estimation error, control input is the filter gain, and control objective function is the accumulated estimation error.

Decision-Making under On-Ramp merge Scenarios by Distributional Soft Actor-Critic Algorithm

no code implementations8 Mar 2021 Yiting Kong, Yang Guan, Jingliang Duan, Shengbo Eben Li, Qi Sun, Bingbing Nie

In this paper, we propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called the Shielded Distributional Soft Actor-critic (SDSAC).

Decision Making

Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function

1 code implementation2 Mar 2021 Haitong Ma, Jianyu Chen, Shengbo Eben Li, Ziyu Lin, Yang Guan, Yangang Ren, Sifa Zheng

Model information can be used to predict future trajectories, so it has huge potential to avoid dangerous region when implementing reinforcement learning (RL) on real-world tasks, like autonomous driving.

Autonomous Driving Collision Avoidance +3

Recurrent Model Predictive Control

no code implementations23 Feb 2021 Zhengyu Liu, Jingliang Duan, Wenxuan Wang, Shengbo Eben Li, Yuming Yin, Ziyu Lin, Qi Sun, Bo Cheng

This paper proposes an off-line algorithm, called Recurrent Model Predictive Control (RMPC), to solve general nonlinear finite-horizon optimal control problems.

Model Predictive Control

Mixed Policy Gradient: off-policy reinforcement learning driven jointly by data and model

2 code implementations23 Feb 2021 Yang Guan, Jingliang Duan, Shengbo Eben Li, Jie Li, Jianyu Chen, Bo Cheng

Formally, MPG is constructed as a weighted average of the data-driven and model-driven PGs, where the former is the derivative of the learned Q-value function, and the latter is that of the model-predictive return.

Decision Making Reinforcement Learning (RL)

Recurrent Model Predictive Control: Learning an Explicit Recurrent Controller for Nonlinear Systems

no code implementations20 Feb 2021 Zhengyu Liu, Jingliang Duan, Wenxuan Wang, Shengbo Eben Li, Yuming Yin, Ziyu Lin, Bo Cheng

This paper proposes an offline control algorithm, called Recurrent Model Predictive Control (RMPC), to solve large-scale nonlinear finite-horizon optimal control problems.

Model Predictive Control

Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning

no code implementations17 Feb 2021 Baiyu Peng, Yao Mu, Jingliang Duan, Yang Guan, Shengbo Eben Li, Jianyu Chen

Taking a control perspective, we first interpret the penalty method and the Lagrangian method as proportional feedback and integral feedback control, respectively.

Autonomous Driving reinforcement-learning +1

Steadily Learn to Drive with Virtual Memory

no code implementations16 Feb 2021 Yuhang Zhang, Yao Mu, Yujie Yang, Yang Guan, Shengbo Eben Li, Qi Sun, Jianyu Chen

Reinforcement learning has shown great potential in developing high-level autonomous driving.

Autonomous Driving

Robust Memory Augmentation by Constrained Latent Imagination

no code implementations1 Jan 2021 Yao Mu, Yuzheng Zhuang, Bin Wang, Wulong Liu, Shengbo Eben Li, Jianye Hao

The latent dynamics model summarizes an agent’s high dimensional experiences in a compact way.

Ternary Policy Iteration Algorithm for Nonlinear Robust Control

no code implementations14 Jul 2020 Jie Li, Shengbo Eben Li, Yang Guan, Jingliang Duan, Wenyu Li, Yuming Yin

The simulation results show that the TPI algorithm can converge to the optimal solution for the linear plant, and has high resistance to disturbances for the nonlinear plant.

Mixed Reinforcement Learning with Additive Stochastic Uncertainty

no code implementations28 Feb 2020 Yao Mu, Shengbo Eben Li, Chang Liu, Qi Sun, Bingbing Nie, Bo Cheng, Baiyu Peng

This paper presents a mixed reinforcement learning (mixed RL) algorithm by simultaneously using dual representations of environmental dynamics to search the optimal policy with the purpose of improving both learning accuracy and training speed.

reinforcement-learning Reinforcement Learning (RL)

Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic

no code implementations13 Feb 2020 Yangang Ren, Jingliang Duan, Shengbo Eben Li, Yang Guan, Qi Sun

In this paper, we introduce the minimax formulation and distributional framework to improve the generalization ability of RL algorithms and develop the Minimax Distributional Soft Actor-Critic (Minimax DSAC) algorithm.

Autonomous Driving Decision Making +2

Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errors

3 code implementations9 Jan 2020 Jingliang Duan, Yang Guan, Shengbo Eben Li, Yangang Ren, Bo Cheng

In reinforcement learning (RL), function approximation errors are known to easily lead to the Q-value overestimations, thus greatly reducing policy performance.

Continuous Control reinforcement-learning +1

Direct and indirect reinforcement learning

no code implementations23 Dec 2019 Yang Guan, Shengbo Eben Li, Jingliang Duan, Jie Li, Yangang Ren, Qi Sun, Bo Cheng

Reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks.

Decision Making reinforcement-learning +1

Adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints

no code implementations26 Nov 2019 Jingliang Duan, Zhengyu Liu, Shengbo Eben Li, Qi Sun, Zhenzhong Jia, Bo Cheng

CADP linearizes the constrained optimization problem locally into a quadratically constrained linear programming problem, and then obtains the optimal update of the policy network by solving its dual problem.

Relaxed Actor-Critic with Convergence Guarantees for Continuous-Time Optimal Control of Nonlinear Systems

no code implementations11 Sep 2019 Jingliang Duan, Jie Li, Qiang Ge, Shengbo Eben Li, Monimoy Bujarbaruah, Fei Ma, Dezhao Zhang

The warm-up phase minimizes the square of the Hamiltonian to achieve admissibility, while the generalized policy iteration phase relaxes the update termination conditions for faster convergence.

Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks

no code implementations6 Jun 2019 Long Xin, Pin Wang, Ching-Yao Chan, Jianyu Chen, Shengbo Eben Li, Bo Cheng

As autonomous vehicles (AVs) need to interact with other road users, it is of importance to comprehensively understand the dynamic traffic environment, especially the future possible trajectories of surrounding vehicles.

Autonomous Vehicles feature selection +2

Distributed Model Predictive Control for Heterogeneous V ehicle Platoons Under Unidirectional Topologies

no code implementations3 Mar 2017 Y ang Zheng, Shengbo Eben Li, Keqiang Li, Francesco Borrelli

This paper presents a distributed model predictive control (DMPC) algorithm for heterogeneous vehicle platoons with unidirectional topologies and a p r i o r i unknown desired set point.

Model Predictive Control

Cannot find the paper you are looking for? You can Submit a new open access paper.