Multi-step dual control for exploration and exploitation in autonomous search with convergence guarantee

12 Mar 2022  ·  Yuan Tan, Jun Yang, Wen-Hua Chen, Shihua Li ·

Motivated by the recently proposed dual control for exploration and exploitation (DCEE) concept, this paper presents a Multi-Step DCEE (MS-DCEE) framework with guaranteed convergence for autonomous search of a source of airborne dispersion. Different from the existing stochastic model predictive control (SMPC) algorithm and informative path planning (IPP) approaches, the proposed MS-DCEE approach uses the current and future input to not only drive the agent towards the estimated source location (exploitation) but also reduce its estimation uncertainty (exploration) by actively learning the operational environment. Unknown source target position, together with unknown environment, impose significant challenges in establishing the recursive feasibility and the convergence of the proposed algorithm. To address them, with the help of the property of Bayesian estimation, we develop a two-step approach where the unbiasedness of the mean estimation is assumed first and then the randomness of the mean estimate under each collected information sequence is accounted. Based on that, we develop a MS-DCEE scheme with suitable terminal ingredients where recursive feasibility and convergence are guaranteed. Two simulation scenarios are conducted, which show that the proposed MS-DCEE algorithm outperforms the SMPC, the IPP and the single-step DCEE approaches in terms of searching successful rates and efficiency.

PDF Abstract
No code implementations yet. Submit your code now

Datasets


  Add Datasets introduced or used in this paper

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods


No methods listed for this paper. Add relevant methods here