Research status and prospect of deep reinforcement learning in automatic control
CAO Kai1, ZHU Yong1, GAO Qiang1*, LIU Jinhua2
1. National Research Center of Pumps, Jiangsu University, Zhenjiang 212013, China; 2. International Shipping Research Institute, GongQing Institute of Science and Technology, Jiujiang 332020, China
Abstract：Autonomous perception and intelligent control in complex environments are current research hotspots in the field of automatic control, and artificial intelligence provides the possibility for its realization. In recent years, with its strong perception and decision-making ability, deep reinforcement learning has demonstrated its excellent effect in solving global control problems under complex and unknown working conditions in the field of automatic control. Aiming at the successful application of deep reinforcement learning in automatic control field and its shortcomings in generalization, robustness and reliability, this paper summarizes the research progress of deep reinforcement learning and its application in automatic control field. Firstly, the development process and basic principle of deep reinforcement learning are briefly described. Secondly, deep reinforcement learning is divided into two types of algorithms: value function-based algorithm and policy gradient-based algorithm. Then, the basic principles, mathematical models, and improved methods of these two kinds of algorithms are discussed and analyzed systematically. Moreover, the applications of deep reinforcement learning in automatic control fields such as UAV flight control, mobile robot trajectory control, vehicle automatic driving control and hydraulic servo control are summarized. On this basis, the advantages and disadvantages of the deep reinforcement learning algorithm are summarized. Finally, the challenges and development trends of deep reinforcement learning in the field of automatic control are summarized and prospected, according to the cutting-edge research achievements in the field of automatic control in recent years, the research ideas of optimizing and solving the key problems of deep reinforcement learning are put forward and demonstrated accordingly.
曹凯,朱勇,高强*,刘金华. 深度强化学习在自动控制领域研究现状与展望[J]. 排灌机械工程学报, 2023, 41(6): 638-648.
CAO Kai,ZHU Yong,GAO Qiang*,LIU Jinhua. Research status and prospect of deep reinforcement learning in automatic control. Journal of Drainage and Irrigation Machinery Engin, 2023, 41(6): 638-648.
