Design of super-heated steam temperature adaptive PI controller based on actor-critic reinforcement learning
1. Institute of Geophysics and Geomatics, China University of Geosciences, Wuhan, Hubei 430074, China; 2. College of Electromechanical Engineering, Wuhan City Polytechnic, Wuhan, Hubei 430070, China; 3. School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China
Abstract： To solve the problem that the conventional PID with unsatisfactory control effect when the structure and parameters of the boiler superheated temperature model changed greatly, an adaptive PI controller was proposed based on actor-critical (AC) reinforcement learning (RL). A radial basis function neural network (RBF-NN) was used to realize AC-RL structure. The PI controller parameter was used as output of actor network and evaluated by the critical network to generate temporal difference (TD) error signal. The weights of RBF-NN were constantly updated by the TD error signal online. The structural characteristics of boiler super-heated steam temperature control system were introduced, and the RL-PI controller design and algorithm implementation steps were given to complete the design of boiler super-heated steam temperature control system. The simulations of the typical nonlinear time-varying super-heated steam temperature system were carried out with the proposed adaptive RL-PI controller under six working conditions of normal working conditions, gain increase, inertia increase, gain mutation, inertia mutation and disturbance. The results show that compared with model predictive control, fuzzy control and conventional PI cascade control methods, the proposed RL-PI controller has stronger self-learning ability, faster convergence speed and stronger robustness.
QIAN H, FENG Y Q. Design of adaptive predictive controller for superheated steam temperature control[J]. Journal of System Simulation, 2019,31(5):1002-1009. (in Chinese)
ZHANG J Y, GAO J, FENG X G. Main steam temperature control of thermal power boiler based on dynamic matrix control[J]. Journal of Xi′an Jiaotong University,2019,53(10): 96-102,174. (in Chinese)
WANG Z D, GUO J X, XIAO W. Photovoltaic system MPPT algorithm based on adaptive radial basis function neural network [J]．Journal of Hunan University（Natural Sciences）,2019,46(10):96-100. (in Chinese)
［4］
ASGHARNIA A, JAMALI A, SHAHNAZI R, et al. Load mitigation of a class of 5-MW wind turbine with RBF neural network based fractional-order PID controller [J]. ISA Transactions,2020,96:272-286.
YANG G, JIA Y X, CHEN X, et al. Stack overflow question post classification method based on deep lear-ning[J]. Journal of Jilin University (Science Edition), 2021,59（4）:922-928. (in Chinese)
LIU Y Q, WANG T H, XU X. New adaptive activation function for deep learning neural networks[J]. Journal of Jilin University (Science Edition), 2019,57（4）:857-859. (in Chinese)
［7］
DA SILVA F L, COSTA A H R. A survey on transfer learning for multiagent reinforcement learning systems[J]. Journal of Artificial Intelligence Research, 2019,64:645-703.
［8］
FORUZAN E, SOH L K, ASGARPOOR S．Reinforcement learning approach for optimal distributed energy management in a microgrid[J]．IEEE Transactions on Power Systems,2018,33(5):5749-5758.
［9］
XIE X W. PSS control of multi machine power system using reinforcement learning[C]∥Proceedings of the 2021 5th International Conference on Robotics and Automation Sciences. [S.l.]:IEEE,2021: 132-135.
[10]
LIM H K, KIM J B, HEO J S, et al. Federated reinforcement learning for training control policies on multiple IoT devices[J]. Sensors, doi: 10.3390/s20051359.
[11]
WANG Z, HONG T Z. Reinforcement learning for building controls: the opportunities and challenges [J]. Applied Energy, doi: 10.1016/j.apenergy.2020.115036.
2022, Vol. 43 
