Study On Optimal Frequency Control Of Microgrid Via Reinforcement Learning

As an important development paradigm of smart grid in the future,microgrid has attracted much attention of scholars.Compared with traditional power system,microgrids can make full use of various distributed energy sources and have better flexibility,economy and reliability.However,microgrid is a complex cyber-physical system,and its dynamics have many complex characteristics,such as strong nonlinearity,parameter uncertainty.The control and optimization of microgrid have always been a difficult problem.The systematic study of the intelligent control and optimization of microgrid is not only conducive to the advancement of complex nonlinear control theory,but also can provide a reliable theoretical basis for the actual microgrid construction.Aiming at the complex characteristics such as strong nonlinearity,parameter uncertainty and low inertia,the optimal frequency control in islanded microgrids are studied with reinforcement learning,optimal control,distributed cooperative control and Lyapunov stability.The main contents of this thesis are as follows.For the strong nonlinearity of microgrids,the optimal distributed secondary frequencycontrol is studied based on the offline reinforcement learning algorithm.A data-driven based distributed optimal frequency control framework is proposed.A dual heuristic dynamic programming algorithm is designed to numerically solve the discrete-time Hamilton-Jacobi-Bellman equation of the optimal frequency control,so as to obtain a quadratic frequency controller with optimal performance.A rigorous theoretical analysis of the proposed algorithm is made and an experimental platform is built for simulation experiments.The research results show that the designed controller has the advantages of adjustment and it is model-free.With the proposed controller,we can realize the proportional distribution of load power while maintaining the stability of the microgrid frequency quickly and efficiently.For the parameter uncertainty of the microgrids,the optimal secondary frequency control of the microgrid is studied based on the online reinforcement learning algorithm.The continuous-time dynamics model of the frequency dynamic system of the islanded AC microgrid is established.An adaptive frequency controller based on the online reinforcement learning algorithm is proposed.A data-based value iteration algorithm is designed to numerically solve the algebraic Riccati equation of the continuous-time microgrid system,so as to obtain the optimal feedback control gain.A rigorous theoretical analysis of the designed algorithm is proved and an experimental platform is built for simulation.The results show that the designed controller has the same frequency adjustment function as the linear quadratic regulator,but it does not depend on the parameters of the microgrid system,i.e.,it has the advantages of model-free control and adaptive control.Compared with the existing distributed controller,the designed controller has better dynamic performance.For the parameter uncertainty of the microgrid system,the optimal finite time frequency control of microgrid is studied based on online reinforcement learning algorithm.In order to further improve the efficiency of frequency control of microgrid and ensure optimal control performance,an optimal finite time frequency control scheme based on online reinforcement learning algorithm is proposed.An optimal finite-time frequency controller is designed.And after rigorous theoretical analysis,sufficient conditions have been obtained to ensure that the closed-loop system has optimal performance and finite time stability.Based on the inverse optimal control theory,a construction method of fractional loss function is proposed.An online reinforcement learning algorithm is proposed to numerically solve the continuous-time Hamilton-Jacobi-Bellman equation to obtain the optimal finite-time frequency controller.Numerical simulations are used to verify the effectiveness of the iterative learning algorithm.Finally,a simulation experiment is carried out based on the MATLAB\Simscape toolbox.The results show that the designed controller has the advantages of model-free and adaptive control.Especially when the system parameters change,compared with the quadratic regulator with the best performance,the stability time of the proposed controller is more controllable.For the low inertia of the microgrid,the anti-disturbance control of the microgrid frequency system is studied based on the online reinforcement learning algorithm.In order to improve the anti-disturbance performance of the microgrid,an adaptive sliding mode controller is designed based on the online reinforcement learning algorithm.Firstly,a state space model of load frequency control is established based on the small signal analysis theory.The load frequency control problem is decomposed into two sub-problems by replacing variables,the design of optimal sliding mode surface sliding and the design of second-order sliding mode controller.An online iterative algorithm is designed to obtain the optimal sliding mode surface parameters.Based on the Lyapuno stability theory,the closed-loop stability of the second-order sliding mode system is analyzed.And the sufficient conditions for the finite-time stability is given.A microgrid test system based on Simscap is built to verify the effectiveness of the proposed controller.The experimental results show that,compared with PID and H_?optimal controller,the closed-loop system have better robust stability under the proposed controller.