Distributed Secondary Control Of Islanded Microgrids Based On Reinforcement Learning

As a beneficial supplement after the traditional power grid is connected to the distributed generators,microgrids can make use of various modern power technologies to make up for the uncertainty of the distributed generators output,and guarantee the power supply reliability of users to the greatest extent in the islanded state after the large power grid is disconnected.However,due to the small inertia time constant,the frequency voltage response speed of the islanded microgrids is particularly fast.Therefore,it is necessary to study the control problem of the islanded microgrids,so as to ensure the stability of the grid frequency and voltage.Due to the fact that inverters in isolated microgrids usually operate in droop control mode,the secondary control technology of islanded microgrids in the peer-to-peer operation mode are proposed to solve the voltage and frequency deviation caused by primary control.In order to make up for the shortcomings of traditional secondary control methods in controller design,communication,calculation and other aspects,from the aspects of distributed information interaction,secondary control method,operation control mode of microgrids,a distributed secondary control strategy of islanded microgrids based on reinforcement learning is proposed in this study to solve the frequency and voltage deviation of islanded microgrids.The main contents are as follows:(1)This study introduces three typical control methods of microgrids inverter: PQ control,V/f control and droop control,as well as the related control principle and model.The virtual impedance loop to realize droop decoupling control are applied according to the actual situation that the line resistance in microgrids cannot be ignored.Furthermore,the characteristics of the three operation control modes of microgrids,which are master-slave,peer-to-peer and hierarchical,are also studied to provide theoretical support for the subsequent secondary control of islanded microgrids.(2)Compared with the disadvantages of centralized control,the distributed system is adopted to realize the secondary control of islanded microgrids.It starts with a typical distributed information interaction method and further expands the use of pinning consistency algorithm to achieve distributed control.This method has the characteristics of few controller installations and simple communication structure.In the mode of peer-to-peer operation,a secondary control strategy of islanded microgrids.based on pinning control is proposed.By presetting the reference value of secondary control to the pinned point,the other agents search and synchronize to the pinned point according to the communication structure,which not only solves the problem of voltage and frequency deviation in the primary control of the islanded microgrids,but also adapts to the changing demand of the microgrids structure.(3)Considering the diversity of secondary control models and the complexity of problem solving,the reinforcement learning method is introduced to realize the objective of secondary control of islanded microgrids through intelligent decision-making.The typical Q learning and Q(λ)learning related model of two kinds of reinforcement learning method are introduced respectively,and then the pinning control method are combined furtherly.A distributed secondary control method for islanded microgrids based on QP learning(Q&Pinning learning)and QP(λ)learning(Q(λ)&Pinning learning)is proposed,in which the reinforcement learning actions are replaced with the pinning control method,which can not only ensure the characteristics of reinforcement learning autonomous learning control and intelligent decisionmaking,but also reduce the number of the reinforcement learning controllers and reduce communication and computing cost greatly.Then the implementation effects of multiple secondary control methods are compared and analyzed in the peer-to-peer control mode of islanded microgrids to explain the improved advantages of distributed secondary control of islanded microgrids based on reinforcement learning.