Research On Automatic Power Generation Control Based On Multi-agent Transfer Reinforcement Learnin

In recent years,in order to address the issues of energy shortage and environmental pollution,the complementary systems of water-fire-light power generation and electric heating-cooling power generation have gradually achieved success.However,due to the uncertainty and volatility of renewable energy,the grid’s frequency control capability decreases as new energy sources are integrated into the system,making stable operation difficult.Automatic generation control(AGC)is one of the important means to solve the problem of stable operation of the power grid caused by load changes.This paper explores and solves the above problems based on the multi-energy complementary generation system as the research object,using multi-agent transfer reinforcement learning algorithm to obtain the optimal control strategy in the complex multi-region multi-energy generation system environment.Firstly,in order to adapt to the development of green economy and promote the transformation of energy structure,the traditional power generation system has gradually transformed into a multi-energy complementary generation system.This paper further discusses the importance of achieving AGC in a complex multi-region multi-energy complementary power generation system for ensuring its safe and stable operation.Additionally,the current domestic and foreign frequency requirements for AGC and the status of different AGC strategies are also discussed.Secondly,this paper categorizes the AGC frequency regulation methods,and then classifies the secondary frequency control in AGC according to the control objects.Subsequently,a modeling analysis is conducted on the multi-energy complementary generation system model,including the derivation of transfer functions and the establishment of mathematical models for the components of the system,such as prime movers,governors,tie-lines,micro gas turbines,battery energy storage,and small hydropower.Finally,a detailed discussion is provided on markov decision processes,bellman equations,the principles of reinforcement learning,and the pros and cons of various deep reinforcement learning methods,as well as their development.Furthermore,this paper proposes an AGC algorithm based on multi-agent Proximal Policy Optimization with Convolutional Neural Network(MAPPO-CNN).Firstly,the proximal policy optimization algorithm is used to limit the update range of the new and old policies to a reasonable range.Then,a convolutional neural network is introduced to enhance the information extraction ability of the agents for collecting data.Finally,multi-agents enable information sharing between different regions.Simulations on IEEE standard two-area load frequency control system model and five-region multienergy complementary generation system model demonstrate that the proposed MAPPO-CNN algorithm has stronger robustness,significantly reduces power fluctuations between regions,and maintains system frequency stability.Finally,a multi-agent transfer soft actor-critic algorithm with long-short term memory(MATSAC-LSTM)is proposed for AGC.Firstly,the soft actor-critic algorithm is used with maximum entropy strategy to avoid the agents getting stuck in local optima.Then,a long-short term memory network is introduced to extract temporal features from the collected environmental state variables,such as regional control errors.Finally,transfer learning is applied to significantly reduce the training time of the agents.Simulations on improved IEEE standard two-area load frequency control system model and a simplified system in a province of China demonstrate that the proposed MATSAC-LSTM method has advantages such as fast training speed and high control performance.