Research On Optimization Method Of Integrated Energy System Based On Deep Reinforcement Learning

In recent years,the global energy crisis and environmental problems have become increasingly prominent,and China has actively put forward the national development goal of "carbon peaking and carbon neutrality",and the national energy revolution and transformation have been continuously promoted.In this context,new energy sources(such as wind power,solar power,etc.)are developing rapidly and the installed capacity is increasing day by day.At the same time,renewable energy is characterized by randomness,volatility and uncertainty,which reduces the margin of safe operation of the power grid and makes the system regulation more difficult.Integrated energy system(IES)covers a variety of different forms of energy,such as wind,light,heat,etc.,on the basis of coordinated operation and interaction to achieve the integration of energy systems,IES multi-energy complementary mutual aid to reduce the adverse impact of uncertain power sources on the system.However,the following problems still exist:① The interaction of multiple types of loads such as heat,cold and electricity,the traditional single load prediction accuracy is low,and it is difficult to meet the existing requirements of optimal dispatch for load prediction accuracy.② The working characteristics of power system,thermal system and natural gas system are different,and the dynamic effects vary too much.The existing modeling does not fully consider the influence of dynamic effects,which makes it difficult to meet the existing requirements for the actual operation of the project.③The comprehensive energy dispatch model is large in scale,with strong constraint nonlinearity and difficult to solve,which leads to higher difficulty in system dispatch and makes it difficult to realize fine regulation.To address the above problems,this paper mainly carries out the following work:(1)A comprehensive energy load forecasting method that takes into account the deep coupling characteristics of multiple loads is proposed.Firstly,based on the traditional energy hub(EH)model,the coupling mechanism of multiple loads such as cooling,heating and electricity is analyzed,and the calculation methods of load synchronization index,load complementary index and meteorological gray correlation coefficient are proposed,so that the coupling characteristics of cooling,heating and electricity loads can be quantified;then,based on the load synchronization index,load complementary index and gray correlation coefficient,a high power matrix is used to Finally,a prediction method(PSO-TCN-LSTM)based on particle swarm optimization,temporal convolutional network and long short term memory neural network is proposed for load prediction,and the model is validated by the actual IES multiple load data,and the results show that the algorithm can effectively improve the accuracy of load prediction.(2)An IES model accounting for dynamic effects is established.Based on the consideration of dynamic effects,an integrated energy management system mathematical model is constructed.For the power system,an integrated power system model including power supply equipment,energy storage equipment,heating equipment and cooling equipment is established;for the heat system,partial differential equations are used to analyze the heat network transmission delay,explore the energy storage characteristics of heat pipes,and establish a heat system model that takes into account the dynamic characteristics of the heat network;for the natural gas system,partial differential equations are used to analyze the gas network pipe storage effect For the natural gas system,the dynamic effect of the gas network storage capacity is investigated by using partial differential equations,and a natural gas system model with the dynamic characteristics of the gas network is developed.The combined model provides the basis for IES optimal scheduling based on deep reinforcement learning.(3)The IES optimal dispatching method based on deep reinforcement learning is proposed.Firstly,with the objectives of minimizing operation cost and maximizing power supply reliability,an optimal dispatching model is established to satisfy the constraints of power system,thermal system and natural gas system.Then,the soft actor-critic(SAC)algorithm based on deep reinforcement learning(DRL)is used to solve the model,and the stochastic solution strategy is applied to reduce the computational difficulty of the model and improve the convergence speed of the algorithm.Finally,the feasibility and validity of the proposed model and algorithm are verified by building an IES model with dynamic effects for simulation.The results show that,compared with the stochastic optimization method based on Particle Swarm Optimization(PSO),Double Deep Q-learning Network(DDQN)algorithm and Deep Deterministic Policy Gradient(DDPG)algorithm,the integrated energy scheduling method in this paper is economically superior.