Research On Deep Reinforcement Learning And Its Application To The Optimal Operation Of Integrated Energy System

Energy is an important support for the development of modern society.However,due to the rapid economic growth and the increasing demand for energy,energy supply has become a global challenge.The integrated energy system is the integration of different types of energy,such as electrical and thermal energy,in the region,which achieves coordinated planning and mutual supplementation between multiple energy systems,effectively improving the efficiency of energy use and also promoting the consumption of renewable energy.On the other hand,artificial intelligence technology has developed rapidly in recent years,and deep reinforcement learning-based methods have the advantage over traditional methods in optimising the operation of integrated energy systems that do not rely on modelling accuracy and do not require prediction of wind and light output,and in terms of solution speed deep reinforcement learning algorithms can reach the level of real-time control.Therefore,it is of great strategic significance and application value to study the application of deep reinforcement learning in the optimal operation of integrated energy system.Based on this,this paper uses deep reinforcement learning algorithm to study the equipment side and demand side of integrated energy system.Firstly,an integrated energy system structure incorporating electricity and heat is established on the equipment side,and the operational mathematical model of each equipment in the system is formulated.Based on the consideration of equipment operation and power balance constraints,an integrated energy system optimisation operation model that takes into account economic and carbon emission factors has been developed.Secondly,the basic concept of deep reinforcement learning and the mathematical framework of reinforcement learning Markov decision process(MDP)are expounded.The optimal operation model of integrated energy system is transformed into MDP,and the necessary elements such as state space,action space and reward function are designed.In addition,in order to ensure the stable convergence of the reinforcement learning algorithm,the original proximal strategy optimization algorithm(PPO)is improved by means of learning rate exponential decline,state normalization and strategy entropy.The improved PPO algorithm is used to solve the optimal operation problem of the integrated energy system.The algorithm can learn historical data and can solve the optimal operation strategy of the equipment side of the integrated energy system in real time.By comparing with the particle swarm optimization algorithm(PSO)and the original PPO algorithm and other reinforcement learning algorithms,the effectiveness and advancement of the improved PPO algorithm are verified.At the same time,on the demand side,the traditional power demand response is extended to natural gas,and a demand response model based on price elasticity matrix is established.The improved PPO algorithm is used to solve the time-sharing energy price.By learning historical data,find the optimal time-sharing energy price,reasonably guide users to change their energy consumption habits,and achieve the effect of peak shaving and valley filling.The proposed method is compared with PSO algorithm and genetic algorithm(GA).The results verify the effectiveness of the proposed method in peak load shifting.The electricity and heat demand response models are applied to the integrated energy system.The experimental results show that the demand response can effectively improve the economy and environmental protection of the integrated energy system.