Research On Optimal Scheduling Of Regional Integrated Energy System Based On Deep Reinforcement Learnin

Regional integrated energy system is one of the important measures to comprehensively promote sustainable development,which aims to achieve efficient application of multiple energy sources by using advanced communication and control technologies.However,in the regional integrated energy system,the instability of clean energy and the coupling of multiple energy sources make the physical model very complex,while the access of various intelligent communication devices will generate a large amount of high-dimensional data,making data processing more difficult,and the commonly used traditional optimization algorithms will be difficult to apply due to the increase in computational cost resulting in too slow solution rates.Deep reinforcement learning is a more popular research area in machine learning,which has an efficient ability to solve problems with data and complex models.Therefore,in order to improve clean energy utilization and reduce carbon emissions,and to achieve more generalizability,robustness and efficiency of regional integrated energy system optimal scheduling,an optimal scheduling method of regional integrated energy system based on advantage learning soft actor-critic(SAC)algorithm and transfer learning is proposed.The study was conducted on:1)This paper investigates the theory of reinforcement learning and deep reinforcement learning,outlines the basic elements of deep reinforcement learning model building,and the learning principles of different types of deep reinforcement learning algorithms.2)The operation mechanism and related characteristics of the internal units of the regional integrated energy system are analyzed,and the corresponding mathematical model is constructed,as well as the objective function containing the carbon emission and profit of the regional integrated energy system.3)To establish a model of the state and action space and reward function of the regional integrated energy system based on deep reinforcement learning of policy gradient,and to demonstrate by arithmetic examples that among several deep reinforcement learning algorithms based on policy gradient,the maximum entropy mechanism of Soft Actor-critic makes the optimal scheduling of the regional integrated energy system more robust.4)The SAC algorithm has a complex neural network structure and iterative mechanism,so it will take more time in training.The advantage learning theory is adopted to build a regional integrated energy system model based on the advantage learning SAC algorithm to speed up the convergence speed when the algorithm is trained.The effectiveness of the algorithm to incorporate advantage learning is demonstrated by comparing the performance of several policy gradient-based deep reinforcement learning algorithms,traditional particle swarm algorithms,and mixed integer programming in specific arithmetic cases.5)To improve the learning efficiency of the intelligences and the generalization ability to cope with new scenarios,transfer learning is introduced in this paper.The algorithms show that the optimal scheduling strategy based on advantage learning soft actor-critic algorithm and transfer learning has better robustness,generalization ability and efficient learning efficiency,and can realize flexible and efficient scheduling of regional integrated energy systems.