Research On Optimal Scheduling Strategy Of Microgrid Based On Deep Reinforcement Learning

With the national carbon neutral,carbon peak requirements,the new energy industry has been rapid development.Microgrid has been widely concerned because it can effectively realize the interaction between distributed new energy power generation units and large power grids,and realize energy complementarity and economic operation.Through the Energy Management System(EMS),the microgrid optimally schedules distributed units to achieve energy complementarity and economic operation within the network,which is of great significance for improving energy utilization and building a low-carbon and efficient modern energy system.However,the optimal scheduling of electric heating energy microgrid involves multi-objective,nonlinear and non-convex problems,and the traditional methods have difficulties in calculating real-time performance and convergence in the optimal scheduling solution.Therefore,this paper proposes a data-driven deep reinforcement learning optimization scheduling strategy for microgrid.The main research contents of this paper are as follows:(1)Firstly,the basic working principle of all kinds of distributed power generation units of micro-grid is deeply studied,and a micro-grid system containing electric heating energy is constructed.The power output model of photovoltaic cell,charging and discharging model of energy storage battery,cogeneration model and electric boiler model are established.Then,the reinforcement learning theory is introduced,including Markov decision process,Q learning,Deep Q Network(DQN)and Deep Deterministic Policy Gradient(DDPG).It lays a foundation for the application of deep reinforcement learning in the optimization of microgrid scheduling.(2)A micro-grid scheduling strategy based on DQN algorithm is proposed to solve the problem of the shortage of real-time computation and iterative convergence of the scheduling strategy obtained by the traditional algorithm.Firstly,according to the output characteristics of each distributed unit,a micro-grid scheduling strategy considering the user side load demand is proposed.The optimal scheduling strategy is solved based on DQN algorithm to achieve multi-objective optimization.Under the action of scheduling strategy,the charge and discharge action of energy storage is controlled to fully absorb the renewable energy output,and the real-time electricity price is considered for trading with the large power grid,so as to achieve the optimization goal of the lowest operation cost and the least pollutant emission of the micro-grid.The scheduling results were obtained through the simulation examples,and the output of the scheduling unit was analyzed to verify the feasibility and effectiveness of the scheduling strategy.The trained DQN algorithm model can generate scheduling strategy in real time,which avoids the problem of long operation time and difficult online calculation of traditional algorithm.(3)In order to reduce the stability and accuracy of discrete action microgrid scheduling strategy based on DQN algorithm,a micro-grid scheduling strategy based on DDPG algorithm is proposed.Firstly,the optimization scheduling problem of microgrid is transformed into the optimization problem of continuous action DDPG algorithm,and the optimization algorithm is obtained.The continuous action is output by the optimization algorithm,because there is no need to discretize the action,the output action is more accurate.For the microgrid in this paper,the optimal scheduling is carried out based on two optimization algorithms under the same conditions,and the scheduling strategy is obtained.Through the simulation and comparative analysis of the scheduling results,it is verified that the micro-grid scheduling strategy based on DDPG algorithm can effectively improve the stability and accuracy of the scheduling strategy and better achieve the optimization objectives of the micro-grid.