Online Optimal Dispatch Of Microgrid Energy Based On Deep Reinforcement Learning

Energy management is of great significance for the safety,stability and economy of microgrids.With the large-scale access of sustainable energy with great randomness,the use of massive schedulable units,the increasing diversity of loads,and the development of the electricity trade market,the energy management issue of the microgrid has become more and more complex,which not only affects the stability of the microgrid but also brings great challenges to economic dispatch.In this context,the limitations of conventional model-based dispatch algorithms are gradually appearing because of the increasing difficulty of precise modeling.Data-driven algorithms such as deep reinforcement learning(DRL)algorithms are independent on the quality of the microgrid model,can improve decision-making with historical data as well as process real-time data at a high speed,which making them promising for solving online energy optimal dispatch problems in current optimal dispatch study.However,conventional DRL methods generally having problems such as weak adaptability,low learning efficiency,and poor robustness when facing microgrids with diverse structures and different modes,results in suboptimal dispatch or even failures in completing decisions,which seriously reducing the application value of DRL methods.To solve the above problems,this article chooses an AC microgrid as the controlled object,focuses on the DRL energy online optimal dispatch,and improves the algorithms according to the actual application needs.The main research content and conclusions are as follows:(1)Aiming at the problem that conventional DRL algorithms are difficult to train effectively under strong constraints,a Strong Constraint Resistance Double Deep Q Network(SCR-DDQN)algorithm is proposed,which breaks the step number limitation from the strong constraints on decision-making,and makes the DRL algorithm not only suitable for optimal dispatch in the mode of selling the surplus electricity to the grid,but also can be used for microgrids prohibited to sell the surplus electricity to the grid.The SCR-DDQN algorithm has two stages named pathfinding and optimization.In the pathfinding stage,the improved exploration strategy with branch ε and the reward function based on prior knowledge are used.Asynchronously adjusting the relationship between exploration and exploitation at different decision-making steps,this method effectively weakens the effect of strong constraints.In the optimization stage,the cost of sequence decision-making is optimized via action elimination with the training result on the pathfinding stage.The simulation results prove the stability and feasibility of the SCR-DDQN algorithm.(2)Aiming at the problem that a large action space affects the convergence of the DRL algorithm,a Policy Action Double Deep Q Network(PA-DDQN)algorithm is proposed,which not only improves the economy of the dispatch results,but also reduces the switch time number of single micro-source and is beneficial to prolonging the life of the equipment.PA-DDQN constructs action space with generation strategies,and then parse the strategies into dispatch instructions with a rule-based distribution algorithm.The simulation results prove the feasibility and superiority of the PA-DDQN algorithm.(3)Design and implement the software platform of microgrid energy management system with Python.At first,analyze the functional requirements of the client and server.Then design the system architecture and threads of both the front-end and back-end.Next,use Qt to design the front-end interface,build the back-end functions on the Linux server and use the MySQL database.Finally,realize the functions such as the collection and monitoring of real-time data,collation and analysis of historical data,prediction and scheduling based on intelligent algorithms,simulation and processing of faults.Besides,this platform is equipped with the aforementioned SCR-DDQN and PA-DDQN algorithms,effectively improving the economy of the microgrid.