Research On Energy Optimization Management Scheme Based On Deep Reinforcement Learning In Microgrid

Electricity is vital to transportation,medical treatment,communication and other aspects of modern life,and with the growing popularity of technologies such as electric vehicles and electric heating,the demand for electricity is increasing.In response to the depletion of traditional power generation sources and growing environmental pollution,countries around the world have accelerated the deployment of renewable energy generation facilities.Currently,microgrids are installed on the customer side,integrating traditional distributed generation,energy storage and load components to facilitate the local consumption of renewable energy generation.Further,energy optimization management of microgrids can cope with the randomness of renewable energy generation,load and other factors,and achieve goals such as cost minimization while ensuring safe and stable system operation.Therefore,this thesis conducts an in-depth research in the scheme of microgrid optimization energy management,and designs and proposes deep reinforcement learning-based energy optimization management models and algorithms for single and multi-microgrids.The main contributions of the thesis are as follows:1)A comprehensive review,analysis,classification and comparison of existing microgrid energy optimization management solutions.The microgrid energy optimization management schemes are classified into single and multi-microgrid optimization management schemes according to the type of microgrid.According to the different types of optimization techniques used,the system model-based and data-driven schemes in the single microgrid energy optimization management scheme are discussed and the advantages and disadvantages of the two types of schemes are compared;according to the different control strategies,the centralized,distributed and hybrid schemes in the multi-microgrid energy optimization management scheme are discussed and the advantages and disadvantages of the three types of schemes are compared.2)For single microgrid,an energy optimization management scheme(GRVA3C)based on gate recurrent unit and asynchronous advantage actor-critic algorithm with reward vectorization decomposition is proposed.Most of the existing schemes ignore the temporal characteristics of the input sequence information,in this thesis,the stochasticity and volatility of the input sequence data are learned using the gate recurrent unit.In addition,in order to solve the problem that the multidimensional action space makes it difficult for the network to converge to the best strategy,this paper decomposes and vectorizes the reward function so that it corresponds to each dimension of the action.In turn,the neural network can be updated more precisely for each dimensional action to avoid the exploration of unnecessary action space,so that the network can converge to the optimal strategy more easily and thus produce better optimization results.The experimental results show that the proposed scheme shows better results in both offline training and online testing phases compared with other schemes.3)For the problem of joint optimization of multi-microgrids with power flow between microgrids,a bilayer energy optimization management scheme(BMAPPO)based on multi-agent proximal policy optimization algorithm and optimal power flow is proposed.The scheme is divided into two layers.First,the lower layer uses the multi-agent proximal policy optimization algorithm to determine the power output of various power controllable devices in each microgrid;then,the upper layer uses the optimal power model after second order cone relaxation to solve the optimal power flow between multiple microgrids based on the optimization results of the lower layer to realize the power dispatch between microgrids;finally,the total cost of the upper and lower layers is calculated to update the network parameters.The experimental results show that the proposed scheme accomplishes the optimal energy management of multi-microgrids at the lowest cost while ensuring the online execution speed compared with other schemes.