Research On Microgrid Energy Management Strategy Based On Deep Reinforcement Learning

Microgrid energy management can improve the reliability and economics of microgrids by ensuring the quality of power supply and lowering operational expenses while increasing the utilization rate of renewable energy.Now,because deep reinforcement learning can make real-time decisions in response to changes in the environmental state and can effectively deal with uncertainty during the decision-making process,the use of deep reinforcement learning to implement a microgrid’s energy management strategy has become a research hotspot in recent years.However,the application of deep reinforcement learning methods to microgrid energy management is still in its infancy,and there are still numerous issues that contribute to ineffective energy management or implementation difficulties.This thesis adapts the deep reinforcement learning method to the peculiarities of microgrid energy management in order to enhance the optimization effect of energy management strategies.The following are the major research findings from this thesis:To begin,this thesis develops an energy management model for a microgrid based on the fundamental theoretical framework of deep reinforcement learning and implements energy management using a deep reinforcement learning algorithm.The examination of instances demonstrates that the deep reinforcement learning method is advantageous for uncertain environments due to its rapid response time,and that energy management strategies based on this method can minimize operating costs.However,the deep reinforcement learning method requires the algorithm to learn the energy management strategy during training based on the reward function’s feedback,and the uncertainty associated with renewable energy will interfere with the reward function’s feedback during training,resulting in the reward sparse problem,which reduces the algorithm’s convergence effect and the strategy’s energy management effect.Second,in order to address the issue of reward sparsity induced by the uncertainty inherent in renewable energy,this thesis presents a reward reshape deep reinforcement learning approach for redesigning the reward function and output action.In reward function terminology,step reward and final reward are used to provide feedback on rewards at various training phases,which increases the algorithm’s convergence capacity.In output action words,an expert intervention mechanism is meant to select the appropriate output action based on the restrictions,hence reducing the algorithm’s exploration area.The instances analysis demonstrates that the proposed technique can effectively increase the energy management impact of the learnt strategy,and that it outperforms the model predictive control method in terms of energy management effect and online execution time.Finally,in order to simplify the design of reward functions for large-scale microgrids using the aforementioned methodologies,this thesis offers a two-stage energy management framework based on imitation learning.The framework consists of two stages: agent decision-making and power flow computation.At the agent choice stage,the imitation learning method is utilized to immediately imitate the optimal strategy for scheduling the controlled distributed power source optimization without the need to build a reward function.The power flow calculation stage calculates and adjusts the power flow based on the scheduling findings from the previous stage in order to complete the breakdown of the optimization variables.The instances analysis demonstrates that the two-stage energy management framework based on imitation learning can more accurately mimic the ideal scheduling method,resulting in improved energy management effect and online execution speed.