| The energy problem is becoming more and more serious.The utilization of renewable energy plays an important role in sustainable development.Microgrid is a kind of green and efficient power system,which can effectively use renewable energy.With the in-depth study of microgrid,many scholars have put forward new theories about optimal control of microgrid.However,due to the complexity of energy storage management in microgrid,it is difficult to carry out accurate mathematical modeling,and data-driven method is needed.Reinforcement learning is a data-driven artificial intelligence method,which can be applied to the energy management of microgrid energy storage.Aiming at the energy storage system of microgrid with different optimization objectives,this paper discretizes the continuous scheduling process of microgrid,and uses reinforcement learning algorithm to solve the optimal scheduling scheme.The main research work of this thesis is as follows:This thesis first introduces the research background and current situation of the subject,describes the basic structure and operation mode of microgrid with energy storage,establishes the mathematical model of each micro source,and finally gives the operation constraints of microgrid and the content of reinforcement learning algorithm in data-driven method,which provides a theoretical basis for solving various energy storage optimization control problems of microgrid.Secondly,in the operation process of grid connected microgrid,energy storage can maintain system stability by charging and discharging.Due to the randomness of wind and solar power generation,the charging and discharging operation of energy storage will directly affect the power balance of the system.In order to solve the problem of energy storage economic scheduling in grid connected microgrid,DoubleQ learning algorithm is used to design the energy storage control strategy.Double-Q learning algorithm is an optimization method based on value function iteration,which has no prior requirements for the model and can be used to solve the energy storage control problem of microgrid in pursuit of optimal index.Under the reinforcement learning paradigm,the agent can learn the optimal strategy by setting the power purchase cost of microgrid as the reward for each step of scheduling,so as to achieve the goal of minimizing the operation cost and maximizing the resource utilization of microgrid.The experimental results show that the energy storage control strategy can effectively maintain the stability of microgrid system.Then,because the energy storage system of the microgrid carries out arbitrage trade with the power market of the main grid,it can obtain profits and reduce the operation cost of the microgrid.Aiming at the arbitrage problem of energy storage system,the Double-Q learning algorithm is used to design the arbitrage strategy.Compared with the arbitrage strategy based on Q-learning algorithm,this strategy can significantly avoid the impact of overestimation.Then the carbon market is added to the arbitrage market to increase the arbitrage source and the arbitrage profit.The experimental results show that the scheme significantly improves the arbitrage profit at the algorithm level and scenario level,which proves the effectiveness of the designed strategy.Finally,in the island microgrid,the power balance of the system becomes more difficult due to the loss of power supply from the main grid.In order to solve the problem of power balance in isolated microgrid,a hybrid energy storage system with battery and hydrogen storage system is used to ensure the stability of microgrid system.Due to the high dimension of the state space of hybrid energy storage system in islanded microgrid,the common reinforcement learning algorithm is difficult to deal with.Aiming at the coordinated control problem of hybrid energy storage in islanded microgrid,this paper uses Double deep Q-learning algorithm to design the coordinated control strategy of hybrid energy storage.This method effectively combines the advantages of deep learning and reinforcement learning,and improves the ability of reinforcement learning to deal with problems.The experimental results show that the strategy can achieve the goal of coordinated control of hybrid energy storage system of island microgrid,and effectively improve the utilization rate of renewable energy. |
