A Fuzzy Q-Learning Algorithm For Energy Storage Optimization In Islanding Microgrid

The emergence of clean and green renewable energy sources such as wind and solar energy can effectively alleviate people’s dependence on fossil energy,and can also effectively solve the power supply needs of some remote areas rich in renewable energy.However,due to the randomness and volatility of solar and wind energy,it is necessary to optimize the control of the microgrid containing renewable energy to ensure its stable operation.As an important part of the islanding microgrid,the energy storage system has the function of peak load shaving,frequency and voltage regulation,which can ensure the stable operation of the islanding microgrid system.The complexity of micro-grid system makes it difficult to establish an accurate mathematical model.The problem is solved well by strengthening the model-free feature of learning Q-learning algorithm.In order to solve the dimension disasters of Q-learning when it encounters multistate space,fuzzy control is introduced to approximate Q-function to accommodate learning problems such as continuous state space and motion space.Based on the light/diesel/storage islanding microgrid,this thesis studies the optimal control strategy of the islanding microgrid.The main research content can be divided into the following parts:Firstly,the research background and current status of this topic are introduced,and each unit in the light/wood/storage microgrid is modeled.Their mathematical model,generation principle and output characteristics are described respectively.Aiming at the light/diesel/storage islanding microgrid,an optimization model of islanding microgrid based on Q-learning algorithm is studied.The model adjusts the power output of energy storage system and diesel generator through Q-learning algorithm,improves the utilization rate of renewable energy to a certain extent,reduces the use of fossil energy,and ensures the stable operation of islanding microgrid with lower generation cost.The model is simulated in cloudy and sunny weather respectively,and the results show the validity of the method.Then,in order to cope with the increasingly complex grid structure,electrolytic cells and fuel cells are introduced under the structure of light/diesel/storage islanding microgrid.The electrolyzer,fuel cell,energy storage system and diesel generator are regarded as an independent agent to construct a multi-agent system.Considering the dimension disasters of Q-learning algorithm in the face of more state-action,in order to cope with continuous state and action space,the new algorithm introduces fuzzy control on the basis of Q-learning algorithm and studies a fuzzy Q-learning algorithm.The new algorithm maps continuous state space to continuous action space through fuzzy inference system makes Q-learning possible to be used in continuous state space problems.Using Q-learning to update and iterate to a complete rule base,the agent infers according to the learned rule base,and finally gets an optimal action.The simulation results show that the algorithm is effective in multi-agent system to ensure power supply and improve the reliability of microgrid.Finally,considering that a single energy storage system may not be able to cope with the complex islanding microgrid.Compared with the advantages and disadvantages of existing energy storage technology,the hybrid energy storage system of battery supercapacitor is introduced.For hybrid energy storage system,fuzzy Qlearning algorithm is used to determine the power distribution of hybrid energy storage system.The simulation results show that compared with the single energy storage system,the hybrid energy storage system can better deal with the uncertainty of renewable energy,effectively reduce the fluctuation of battery state of charge,stabilize the battery state of charge to the normal working area,improve the service life of battery and the utilization rate of renewable energy,and reduce the use of diesel generator.