Learning Optimal Control For Multiple Energy Storage Devices In Smart Grid

Smart Grid is a modern grid involving advanced sensing, control, communication, decision-making techniques, which is evoluted from traditional power grid. Compared with traditional power grid, smart grid is more secure, more reliable and more economical. One of the distinctions in architecture and composition between smart grid and traditional grid is that energy storage devices are widely used in smart grid. The optimal control of these devices contributes a lot to the development of smart grid, and becomes an intersting topic in this field. This thesis studies the optimal control of multiple energy storage devices in different scenarios of smart grid, the main contents are as below:(1) We study the energy scheduling problem of smart distribution network with variable upper bound of the load rate, where multiple energy storage devices are contained. By considering the stochastic proporities of the loads and upper bound of the load rate, we model the energy scheduling problem of distributed network as a markov decision process. Then the optimal control policy for energy storage devices is derived. Simulation results show that the obtained optimal policy loweres the excess numbers of load rate for the transmission lines and strengthenes the security of the system.(2) We study the energy management problem of the microgrid with mulitiple regions of the users and multiple energy storage devices under real-time pricing mechanism. Considering the dynamic characteristics of the photovoltaic power generation system and customer’s requirements in different areas, the optimization objective function fot the system is established. Then the problem is modeled as a semi-markov decision process and solved with both theoretical and learning algorithms. Simulation results show that the obtained optimal policy or sub-optimal dispatchs energy in the micro-gird reasonably and reduces the operating cost in long term.(3) In the power grid, time properties of the diverse loads have significant differences. Therefore, we further study the optimal control of active power flow in micro-grid which has diverve loads and multiple energy storage devices, under real-time pricing mechanism. Firstly, physical model of the control problem is built considering the time properties of both loads and output of photovoltaic power generation system. The objective is to operate the micro-gird economically and securely. Then, we model the optimization problem as a finite-horizon Markov decision process and solve it with SARSA-learning algorithm. Finally, simulation results show that the power gird with our proposed algorithm has lower operating fee and is more sucure during peak-load periods.