Intra-Day On-Line Optimization Strategy For Microgrid Operation Based On Approximate Dynamic Programming

The utilization of renewable energy includes centralized and distributed methods.As an important utilization form of distributed renewable energy,microgrid has obtained rapid development worldwide.However,the uncertainty and volatility of renewable energy sources(RESs),such as photovoltaics and wind power generation,bring great challenges to the operation of power systems.Online optimization as an important part of grid operation plays an important role in ensuring the economic and safe operation of the system and the friendly access of RESs.Currently,the intra-day optimization of microgrid mainly based on traditional online optimization algorithms such as model predictive control(MPC).Existing online optimization algorithms have many shortcomings in aspects of the optimality of decision-making and coping with the uncertainty of RESs.Approximate dynamic programming(ADP),as a stochastic optimization algorithm,has obtained extensive attention in recent years.The ADP algorithm bases on the Bellman optimality principle.ADP overcomes the “curse of dimensionality” problem faced by the dynamic programming algorithm and has broad application prospects in the operation of power systems.Based on the current research status of microgrid optimization,this paper proposes several online optimization algorithms based on the different model of microgrid optimization and different forecasting information,which include the piecewise linear function approximation based ADP,look-up table approximation based ADP,deep reinforcement learning algorithm and parametric cost function approximation algorithms.The main research content includes:(1)Considering the traditional MPC method cannot guarantee the global optimality of the solution,a piecewise linear function(PLF)approximation based online optimization algorithm is proposed.A microgrid optimization model considering linearized network power flow constraints is established.By analyzing the physical meaning represented by the slope of the PLF,a slope update method for PLF is proposed based on the marginal value of energy stored in the battery,which significantly improves the convergence speed of the APD algorithm.The simulation results show that compared with the existing intra-day optimization method,the online optimization performance of the proposed ADP algorithm is much better.The proposed algorithm only needs the current system information and the approximated functions to make a global optimal online decision,which avoids the influence of intra-day prediction error on online decision-making,and provides a new method for intra-day operation of microgrid.(2)For the modeling error of the above linearized power flow constraints and the inaccuracy of the battery model,a nonlinear optimization model considering the AC power flow constraint and the detailed battery operation model is proposed,and a look-up table approximation based ADP algorithm is proposed.The online optimization strategy decomposes the original MINLP(Mixed-integer nonlinear programming)problem into multiple NLP(Nonlinear Programming)sub-problems through time decoupling,which reduces the difficulty of solving the established optimization problem.The effectiveness of the proposed algorithm when solving nonlinear optimization problems is demonstrated by the simulations.The simulation results show that the proposed online optimization algorithm performs better than traditional algorithms such as MPC and PSO.(3)Considering the learning ability of table functions is limited,and the intra-day forecasting information is not utilized in the optimization process in the algorithms proposed in(1)and(2),in order to further improve the intelligence of online optimization of microgrid,an online optimization algorithm based on deep reinforcement learning(DRL-MG)is proposed.By carefully designing the state variables and decision variables,the plenty of constraints and huge decision space of the optimization problem which bring great difficulty for deep neural networks can be solved.In addition,the proposed algorithm can make full use of the intra-day updated prediction information and the day-ahead forecasting.Simulation results show that the proposed algorithm has better online decision-making performance than the look-up table approximation based ADP algorithm.(4)With the increase of the controllable micro-generators,the decision space of the problem will become so large that the DRL algorithm proposed in(3)is hard to solve.To overcome this shortcoming,an online optimization strategy based on parametric cost function approximation(PCFA)algorithm is proposed.Similar to the DRL algorithm,the algorithm proposed in this chapter can also make full use of the day-ahead and intra-day prediction information.The modeling method of the PCFA algorithm is similar to the MPC algorithm,but the proposed algorithm introduces the parameter in constraints related to stochastic variables,so that the current decision can consider the prediction error of the near future wind power.Because the established microgrid optimization model is a non-convex optimization problem,in order to find the optimal parameter,a parameter optimization algorithm based on stochastic gradient descent method is proposed.The simulation results show that by introducing the parameter in the constraints,the proposed method can obtain better online decisions compared with the MPC.The algorithm provides a new method for the optimal online operation of microgrids.