Multi-objective Strategy For Smart Generation Control And Equilibrium Reinforcement Learning Theory

Automatic generation control(AGC) is one of the essential functions required in modern power systems as to maintain the frequency quality and sched uled power interchanges. However, the conventional AGC are not more efficient for future power systems, consideration the ever increasing penetration for sustainable energy sources, changing grid structure, development of power electronic devices, and the undergoing electricity market reinform. This is because the conventional AGC cannot deal with the coordination problem between the primary frequency control and secondary frequency control in the same control area, as well as the coordination problem between the secondary controls for differenct control areas. In real power system, these coordination problems will lead the anti-regulation phenomenon to take place in AGC units, and so it is un-economic from the energy saving prospective. Therefore, under the future smart grid paradigm, developing an efficient multi-area smart generation control(SGC) system with grid-to-grid coordination is becoming an apprealing approach. Compared to conventional AGC, SGC can fulfill multiobjective optimization and grid-to-grid coordination with and without electricity market concerned. And hence, it is expected that the SGC can exhibite competitive performance, more flexibility and more adaptivity than conventional AGC.In this paper, a generation participation matrix based SGC model has brifly been described. Two control module, i.e. optimal control moduel and participation factors optimization module, are included in the model. In order to implement the following proposed SGC strategy in realistic power systems, the former module should adopt model- free control methodology, and fast intelligent optimization algorithms should be adopted in priority in the latter module.In competitive electricity market, the generation companies are drived to maximize their profits and customers are inclined to minimize their cost. Those interactive behaviours will naturally lead to a dynamic Nash equilibrium point for all the involved players. Hence, this paper proposes a novel Nash-Q(λ) algorithm to obtain the optimal equilibrium strategy for SGC controller. The long-term profits for Generation Companies as well as the cost of costomers are described using a Q matrix. The Nash equilibrium is set to determine the bilateral contract between generation companies and customers, and eligibility traces λ is used to reassign the immediate reward over the past sampled state and action pairs.Due to the problem that the strategy obtained using Nash equilibrium is easy to limited to local-optimality, this paper proposes a novel correlated equilibrium(C E) based CEQ(λ) learning to pursue the grid-to-grid coordination. The proposed algorithm establishes its reinforcement model via state sampling for the connected areas. Then, CEQ(λ) can be implemented to obtain the optimal equilibrium policy based on the established model and estimated grid state. The proposed CEQ(λ) has been sucessfully established on multiagent JADE platform. In addition, a CE based MORL(λ) is also proposed to solve the generation command dispatch(GC D) problem. The emission, economics and dynamic transient performance in frequency restoration process are considered as three optimization objectives. The weights of the three objectives can be tuned automatically according to the sta te of the power systems.At last, a hybrid intelligent algorithm combining of variable universe fuzzy logic(VFLC) and CEQ(λ) learning has been proposed as to guarantee the system frequency stability. The use of VFLC is to make sure that the novel C EQ(λ) learning can be implemented in real power system since VFLC can decrease the state-and-action pairs a lot and hence the pre-learning process has been shortened and the control precision has been improved. This method has been successfully implemented on a 49-bus power system model established on RTDS platform in Power System Simulation Department, Scientific Research Institute, China Southern Power Grid.This work was jointly supported by the the National Key Basic Research Program of China(973 Program, 2013CB228205), the National High-tech Research and Development Projects(863, 2012AA050209), the National Natural Science Foundation of C hina(51177051, 51477055), and the Hong Kong Polytechnic University projects(A-PL97 and A-PD0K).