Multi-agent System Stochastic Consensus Game Based Smart Generation Control

To adapt to the random environment of the power system when large scale wind power and photovoltaic as well as electric vehicles are connected to the grid, smart grid requires power generation dispatching control system from automation to intelligence. Thus, a new generation of smart generation control(SGC) system in the face of all kinds of new energy access is proposed.The biggest difference of SGC from modern(automatic generation control) AGC lies in that SGC is a multi-agent coordination control system which is adapted to the strong random environment and has the ability of multi-objective dynamic optimization. Four novel methods for SGC including decentralized correlated equilibrium Q(λ)-learning(DCEQ(λ)), decentralized win or learn fast policy hill-climbing with eligibility trace(DWo LF-PHC(λ)), wolf pack hunting strategy based virtual tribes control(WPH-VTC), and wolf pack hunting(WPH) are proposed in this paper to explain the interaction and optimal equilibrium state of multi agents in the SGC system. Furthermore, SGC stochastic optimal control problem was modeled as a stochastic equilibrium game problem based on the proposed four methods respectively to be solved.Under the control performance standards(CPS), SGC will undergo a non-Markov random process, of which the optimal solution can be resolved online by the reinforcement learning. Therefore, a MA DCEQ(λ) algorithm has been proposed for its implementation, which can achieve AGC coordination in a highly uncertain environment resulted from the increasing penetration of renewable energy. The single-agent Q-learning, Q(λ)-learning, R(λ)-learning, and PI control are implemented and embedded in SGC-SP for the control performance analysis. Two case studies on both a two-area power system and the China Southern Power Grid(CSG) model have been done, which verify its effectiveness and scalability. However, multi-equilibrium may emerge when the agent number increases, which inevitably consumes longer time due to the extensive online calculations of all the equilibriums and may even lead to an undesired system instability for SARL and correlated equilibrium.Thus, a novel MA DWoLF-PHC(λ) is developed, which can effectively identify the optimal average policies via a variable learning rate under various operation conditions. Based on CPS, the proposed approach is implemented in a flexible MA stochastic dynamic game-based smart generation control simulation platform. Based on the mixed strategy and average policy, it is highly adaptive in stochastic non-Markov environments and large time-delay systems, which can fulfill AGC coordination in interconnected complex power systems in the presence of increasing penetration of decentralized renewable energy. Two case studies on both a two-area load-frequency control power system and the CSG model have been done. Simulation results verify that MA-SGC scheme based on the proposed approach can obtain optimal average policies thus improve the closed-loop system performances, and can achieve a fast convergence rate with significant robustness compared with other methods. However, the total power references of provincial dispatch centre were achieved through a fixed proportion of the adjustable capacity rather than a dynamic optimization, and multi-equilibrium may emerge as the agent number increases, which may even lead to a severe system stability collapse.Thus, this paper proposes a novel frequency autonomy(FA) to satisfy the requirement of power generation optimization of smart grid and decentralized energy management system. A decentralized virtual tribes control(VTC) is developed which can effectively coordinate the AGC of provincial power grid and the AGC of distribution networks and microgrids. Then a WPH-VTC is designed through combining the multi-agent system stochastic game(MAS-SG) and multi-agent system collaborative consensus(MAS-CC), which is called the multi-agent system stochastic consensus game(MAS-SCG), to achieve the coordination and optimization of the decentralized VTC, such that different types of renewable energy can be effectivelly integrated into the FA. The proposed scheme is implemented on a flexible and dynamic MAS-SG based VTC simulation platform, which control performance is evaluated on a typical two-area load-frequency control power system and a practical Guangdong power grid model in southern China. Simulation results verify that it can improve the closed-loop system performances, increase the utilization rate of the renewable energy, reduce the carbon emissions, and achieve a fast convergence rate with significant robustness compared with those of existing schemes.As the conventional centralized AGC is inadequate to handle the ever-increasing penetration of renewable energy and the requirement of plug-and-play of smart grid, this paper proposes a mixed homogeneous and heterogeneous MA based WPH strategy to achieve a fast AGC power dispatch, optimal coordinated control, and electric power autonomy of an islanding smart distribution network(ISDN). A virtual consensus variable is employed to deal with the topology variation resulted from the excess of power limits and to achieve the plug-and-play of AGC units. Then an integrated objective of frequency deviation and short-term economic dispatch is developed, such that all units can maintain an optimal operation in the presence of load disturbances. Four case studies are undertaken to an ISDN with various distributed generations and microgrids. Simulation results demonstrate that WPH has a greater robustness and a faster dynamic optimization than that of conventional approaches, which can increase the utilization rate of the renewable energy and effectively resolve the coordination and electric power autonomy of ISDN.Based on the above-mentioned study, a multi-agent SGC simulation platform of power system based on JADE is developed using Java multi-agent development tools JADE in this paper. All the methods mentioned above can be integrated into the JADE platform, meanwhile, it can be very convenient to compare the effects of different SGC control strategies on the cooperative control of multi-area interconnected system. Additionally, it can be further studied in the engineering practical problems such as the architecture of the multi-agent communication system, the communication delay and the parallel computing to verify the effectiveness of the proposed methods.