Security Assessment And Stability Control For Complex Power System Based On Reinforcement Learning

With the increasing connection of new energy and power electronic equipment,and the deployment of ultra high voltage AC and DC projects,the national grid of China has become larger in scale,more complex in structure,and more changeable in operating condition.The rapidly increasing load demand also makes the transmitted power gradually approach the capacity limit.When the power system suffers from fault disturbance or malicious attack,it is easy to cause large-scale power flow transfer,serious cascading failures or transient instability problems,which ultimately threaten the safe and stable operation of the power system.Whereas,most of the existing methods for power system security assessment and stability control exist the problems of heavily relying on expert experience,poor adaptability to changing operating conditions,or low efficiency.In recent years,with the rapid improvement of computing power,artificial intelligence technology represented by deep learning and reinforcement learning has developed explosively.Its application in speech recognition,medical diagnosis,automatic driving and other fields has made remarkable progress.Reinforcement learning has the advantages of being fast,accurate and efficient when applied to decision-making problems.Hence,this paper deeply explores the inherent logical relationship between the algorithm characteristics of reinforcement learning and power system-related problems.The reinforcement learning based security assessment and stability control methods for complex power system are studied in this paper,in order to improve the accuracy and efficiency of decision-making for the assessment and control.The main research contents are as follows:(1)Aiming at the problem of static security assessment of complex power systems under sequential attacks,a power system resilience assessment method based on deep reinforcement learning algorithm is proposed.It evaluates the resilience of power systems under different operating conditions by deciding the shortest attack sequence that leads to the collapse of the system grid.A cascading failure model that fully considers generator characteristics and active power transmission loss is established to improve the accuracy of evaluation results.An improved priority experience playback mechanism is proposed to accelerate the convergence speed of agent training.Simulation results of power systems in different scales verify the effectiveness of the proposed method.(2)Aiming at the problem of dynamic safety assessment of complex power systems under severe faults,a reinforcement learning-based weak line assessment method is proposed considering transient stability constraints.The line weakness index is designed based on the collection of instability fault and the trained Q-value table.The double-Q learning and prioritized experience replay technique are introduced to improve the stability of the evaluation results.The simulation results of power systems in different scales show that the proposed method can screen out weak lines considering transient stability constraints,and reduce the amount of transient simulation calculations.(3)Aiming at the problem of transient rotor angle instability of complex power systems caused by serious faults,a knowledge fusion based deep reinforcement learning method is proposed for the intelligent predetermination of generator rejection scheme.A linear decision space is designed to avoid "dimensional explosion" when there are too many controllable objects.A knowledge fusion framework is proposed to apply knowledge rules to the decision-making in a way of invalid actions masking,in order to improve decisionmaking efficiency.A composite policy network is designed to extract multi-source heterogeneous state features of power systems to improve decision-making quality.The simulation results verify the effectiveness of the proposed method.(4)Aiming at the problem that the changed operating conditions and the time delay weaken the inter-area low-frequency oscillation damping,an adaptive wide-area damping controller based on multi-level Actor-Critic reinforcement learning is proposed.The adaptation of operating condition is accomplished by the online network update.An adaptive time-delay compensator based on the Pade approximation transform is designed to compensate the stochastic time delay.A double-loop supplementary damping control structure is proposed.to coordinate the active and reactive signals of the inverters of VSCHVDC.Digital simulation and hardware-in-the-loop experiments verify the feasibility and effectiveness of the proposed control.