Research On Partition Cooperative Voltage Control Of Active Distribution Network Based On Multi-agent Reinforcement Learning

With the proposal of the national dual-carbon strategy,the penetration rate of distributed generation in the distribution network continues to increase,effectively improving energy utilization and reducing environmental burdens.At the same time,the high proportion of distributed generation grid-connected transforms the traditional distribution network into an active distribution network with multiple power sources,which changes the power flow distribution and causes power flow reversal;and the uncertainty of distributed generation output will also lead to frequent voltage fluctuations Even exceeding the limit will affect the power supply quality of the distribution network.These have brought many challenges to the operation and control of the distribution network.In this paper,research is carried out on the problem of optimal control of distribution network voltage with high proportion of distributed generation access,and a cooperative control strategy for active distribution network partitions based on multi-agent reinforcement learning is proposed.Using the exploratory learning ability of multi-agent reinforcement learning,a model-free partition cooperative control method is developed,which can realize real-time voltage control based on the offline training-online execution mode.Unlike the local voltage control method which is difficult to achieve overall coordination and the centralized voltage control which relies heavily on the monitoring and communication of the whole network data,the partition coordination strategy in this paper can realize the regional autonomy-global coordination control pattern under weak communication conditions.Firstly,the regional division method and implementation of the distribution network are described.The electrical distance is represented by the reactive power-voltage sensitivity matrix,the partition is divided by the spectral clustering algorithm based on graph theory,and the modularity is used as an index to measure the quality of the regional division.The optimization of the partition,the verification and adjustment of the partition scheme are introduced in sequence And so on the implementation process,and carried on the numerical example verification.Secondly,this paper introduces the acquisition process of regional cooperative control strategy for multi-agent reinforcement learning.A mathematical model of voltage control is established to minimize voltage deviation and converted into a partially observable Markov decision problem.Each region is modeled as an agent,and the multi-agent double delay deep deterministic strategy gradient(PER-MATD3)algorithm based on the preferential experiential replay mechanism is used to get the zonal cooperative control strategy model.Through experiments,it is proved that this model can effectively solve the voltage overlimit caused by grid-connected distributed generation,and can meet the requirements of real-time voltage control.Then,aiming at the difficulty of obtaining the accurate physical model of the active distribution network and relying on the traditional power flow calculation to train the deep reinforcement learning model,the construction method and process of the data-driven power flow calculation model are introduced.The historical operation data of the power grid is used to train the data-driven power flow calculation model based on graph convolutional neural network to fit the nonlinear relationship between node power and voltage.Embedding the trained model into the agent-grid interactive environment can replace the traditional power flow calculation program and be used to calculate the reward value of agent learning.Finally,aiming at the problem of multi-agent trust assignment,a multi-agent reinforcement learning partition cooperative control strategy considering trust assignment is studied.Aiming at minimizing the voltage deviation and network loss,and converting the voltage control problem into a partially observable Markov decision problem,an improved multi-agent flexible action-assessment algorithm(COMASAC)based on counterfactuals is used to design a trustbased Allocated distribution network partition cooperative control strategy model.Through experiments,it is verified that the model can solve the trust assignment problem and obtain more efficient control performance.