Research On Source-Load-Storage Coordinated Operation Optimization Of Active Distribution Network Based On Deep Reinforcement Learning

With the development of power electronics technology and the deployment of advanced metering infrastructure,the traditional distribution network is being transformed into the active distribution network(ADN)with multiple distributed energy resources(DERs)and more abundant control means,which has great potential in promoting low-carbon clean development,improving system operation efficiency,cutting peaks and filling valleys.At the same time,the secure and economical operation of ADN faces new challenges.The randomness and intermittency of distributed renewable generation(DRG)and the massive disordered access of demand-side resources increase the uncertainty of power flow distribution,which affect the secure operation and reliable power supply and greatly increase the difficulty of dispatch and control of ADN.In recent years,AI technology represented by deep reinforcement learning(DRL)develops rapidly.It has significant advantages in feature extraction of massive data,complex mapping relationship learning,online continuous decision-making,and is widely used in various fields.Therefore,the coordinated operation optimization method of source,load and storage in ADN based on DRL is expected to have a good application prospect.To fully exploit the flexibility potential of DERs,and overcome the randomness,uncertainty,difficulty in dispatch and other problems,this paper gradually in-depth studies the source-load-storage coordinated operation optimization of ADN from three perspectives:"source-storage" coordination,"load" side resource coordination,and"source-load-storage" coordination.The main work is summarized as follows:(1)In terms of exploring the source-side flexibility of ADN,a regional agents cooperative operation optimization method of ADN with consideration of "source-storage"coordination is proposed.Firstly,a "source-storage" coordinated operation optimization model of ADN is constructed,which includes DRG,controllable distributed generation(CDG),and DES.Then,based on the multi-agent deep reinforcement learning(MADRL)theory,the model is transformed into a Markov game(MG)and regional agents are constructed.Finally,the multi-agent twin delayed deep deterministic policy gradient(MATD3)algorithm is used for centralized training to realize distributed deployment.The case study shows that the proposed method can form adaptive strategies to effectively cope with the umcertain factors in ADN through exploration and learning,ensure the economic operation of ADN,and alleviate the high proportion of DG access problem.In addition,this method has the advantages of online decision making.It can effectively reduce the difficulty of centralized control of ADN,and maintain the robustness of decision making in the case of a certain degree of communication failure.(2)In terms of exploring the load-side flexibility of ADN,taking electric vehicles(EVs)as the research object,a collaborative charging optimization method for EVs based on DRL was proposed.Firstly,EV is regarded as an adaptive subject,and a collaborative charging optimization model is constructed for each EV.Then,based on the MADRL theory,the model is transformed into a MG and EV agents are constructed.Finally,the MATD3 algorithm is used for centralized training and distributed deployment.The case study shows that the proposed method can avoid deepening the peak-valley difference and load fluctuation,and effectively reduce the charging cost of an EV.In addition,this method can deal with the uncertainty of EV charging behavior and rapidly generate collaborative charging strategies.(3)In terms of exploring the flexibility of "source-load-storage" coordination,based on the above studies,this paper further proposes a regional agents cooperative operation optimization method in ADN considering the coordination of "source-load-storage".Firstly,a distribution network partitioning method is proposed considering the electrical coupling degree and power matching degree.Then,a "source-load-storage" coordinated operation optimization model of ADN was constructed,including DG,DES,EV,and demand response(DR)load.Based on the MADRL theory,the model was transformed into MG and regional agents were constructed.In addition,considering the more complex "source-load-storage"coordination environment,the exploration difficulty of the DRL algorithm increases,and an improved MATD3 algorithm based on the gated recurrent unit and attention mechanism is proposed and used to complete the centralized training and distributed deployment.The case study is given to verify the effectiveness of the proposed method in promoting source-load-storage coordinated operation optimization of ADN.