Study On Heat Distribution Optimization Of Multiple Heating Substations Based On Policy Gradient

Central heating has been widely used in China with the continuous development of industrial technology.However,there is still a large gap between the optimal control technology of China’s heating network and that of the developed European countries.The problem of heat distribution among multiple heating substations is still unsolved.Ways to create suitable heating strategies to obtain good heating results and reduce the waste of resources are still being explored.Due to the non-linear,considerable lag,time-varying,and strong coupling characteristics of the heat network system,traditional control methods are difficult to achieve better results.To solve the problem of heat distribution in the primary pipe network of multiple heating substations in the central heating system,a heat regulation controller based on a strategic gradient learning algorithm is designed in the paper to achieve the goal of uniform heat distribution and on-demand distribution,and the output results are analyzed to prove the control optimization effect.Firstly,the weather data of China Weather Network and the heat supply data of the heat supply company were used as the training samples for each part of the neural network to ensure that the training data were accurate and effective.For the heat load prediction model,the accuracy of the LSTM neural network can meet the requirements,so the 24-hour heat load prediction model of multiple heating substations is built using the LSTM neural network.Then the 24-hour heat load prediction value of each heating substation is calculated based on the model.After determining the algorithm to be used,the model is built by Tensor Flow,and the corresponding code is written to obtain the heat load prediction sequence for each heating substation for the next 24 hours.Then for the heating substation temperature control model,the LSTM deep neural network in the Tensor Flow deep learning framework was also used to model the heating substation system,and the deep neural network model was written in Python to obtain the neural network model representation between each heating substation.In order to get the optimized control sequence,finally,the neural network of the optimized control strategy is built by a reinforcement learning algorithm based on the DPPO algorithm of the Actor-Critic structure,and the optimized allocation of multiple heating substations is finally obtained by continuously optimizing the control strategy on the basis of the environment of the heating substation model,and finally,the primary network water supply flow control sequence is obtained.In this regard,the predicted heat load values will be used as part of the utility function calculation in reinforcement learning,and then the mathematical description of the primary side of the centralized heating system,i.e.,the objective function,will be established according to the actual operating conditions of the heating system,which will be solved using the reinforcement learning algorithm to obtain the sequence of optimized set values for the water supply flow rate on the primary side of each heating substation.The heating substation model is used as the model network in the environmental part of the learning strategy in reinforcement learning to obtain the final optimized control strategy.Finally,this setpoint and the calculated heat load values are used to compare with the heat load values calculated from the actual operating conditions and the predicted heat load.The comparison results are used to further improve and optimize the algorithm.