Research On Intelligent Operation Ticket System Based On Shaping-deep Deterministic Policy Gradient

The switching order is a significant measure in the operation management of power systems in China,which can prevent incorrect operations.The staff are asked to obey the switching orders when performing switching operations.In this way,the incorrect operation can be prevented and the loss of the equipment,finance and the personal safety caused by these operations can be minimized.With the extension of power grids and the rise of the amount of equipment categories,the assignment of switching operations is more complicated.This wants the automation level and the intelligence of operation ticket systems getting improved.Most of operation ticket systems are based on the expert system at present.It's usually difficult to establish the knowledge base.Besides,the management and the maintenance of the knowledge base will be hard if the operation rules,the equipment or the grid needs changing.With the development of artificial intelligence,deep learning and reinforcement learning have become two efficient measures in the area of machine learning.Combining the powerful perception of deep learning and the decision-making of reinforcement learning can solve many complicated perception-decision problems.We were considering to solve the problem of the switching order generation by the technology of deep reinforcement learning and then proposed an interference method named Shaping-Deep Deterministic Policy Gradient,which is implemented by the shaping technology and deep reinforcement learning.Then we designed the entire operation ticket system to improve the intelligence and the universal property of traditional operation ticket systems.The learning process of deep deterministic policy gradient is more steady and the speed of this process is more quick.In order to reduce the variance of this algorithm,the shaping technology is applied to this process.Following this,the entire switching order generation procedure is designed and implemented.Besides,the experience replay and the target network,which can remove the relevance among those data,are introduced into the process of the network training to further improve the system's steadiness.According to the operation rules,an assessment function is proposed to grade the state of the power grid.If the grid is in a good state,this function will give it a high score.Then,a reward function is implemented based on the assessment function,which is used to realize the conversation between the agent and the environment.Meanwhile,a data processing module and a convolution module are also applied to this system.These two modules can extract characteristics from the original data.This can improve the perception of the network.The operation ticket system in this article is implemented by the DJANGO.The data base is established by the MySQL.The inference algorithm is implemented by the TensorFlow and the Tensor Layer.This system can generate operation tickets by itself according to the operator's orders.Operators can also check or modify the previous tickets.By testing,it has been proved that this system is able to get correct operation tickets quickly by self-learning and can improve the universality and the flexibility of operation tickets systems.It can also give operators directive suggestions.