Research And Implementation Of Deep Learning Computation Graph Optimization Based On Reinforcement Learning

Deep learning has brought great changes to many fields such as computer vision and natural language processing,and with the rapid development of deep learning,the deep learning models applied to practical problems become increasingly large and complex,and the time cost of model inference and training increases significantly,therefore,how to accelerate the computational process of deep learning models has received much attention from researchers in recent years.This thesis is concerned with graph substitution-based computation graph optimization.in deep learning frameworks and deep learning compilers,deep learning models are abstracted as computation graphs,and subgraphs of the target computation graph can be equivalently replaced with another subgraph in the deep learning framework by invoking graph substitution rules,which can simplify the topology of the computation graph after substitution,thus improving the operational efficiency of the deep learning model.Jia Zhihao and other researchers introduced TASO,a deep learning optimizer that can automatically generate graph substitution rules.The problem addressed in this thesis is to select some rules from the set of graph substitution rules composed by TASO to form a sequence and optimize the target computation graph using the rules in the sequence.The challenge of this problem is that the many rules generated by TASO constitute a huge search space,and selecting the optimal sequence of graph substitution rules for a particular computation graph has been shown to be an NP-hard problem by related studies.This thesis proposes a computation graph optimization algorithm CGORL based on deep reinforcement learning,which uses the graph substitution rules contained in TASO to optimize computation graphs.First,the computation graph optimization problem requires processing data of graph structure,and the complexity of the graph structure makes it very difficult to extract the features of the computation graph manually.For this problem this thesis builds a computation graph embedding model based on graph convolutional neural networks,which can map the computation graph into a low-dimensional feature vector space;second,the CGORL algorithm of this thesis is designed based on the proximal policy optimization algorithm The algorithm designs reward values based on the computation graph running time estimated by the cost model,and the action space is the graph substitution rule contained in TASO;finally,for the sparse reward problem in computation graph optimization,this thesis solves it by adding the intrinsic curiosity module in the CGORL algorithm.Through experimental analysis,the CGORL algorithm achieves certain optimization results and reduces the time overhead for both the Inceptionv3 and Nasnet-a models compared to the backtracking algorithm in TASO;finally,this thesis presents a prototype system for computation graph optimization based on the research results of the CGORL algorithm and developed using the Django framework.Users can upload ONNX files and use the computation graph optimization service of this prototype system to optimize the deep learning models saved in the files.