Research On Virtual Network Embedding Algorithm Based On Deep Reinforcement Learning

Virtual network embedding is the core problem of network virtualization,and the performance of embedding algorithm determines the efficiency of resource sharing in network virtualization.Existing heuristic embedding algorithms often follow manually designed static rules,and the rule parameters are fixed and cannot be automatically optimized,which has certain limitations.In view of this,this thesis applies deep reinforcement learning with autonomous learning ability to optimize virtual network embedding(VNE).The main works are as follows:(1)In the case of having virtual network requests data as training samples,we proposed a VNE algorithm based on policy gradient and graph convolutional network(GCN).The algorithm designed a neural network model composed of GCN layer,convolutional layer,Softmax layer and node filtering layer to make node decision.Among them,the GCN layer modeled the substrate network and provided the feature representation containing topology information,the convolutional layer and the Softmax layer generated the probability distribution of substrate nodes being selected,and the node filtering layer filtered the substrate nodes that do not meet the constraints to avoid the selection of invalid nodes.During the model training process,offline train ing was performed on the entire training samples using the policy gradient algorithm to automatically optimize the node embedding strategy.Then,the trained model was used to directly infer the node embedding solution for virtual network request arriving online,and the k-shortest path algorithm was used to solve the link embedding.The experimental results showed that the proposed algorithm has high request acceptance rate,revenuef-cost ratio and substrate resource utilization.(2)In the absence of training samples,we proposed a VNE algorithm based on active search and pointer network.In this algorithm,the embedding of virtual network request nodes was regarded as a sequence-to-sequence problem,and a pointer network was introduced to solve the sequence generation problem.A neural network model composed of GCN layer,pointer network(composed of encoder and decoder)and node filter layer was designed for node embedding.Among them,the GCN was used to extract the substrate network nodes feature representation as the input of the pointer network,and then the node embedding solution was obtained through the decoder and node filtering layer.In the process of model training,an active search algorithm for online optimization of a virtual network request was proposed.For a virtual network embedding instance,the active search started with the random strategy and took the link embedding cost as the reward signal.according to this signal,the model parameters were automatically optimized,and the best embedding solution sampled during the search process was recorded as the final embedding solution.Experimental results showed that the proposed algorithm improved the request acceptance rate and the revenue-cost ratio.