Design And Implementation Of Virtual Network Embedding Algorithms Based On Reinforcement Learning And QoS Awareness

With the rapid development of multimedia and the emergence of a series of large-scale and massive data services such as cloud computing and mobile Internet applications,how to improve the flexibility of network resource management and allocation to meet diverse business requirements has become a major challenge facing the Internet.The emergence of network virtualization technology has solved this problem well.Network virtualization enables multiple different virtual networks to share the resources of the same substrate network,so that network infrastructure providers can flexibly manage and allocate resources.Virtual network embedding(VNE)determines which resources of the substrate network can be used by a virtual network with resource requirements.Therefore,the VNE algorithm determines the resource utilization of the substrate network and the ability to support multiple VNRs.Most of the existing VNE algorithms focus on improving the probability of the current VNR being accepted,and improve the acceptance ratio of VNRs by optimizing the node mapping phase,while the link mapping phase only uses the simple shortest path algorithm and ignores the influence of the difference of link resources,resulting in the sub-optimal mapping results.Meanwhile,in recent years,various types of multimedia applications have appeared on the Internet.Cisco predicted that approximately 90%of Internet traffic is generated by quality of service(QoS)-sensitive applications,and the QoS requirements of different applications are usually different.However,currently most VNE algorithms do not consider the QoS requirements of VNRs,and only a few VNE algorithms consider delay metric,which cannot fully meet the resource requirements of QoS applications;and the existing VNE algorithms treat all requests consistently,that is,strictly guarantee the QoS requirements of all VNRs,resulting in low request acceptance ratio and resources utilization.To solve the above problems,we first propose a VNE algorithm based on double reinforcement learning,which applies reinforcement learning to both node mapping stage and link mapping stage,so as to optimize both link mapping and node mapping.In addition,we further propose a VNE algorithm of QoS-aware based on reinforcement learning,which classifies different VNRs based on different demands of multiple QoS metrics including delay,jitter and packet loss rate,and introduces reinforcement learning to provide VNE of differentiated service.That is,a mapping with strict QoS guarantee is provided for the VNRs with high QoS level,such as game applications,and a mapping with lower QoS guarantee is provided for the VNRs with low QoS level,such as broadcast video,so as to achieve a balance between QoS guarantee and request acceptance ratio.In this thesis,we first analyze the existing VNE algorithms,and summarizes their pros and cons.Then the two proposed VNE algorithms and their performance evaluation are introduced in detail.Finally,taking the QoS-aware VNE algorithm as an example,the design and implementation of the simulation experiment and evaluation platform are introduced.Simulation experiments show that the proposed algorithm based on double reinforcement learning improves the request acceptance ratio and resource utilization of the hierarchical substrate network compared with the existing heuristic and single-layer reinforcement learning VNE algorithms,while the QoS-awareness algorithm improves the request acceptance ratio and resource utilization under the same time complexity by only sacrificing less QoS.