Research On Cloud Computing Virtual Machine Placement Algorithm Based On Deep Reinforcement Learning

As an increasingly universal computing paradigm,cloud computing creates virtual machines and allocates the corresponding computing resources on physical machines in the data center for a massive amount of users.Virtual machine placement is a key issue in cloud computing,aiming to dynamically assign appropriate physical machines to virtual machines created on demand to improve the quality of cloud computing services and reduce energy consumption in data centers.As a current research hotspot,the virtual machine placement algorithm based on deep reinforcement learning has considerable advantages over traditional methods on handling complex optimization objectives and dynamically adapting to changing environments.However,the existing approaches based on deep reinforcement learning have shortcomings on insufficient anti-overfitting and poor state abstraction capability.Aiming at the drawback above,this paper proposed a virtual machine placement algorithm that integrates mutual information feature extraction and deep reinforcement learning as the improvement scheme.Firstly,this paper addresses the problems faced by deep reinforcement learning algorithms in dealing with the virtual machine placement problems such as easy overfitting and difficult convergence in complex states and high-dimensional actions.We propose a deep reinforcement learning model based on Dueling-DDQN,and further promote improvements in exploration approaches and training methods using Boltzmann exploration and Multi-step Bootstrapping for the virtual machine placement scenario.Compared with the unimproved deep reinforcement learning model,the average reward of the enhanced model in this paper is about 20%higher,the convergence time point during training is about 16%earlier.The oscillation amplitude of the reward curve after convergence is reduced by about 30%,the allocation failure rate is stabilized at 2%,and the model stability is improved.Compared with the classical First Fit algorithm,the algorithm we proposed results in a 15%reduction in data center energy consumption,about 60%improvement in user service quality,and a 20%improvement in the average integrated reward.Secondly,this paper addresses the problems of weak representation capability and poor model stability of existing deep reinforcement learning-based models in handling complex state spacesOn the basis of Dueling-DDQN model,this paper further proposes a state space pre-training approach based on Deep InfoMax and improves Deep InfoMax for the"physical machine order uncorrelation" and "time-dependent" characteristics in the virtual machine placement scenario.The experimental results show that compared with the method without feature extraction,the average reward of this method is improved by about 6%,the average energy consumption is reduced by 10%,the quality of user service is improved by 9%,and the convergence is more stable.The over-fitting phenomenon is reduced and performance of multiple models are enhanced at the expense of less time consumption.Research shows that the algorithm proposed in this paper has the advantages of fast processing speed,strong anti-overfitting capability,and strong stability.It is a cloud computing virtual machine placement method that can effectively adapt to complex environments.