Markov Game Based Pricing Strategy For Competing Cloud Providers

With the development of the Internet,cloud computing technology is becoming more and more mature,and its huge commercial prospect attracts more and more enterprises to enter the cloud market providing cloud services,thus forming a competitive cloud market with multiple cloud providers.In the cloud market,multiple cloud providers compete with each other to attract users and maximize profits.In the extremely competitive environment,each cloud provider needs a reasonable pricing policy to maximize profits.By referring to a lot of literatures,related research does not consider the extremely competitive environment with multiple proactive providers and the pricing process built as repeated games.Based on this,this paper analyzes the pricing strategies of cloud providers in the competitive environment.In the cloud market,multiple cloud providers compete with each other,and their pricing strategies affect each other,so they have to update optimal prices periodically,which is a complicated Markov game.This paper will examine it based on two kinds of multiagent reinforcement learning algorithm,Nash Q-learning and Minimax-Q.Our main contribution are as follows:(1)According to the characteristics of competition among the cloud providers,this paper firstly presents the basic settings of the game,including choice behavior of cloud users,the marginal cost and calculation method of expected profits of cloud providers.(2)In the framework of Markov games,the competitive pricing process of multiple cloud providers in each period is treated as a general-sum game intuitively.Then,based on equilibrium strategies,this paper uses Nash Q-learning algorithm to analyze equilibrium pricing strategies of competing cloud providers.For Nash Qlearning,when there are more than one Nash equilibrium,we cannot find specific and effective solutions in related literature.So,by considering expected profits and Fictitious Play(FP)algorithm,we firstly propose four available approaches to choose Nash equilibrium.We then train cloud providers based on these four approaches and obtain four different equilibrium pricing strategies.By the four equilibrium strategies contrast each other,and the comparison with other pricing strategies in two typical scenarios of cloud market,we discover the two equilibrium strategies which are obtained by FP algorithm have relatively better performance,but they are not stable.The experimental results provide references for the optimization of Nash Q-learning algorithm.(3)Due to the characteristic of competing cloud providers,therefore in the framework of Markov games and from the perspective of a zero-sum game,this paper uses the Minimax-Q algorithm to analyze the pricing strategies of competing cloud providers based on the setting of only two proactive cloud providers in the market.Firstly,we train cloud providers with Minimax-Q algorithm and joint Q-learning algorithm and obtain four different pricing strategies.Then,in the three typical scenarios of cloud market,we compare these four strategies with other pricing strategies.We find that the performance of the two Minimax strategies is outstanding and stable in each scenario.Finally,we compare the two Minimax pricing strategies with the four equilibrium strategies based on Nash Q-learning algorithm.We find that the two Minimax strategies have more outstanding performance with the setting of extremely competition of cloud providers.In addition,we find that the four pricing strategies based on Minimax-Q algorithm and joint Q-learning algorithm would rarely choose low prices.The experimental results show that the price war is not conducive to long-term profits.In addition,Minimax-Q algorithm is more suitable for the research of pricing policy with extremely competing cloud providers,and it provides references for optimizing the pricing strategy of competing cloud providers in the real situations.