Research And Realization Of The Real-time Bidding Model Based On A Constrained Markov Decision Process

As one of the fastest growing areas in the Internet industry,online advertising is a marketing pattern that uses the Internet to locate the target users.The real-time bidding allows advertisers to bid on real-time for each ad impression.Typically,the real-time bidding is the second price sealed auction,which encourages bidding with real value.However,the optimal bid price is largely unknown in practice due to various factors.Therefore,how to optimize the bidding strategy becomes the core issue in the real-time bidding.In most real-time bidding systems,it is an important basis of bidding strategy to predict click-through rates of the ads.In recent years,researchers have made great progresses in using deep neural networks to predict click-through rates.However,the existing models still have much room for improvement in the ability to express high-and low-order interactions.In addition,most of the existing bidding strategies use static optimization methods,which calculate bidding price in a sequential and independent way.This may result in budgets being exhausted too early to maximise the revenues of the advertisers.Therefore,another challenge is how to dynamically optimize the bidding strategy under the constraints of a given budget.The main contributions of this thesis are:1)Proposing a hybrid model which is a deep neural network based on product and factorization machine.It further enhances the ability of the model to express high-and low-order interactions,and improves the accuracy of the model prediction.Experiments on iPinYou data set show that the AUC value of this model increases by 3%compared with LR model,2.6%compared with FM model,1.6%compared with FNN model,1.5%compared with PNN model,and 0.57%compared with DeepFM model.2)Applying reinforcement learning,modelling the bidding strategy as a dynamic optimization problem by a Constrained Markov Decision Process.This bidding model uses the budget as the constraint and the number of clicks as the reward.The experiments on the real-world dataset demonstrate that this model consistently better than the baselines on various aspects,including the number of clicks,win rate,CPM,CPC,etc.3)With the designed click-through rate prediction model and bidding model as the core,designing a bidding system to calculate the bid price,which can be used for the demand side platform.