Technology Research On Parallel Training For Automatic Database Tuning And Database Parameter Ranking

Database parameter auto-adjustment is a hot topic concerned by both academia and industry.The end-to-end parameter configuration of database can be realized by deep reinforcement learning.However,the existing database automatic parameter adjustment system has too many parameters to be adjusted,and the training time of deep reinforcement learning model is too long.As a result,the man-machine interaction in the parameter adjustment process is tedious and the practical application effect is not good.How to accelerate the training speed of deep reinforcement learning model and effectively reduce the database parameters to be adjusted is a very valuable work.On the basis of CDBTune(Cloud Database Tune)that is one of database parameter autoadjustment system,inter-process communication as well as synchronization of the operating system,a parallel training system in which multiple models share samples is realized by using multiple CDB(Cloud Database)instances.The system includes a shared model and multiple independent local models.The shared model is responsible for saving the parameters of the latest stable model,and the local model is responsible for collecting samples in parallel and synchronizing the parameters of the latest stable model from the shared model in real time.On this basis,based on the random forest model in the machine learning library scikit-learn,the database parameter ranking function,which looks gini index as the criterion for judging the importance of database parameters,is realized by using grid search technology to determine the optimal hyperparameters of the random forest model,standardizing various data collected during the training process as well as learning and updating the random forest model.Experiments show that the parallel training system with multi-models sharing samples compare to the serial training system,when using two CDB instances for parallel training,its convergence time is reduced by 17.2%,and the acceleration ratio is 1.2;using three CDB instances for parallel training,the convergence time is reduced by 27.6% and the acceleration ratio is 1.38.Given the same performance goals,the time required for the parallel training system using 3 CDB instances is reduced by up to 60% compared to a serial training system,and the speedup ratio is up to 2.5.Given the same training time,the parallel training system using three CDB instances achieves the performance of 161% of the serial training system maximally.On the basis of CDBTune system,on average,the model based on random forest ordering only needs to adjust 5% of the parameters to reach 90% of the optimal performance,adjust 14% of the parameters to achieve 95% of the optimal performance,and adjust 38% of the parameters to achieve optimal performance.