Research On Parameter Tuning Of Distributed Database Based On OtterTune

In recent years,in response to the growing demand for enterprise level application performance,large enterprises both domestically and internationally have adopted various types of databases.For massive data processing,distributed databases have become an important choice due to their highly flexible data storage and processing capabilities.In order to support HTAP(hybrid transaction/analytical processing),more and more enterprises are adopting enterprise level distributed databases.TiDB developed by Ping CAP company is a new domestic open source distributed database that can meet the above challenges.Database systems usually have hundreds of adjustable parameters,and the default parameter configuration is usually not optimal.The setting of these parameters directly affects the performance of the database,including the performance of the entire database system in terms of read and write performance,storage capacity,stability,and other aspects.Therefore,it is necessary to set parameters reasonably based on actual scenarios and requirements to maximize the data processing capability of the database system.To solve this issue,the database research team at Carnegie Mellon University developed the OtterTune tool,which can use historical tuning databases to construct supervised and unsupervised machine learning models.These models are used to map workloads,calculate and recommend optimal parameters,and thus achieve the DBMS parameter tuning process.This parameter tuning method not only improves the performance of the database system,but also reduces the complexity of the tuning process and the maintenance cost of the database.This paper takes the domestic distributed database TiDB as an example to study the parameter tuning of distributed databases.The aim is to provide scientific and reliable parameter tuning solutions for large-scale enterprise applications,and improve the performance and stability of database systems.The specific work is as follows:Firstly,this paper analyzes the architecture of TiDB database and the working principles of its important components,and studies the performance optimization problem of TiDB,proposing ideas for parameter optimization.Secondly,the Lasso regression algorithm and Gaussian process regression model were selected and implemented,and a parameter selection rule based on weight analysis was proposed.The OtterTune tool was analyzed technically.Three classic scenarios were designed for the core component of Ti KV in TiDB,and OtterTune was adapted to Ti KV scenarios.The Go-YCSB benchmark testing tool was used for scenario simulation.Finally,suitable parameters were selected based on the parameter selection rules proposed in this paper for three different scenarios.The OtterTune tool was used to tune Ti KV parameters,and the TiDB Dashboard was used to analyze the tuning effect and evaluate the parameter selection rules.The research results indicate that the performance of the database configured with tuning parameters is improved by 15.5%～23.5% compared to the database configured with default parameters,and the parameter selection rules based on weight analysis are reasonable and effective in the tuning scenario of this paper.