Deep Autoregressive Model For Cardinality Estimation

Cardinality estimation is a core and fundamental task in query optimization for database systems,and it plays a vital role in the modern database research field.Traditional methods for solving the cardinality estimation problem usually rely on a single statistical method,such as histograms,sketches,or kernel density functions.However,these methods have proven to be insufficiently accurate and unable to handle complex scenarios in the era of massive data.Therefore,researching better cardinality estimators has become necessary to improve the performance of query optimization in modern database systems.In recent years,introducing machine learning to database problems become a hot topic in the database field.Learning-based cardinality estimators can leverage the characteristics of database tables and provide more accurate estimation results.To explore more advanced cardinality estimation technologies,the main contributions are as followed:1.In real data scenarios,general learning-based methods do not effectively utilize and optimize the sparsity of data and other features.Meanwhile,due to poor interpretability of learning methods,errors often propagate during the learning process.Therefore,this paper proposes a cardinality estimation method SAM-CE based on a smoothing strategy,which can effectively improve the quality of estimator sampling and distribution modeling.Through testing on authoritative datasets,SAM-CE achieves advanced cardinality estimation accuracy in single-table cardinality estimation tasks.2.Based on SAM-CE,this paper improves the join query and designs a multi-table estimation strategy that learns the full join table distribution of data tables,enabling the smoothing depth autoregressive cardinality estimator to handle more complex query work scenarios.Accuracy experiments on public datasets show that this method achieves highprecision cardinality estimation for complex queries.3.Based on the autoregressive cardinality estimation method above,this paper combines with the Postgres database to implement a cardinality estimation system.This system includes a parsing layer,an information extraction layer,a model layer,and an application layer,which are designed with low coupling principles.This system can help detect the quality of database cardinality estimation and improve the performance of database query optimization by introducing the cardinality estimation algorithm proposed in this paper.