Remote Sensing Image Tile Storage And Its Key Technology For Distributed Storage System Ceph

In recent years,remote sensing images have become one of the most important geographic data for scientific research,industry applications,and Internet companies.In addition,in order to effectively improve the efficiency of rapid sharing,processing,analysis and display of remote sensing image data,image slicing technology has gradually become one of the important methods of image external service provision.Therefore,remote sensing image tiles have also become the basis for external services provided by various industries Data,how to efficiently organize,store and manage massive remote sensing image tiles is the key to providing high-performance services to remote sensing images.The traditional remote sensing image tile storage system based on database and file system not only performs poorly in terms of storage performance,but also is difficult to expand and maintain,and can no longer satisfy the storage of massive remote sensing image tiles.In the new generation of No SQL databases and HDFS distributed file systems based on the master-slave architecture,due to the single-point failure of the master node,the master node tends to become the performance bottleneck of the entire storage system.At this time,the distributed storage system Ceph based on the idea of decentralized architecture has become the most massive storage of remote sensing image tiles because it is designed as a system with no single point of failure,high performance,high reliability and high scalability.Good choice.But it has obvious performance defects when storing small files like massive remote sensing image tiles.In view of the above problems,based on the predecessor's small file merging scheme,this paper fully considers the spatial and temporal characteristics of remote sensing image tiles,and designs and implements a multi-level optimized storage method TMOSM for massive remote sensing image tiles.The main research contents of this article are as follows:(1)Multi-level optimized storage method TMOSM for massive remote sensing image tiles is designed.Aiming at the problem of Ceph's poor performance when storing massive remote sensing image tiles and other small files,this paper designs a multilevel optimized storage method TMOSM for massive remote sensing image tiles based on remote sensing image tile merging and prefetching cache strategy.The read and write performance of remote sensing image tile storage for Ceph distributed storage system is optimized from the aspects of remote sensing image tile merge storage strategy,global mapping index strategy and prefetch cache strategy.Experimental results show that this method can effectively improve the read and write performance of the Ceph remote sensing image storage system.(2)Propose a combined storage strategy for remote sensing image tiles based on Z curve and consistent hashing.In order to merge remotely sensed image tiles with high spatial correlation and consider the scalability of the merged service cluster,this paper analyzes the deficiencies and defects of the existing data division methods,and proposes a remote sensing based on Z curve and consistent hashing Image tile merging strategy,which can merge the spatially adjacent tiles into a remote sensing image tile data set,and can allocate the data amount of the merged data set according to the performance of the merge service node,taking into account the merge service cluster's Scalability and data migration issues.(3)Designed tile prefetching technology and cache replacement strategy that take into account the temporal and spatial characteristics of tiles.In order to give full play to the storage performance of the cache server and improve the reading performance and access efficiency of remote sensing image tiles,this paper has developed a prefetch strategy based on depth-first strategy and breadth-first strategy,making full use of the pause time when users perform map operations,according to The user's access feature prefetches the tile data that may be accessed next to the server's local cache to improve the user's response speed and the efficiency of accessing the tiles.In addition,a tile cache replacement strategy based on the spatiotemporal feature value of the tile is designed,and the access characteristics and spatiotemporal characteristics of the remote sensing image tiles are used to improve the reading performance of the remote sensing image tiles.