Research On Distributed Storage And Parallel Processing Of Spatial Data And Its Application In Forestry

A series of links involved in forest resources are inseparable from objective,realistic and accurate data,however,traditional forest resource monitoring and survey methods can no longer meet the needs of fast and accurate data,modern analysis and management technologies such as 3S and computer technology are playing an increasingly important role in forest survey management.With the rapid development of 3S technology,a large amount of spatial data is being integrated worldwide every day,the era of geospatial big data has arrived,the efficient organization,storage and rapid processing of massive spatial data has become a research hotspot.As typical spatial data,remote sensing image and spatial vector have been widely used in forestry,military,mapping,transportation and other fields,many researchers have carried out research on distributed storage and parallel processing of these two types of spatial data,and have achieved certain research results,but in the current research,the storage structure is not well designed in combination with the characteristics of the data,and it is rare to accelerate the parallel processing based on the optimization of the storage.Therefore,when facing the larger volume of data and applications with higher real-time requirements,the current research will reach the bottleneck.This dissertation aims at the shortcomings of remote sensing image and spatial vector data in the research of distributed storage and parallel processing,based on big data and cloud computing technology,this dissertation carry out research on distributed storage and parallel processing methods of remote sensing image and spatial vector data,propose an efficient and fast spatial data storage model and parallel processing method,and design and implement the data storage to processing interface,finally,apply the research results to the forestry field,parallel inversion of forest vegetation coverage.The research contents of this dissertation are as follows:(1)Study the distributed storage model of remote sensing image.In order to solve the problems of the current storage system,such as complex hierarchy and the fixed block strategy,a distributed storage model for remote sensing images,MapImage,is implemented based on HDFS.The storage model fully considers the relationship among image pixel data,image pyramid,and metadata,at the same time,according to the processing algorithm access characteristics,it provides data segmentation strategies by band,rectangle,row,and column for selection,improved system availability.(2)Study the distributed storage model of spatial vector data.The current storage model data access I/O delay is large,and the vector tile pyramid and its metadata are not considered.In order to solve the above problems,VectorTileStore,a memory-level vector distributed storage model compatible with vector tile pyramid is designed based on Alluxio,the model supports the storage of vector tiles and their metadata,the data is grid indexed during storage to provide a memory-level fast data access interface.(3)Research on parallel processing methods of spatial data.In order to accelerate the processing of spatial data,research on parallel processing of spatial data is carried out.Firstly,for the problem of time-consuming of data pyramid construction,parallel construction of image pyramid and vector tile pyramid based on Spark and parallel load to distributed storage model.Then according to the characteristics of the distributed storage model,design and implement the corresponding parallel input format,based on the parallel input format,Spark reads the storage model on demand,filters data that is not related to the algorithm,and speeds up parallel processing.In this paper,the space vector query algorithm and remote sensing image mosaic algorithm are selected as examples.(4)Design and implement the interface of spatial data distributed storage and parallel processing.Based on the research of spatial data distributed storage model and parallel processing,the remote sensing image and space vector data storage,access,and processing interfaces are designed and implemented,and the interface usage rules are elaborated to form a universal framework from spatial data distributed storage to parallel processing.(5)Study on the parallel inversion method of remote sensing of vegetation coverage.Based on the proposed spatial data distributed storage model and parallel processing method,combined with the theory and method of vegetation coverage inversion,the MODIS image of Maoershan Forest Farm of Northeast Forestry University was used as the research object to invert the forest vegetation coverage in parallel.The research and experimental analysis show that the spatial data distributed storage and parallel processing method research carried out in this paper can effectively combine distributed storage and parallel computing technology theories and methods to achieve distributed storage and parallel processing of remote sensing images and spatial vector data,and has better performance than current research,Maplmage access efficiency increased by 25.4%and 36.9%,VectorTileStore access efficiency increased by 11.4%and 40.6%.Finally,the research results are applied to the forestry field,and the remote sensing parallel inversion research of forest vegetation coverage is carried out to provide a new forestry informationization solution.