Research On Key Technologies Of Global Multi-source Remote Sensing Data Integration And Processing Platform

With the rapid development of earth observation technologies and the profusion of data acquisition platforms in recent years,remote sensing data is becoming increasingly available and gradually exhibiting more characteristics,such as multisource,multi-scale,complex-structured,diversely-formatted,vast-volume,etc.The study of remote sensing is stepping into a new era of big-data.Given the new data condition,in order to further enhance the frequency and accuracy of remote sensing observation on a global scale,it is necessary to produce spatial information products and build an integrated production system in a way of multi-source remote sensing data collaboration.Focusing on the system itself,there are four main issues to consider in the construction of global multi-source remote sensing data integration processing platform.(1)Data storage access: distinct sensors lead to different data storage structures and different access logics.To avoid redundant data copy and disordered increasement of data volume,a universal and transparent data storage and access mode is becoming more necessary;(2)Data organization and collaboration: multi-projection and crossscale data is hard to organize and retrieve,especially in large-scale production systems.Therefore,a flexible data cube structure which is suitable for synergized production is required;(3)Algorithm integration: the multi-layer nesting of global spatial information algorithms,variable and default input parameters,and space-time attribute restrictions make the large-scale integration of algorithms more complex,and thus an automatic production workflow integrating framework is becoming prominent;(4)Parallel processing: although parallel processing can be achieved in various ways,remote sensing algorithms still remains highly specialized and professionalized with less parallel processors involved,resulting in the lack of strategies and frameworks for efficient parallel processing of spatial information products.Aim at solving the aforementioned issues,this paper focuses on a series of key technologies for global multi-source remote sensing data integration,as well as the construction and implementation of the corresponding system platform.The contributions and innovations of this research are listed as follows:(1)A unified data structure and format abstraction library is designed.In view of the diversity of remote sensing data formats and the requirement for the collaborative use of multi-source remote sensing data,the hierarchical structure of data format that all data formats and format libraries follow is abstracted,and the unified I/O operation of different data formats is realized,which significantly releases the burden on remote sensing scientists to familiarize themselves with various format libraries and reduces the amount of coding.(2)A flexible multi-dimensional remote sensing data cube structure is designed.Aiming at the cooperative use of multi-scale data,a unified hierarchical division benchmark is constructed for multi-scale remote sensing data,which solves the benchmark problem of correlation among multi-scale data,organizes data cubes in real time in memory,saves the physical storage space for large-scale remote sensing data,and makes the production of large-scale synthetic products feasible in cluster environment.(3)A series of key technologies for the service-oriented integration of spatial information product algorithms are proposed.Fully considering the characteristics of multi-source collaborative spatial information products,such as recursive-nested invocation,complex processing model and heterogeneous processing platform,a complete set of technical systems,such as the production system architecture of information products,the construction of production business processes and the service integration of production models,have been established.The synthetical integration and collaborative production of multi-domain models are realized,and the difficulty of the integration of algorithms in industry domain is reduced.(4)A 2-layer high performance algorithm parallelization strategy is proposed.The upper layer employs task-level parallelization of algorithms,including the parallel computing strategy and the corresponding parallel accessing file system.The lower layer employs an image processing accelerating framework on the basis of memory data link-list,which serves as the general foundation of efficient remote sensing product processing.