Research On Key Technologies Of Dimension Reduction Storage And Hybrid Management For Striping Spatial Big Data In Highway Operation Management

Highway plays an important role in the modern transportation system with its characteristics of flexibility, large volume and high speed. Highway operations management is a vital part of its whole lifecycle, including conservation, traffic, security and service of it. Being part of highway information construction, spatial information technology runs through the whole process of highway operation management. In recent years, with China’s high-resolution earth observation systems and Beidou satellite navigation systems building steadily, and technology of Internet of Things and Internet of Vehicles developing continually, spatial data of highway operation management gradually shows big data features of volume, variety, velocity and value. Since traditional storage and management technology is not only difficult to meet the needs of big data, but lack of pertinence on striping spatial data. Therefore, how to efficiently manage highway operations management spatial big data is urgent to be solved.Based on NoSQL databases and distributed cloud storage technology, considering features and management needs of highway operation management spatial data, this paper proposed striping dimension reduction organization model and multi-state hybrid storage architecture of highway spatial big data, to address storage and retrieval efficiency problem of highway operation management spatial big data, which is massive, multi-source and heterogeneous. The main contents of this paper are as follows:(1) Feature analysis of highway operation management spatial big data. Through analysis of different data sources, we summarized its big data feature and striping spatial distribution characteristic. Further we classified the data according to its feature and clarified its need of storage and management, to make the design of data organization and storage more targeted.(2) Striping dimension reduction organization model of highway spatial big data. By analyzing the theories and methods of spatial data dimension reduction, we pointed out that spatial grid is not only an effective way of reducing the spatial data dimension, but also the basis for spatial encoding. Through comparing spatial scale of Geohash grid and highway space, we proposed a highway spatial grid division method. By combining Geohash grid based spatial data dimension reduction method and one-dimensional linear referencing system of highway itself, we proposed striping dimension reduction organization model of highway spatial big data with storage and retrieval methods of point, line and polygon.(3) Multi-state hybrid storage architecture of highway spatial big data. On the basis of analysis of different spatial big data storage technology, we proposed to integrate NoSQL databases, distributed cloud storage and spatial database engine seamlessly, to achieve hybrid storage of multi-state highway spatial big data of dynamic and static, structure and unstructured, spatial and non-spatial. We proposed an index-associated coordinating management engine in hybrid storage, which uses dimension reduction of spatial information as index to build connections among the highway spatial big data. Through that we achieved seamless integration and storage of highway spatial big data. Further, we designed remote sensing image pre-partition strategy according to the striping feature of highway space to improve the efficiency of on-demand data access in the hybrid storage architecture.(4) Highway operation management spatial big data management prototype system. We designed the framework for the prototype system and realized several core functions, including data warehousing, data querying, system monitoring, and data visualization. Then we demonstrated application of prototype system in emergency processing of highway geological disaster. Through comparison experiment of querying performance with traditional spatial database engine, we demonstrated that the proposed storage and management technology of highway operation management spatial big data is feasible and has good performance.