Could Computing Model Base On Hadoop And Meteorological Application

The rapid development of Internet technology brings spurt growth of the massive data, so the traditional technology meets with inevitable efficiency bottleneck in dealing with these data. Especially in meteorology, remote sensing, geological disaster monitoring, and other special industries, the people’s property safety and social harmony and stability is related to whether can deal with these massive data in time safely and high effcient way. Cloud computing arises at the historic moment, and it has inherent technical advantage in handling massive data. The open source cloud platform Apache Hadoop is supported by well-known Internet companies and database manufacturer. Hadoop is getting more and more attention of researchers at home and abroad and has been becoming a hot spot in massive data processing field.This paper makes further research and exploration on HDFS and MapReduce of Hadoop. Combining with meteorological disaster monitoring and evaluation system, apply Hadoop in meteorological business system based on GIS. This article mainly does the work as follows:First, analyze characteristics of meteorological data, and points out a problem of directly storing meteorological data in Hadoop. According to the characteristics of meteorological data, design a file merging algorithm based on the trie tree. Then the experiment shows that the efficiency and security of data processing have been efficiently improved.Second, design and implement a Hadoop-based framework to storage and compute massive meteorological heterogeneous data. When real-time extracting effective data from massive meteorological data, the problems such as performance and efficiency, data privacy and security, disaster backup and so on are often considered, take advantage of HDFS to store meteorological data and MapReduce to construct parallel algorithm, propose a Hadoop-based massive meteorological heterogeneous data storage and processing framework. The practical application indicates that the framework proposed by this paper can not only more effectively process data, but also can effectively reduce cost and ensure the data security.This article combines the model proposed by the paper with the actual business, and make use of distributed framework technology to meet with the project’s needs, and deploy the model into the instance and use the actual running results to test and vertify the value of the model. The work not only has important theoretic value but also practical value in studying big data processing technology.