Research And Application Of Runoff Analysis Method Based On Hadoop Platform

Droughts and floods have seriously restricted our national economic construction,the demand for water monitoring and analysis has become higher and higher.Modern hydrological monitoring and analysis system requires not only accurate short-term predictive capabilities to prevent sudden natural disasters,but also medium and long-term analytical capabilities to develop response measures.Due to its slow speed,high energy consumption and poor scalability,traditional single-server model of hydrological monitoring system is unable to meet the new demand.Because of the characteristics of resource allocation and scheduling,cloud platform is the development direction of hydrological information in water conservancy industry.In this paper,runoff is taken as the object of analysis.Runoff refers to the amount of fluid that flows through a closed channel or an open channel in a unit time.In the runoff analysis,the initial hydrological conditions and climatic conditions are important factors which influence the future runoff.Therefore,it is an effective method to improve the accuracy of medium and long term runoff analysis by determining the initial hydrological conditions and the reliable future precipitation analysis ability.This paper studies the methods of Hadoop cloud computing and runoff analysis,designs and implements a hydrological monitoring and analysis platform based on runoff analysis.Main tasks are as follows:(1)The hydrological information monitoring system based on Web application server is designed by using the mode of B/S and C/S.B/S model interacts directly with the user.C/S mode is used to receive hydrological acquisition terminals and external data.(2)Based on the traditional hydrological platform,an additional Hadoop computing cluster is added.In this paper,a new hydrological information monitoring platform consisting of Web server and Hadoop computing cluster is designed and constructed.System uses HDFS and MapReduce to solve the problem of storing and processing hydrological data.HDFS provides distributed file storage,and MapReduce provides a parallel data processing framework.(3)The application of random forests and support vectors in classification and regression problems is studied in detail.Compared with traditional decision tree,random forest has better generalization ability when the training sample classification effect is guaranteed.Due to the low requirement of the sample size,support vector machine is also applied to the problem of classification regression.On the basis of runoff forecasting service,the random forest and support vector machines are used to predict the monthly runoff of the the Yellow River River Basin.The corresponding prediction model is found by comparison.(4)In the process of constructing random forest,different from the traditional stand-alone model,the training process is decomposed into several Map sub-tasks,which are distributed to different sub-nodes of the cluster.After all the subtasks are generated,the result is returned to the master node's HDFS file system and the complete model is obtained.Experimental results show that this parallel method can improve the calculation speed,and the random forest model of combination prediction of monthly runoff is more accurate.It is a practical method for runoff analysis.