The Research And Application Of Massive Traffic Data Based On Hadoop

In order to better manage urban traffic system, people have built intelligent transportation systems. In this system, people could collect traffic information and monitor the status of the roads by installing a variety of detecting sensors on streets. However, with the increasing complexity of the road network, the rapid growth in car ownership, intelligent transportation systems have collected vast amounts of low-value density information. Thus, the question how quickly and accurately to dig out useful information from these large-scale low-value data to solve the problem of urban traffic has been becoming the pursuit of researchers.In traffic problem, the discrimination and forecast of traffic jams is an important area. In previous method, people have to calculate some traffic parameters to finish the judgement of normal blocking point, and then solve the congestion problem by planning of roads and control of traffic lights. However, the data objectively reflecting the status of roads and the real-time commands by police have not been considered. Moreover, the increasing amount of data leads to the geometric growth of calculation, the large consumption of time, the loss of real-time forecasts of traffic. In addition, the division of traffic cells is the meso-level of the research on the rules of transportation, and the reasonable division is helpful to develop effective traffic management measures. We have completed some innovative researches on traffic data for mining applications. These researches are based on actual data from transportation system in Hangzhou, including microwave detection data, floating car GPS data, video surveillance data,road network data, and are combined with the advantages of Hadoop platform for massive data processing. And these researches are presented as following:1. The concept of traffic abnormal blocking points and the distributed detection algorithm are firstly proposed in this paper. It plays an important role in guiding the real-time optimization of limited police force and it is essentially different from traffic state classification criterion.By introducing congestion probability in history, the concept of "abnormal" is defined and the distributed computing model is built. And further, by the effect of "cumulative abnormal", the accuracy of real-time warning could be enhanced. What’ more, the proposed algorithm has the nature of "self-learning", which is presented in continuously updating of the historical congestionprobability. And the applicability of the method will not be affected greatly, even if changes occur on traffic organization and road infrastructure,.2. A rapid distributed density clustering algorithm is proposed, which is based on massive data of transportation. In general density clustering algorithm, the input parameters can do influence to data clustering. But this algorithm could avoid it and improve the operational efficiency when facing large data sets. Meanwhile, with the use of distributed method in calculation of thresholds for point density, distance between point cells etc., we could achieve point clustering quickly. And it can provide decision-making basis for dividing the traffic cells.