The Research Of Overlapping Communities Detecting Algorithm Using Hadoop Based On Edge Clustering

Nowadays, the world is full of all kinds of relationships. These relationships can be simplified with the principles of topology by lots of network maps. The nodes having close relationship can be divided in the same community, so that we can divide all nodes in to a number of communities, that is complex network division. Depending on the topology of complex networks, revealing its implicit rules and predicting its behavior patterns often have a great practical and theoretical significance. Particularly, it have great theoretical significance and practical value in social networks, biological networks, the Internet and mobile Internet. But in the real world, the relationship between people and people, things and things, people and things are more complicated. Often a node at the same time belong to multiple communities simultaneously. That is the problem of overlapping communities detecting. Therefore, the detecting of overlapping communities have more important practical significance than detecting of general complex networks.Since the beginning of 2000, scientific researchers have found a variety of ways to detect the research of community issues. Such as the improved idea based on group penetration, the idea based on fuzzy clustering, the idea based on decompose non-negative matrix, the idea based on seeds extension and the idea based on mixed probability model. These researches made a lot of achievements. However, these algorithms to find the overlapping network of large and complex community issues, in dividing accuracy, time complexity has certain defective, often can not be used in the actual large and complex networks. To solve this problem, this paper proposes a novel algorithm, clustering based on the similarity of the side edge of the structure, and then through distributed processing, solve the problem of large-scale complex networks.Validated by experiment, the algorithm can find a good overlapping community structure of complex networks. And uses the idea of distributed processing, we can solve the problem of overlapping communities of large scale complex network.