Research On Image Parallel Processing Based On MapReduce Model

In recent years,along with the rise of cloud computing and big data,the amount of data generated by various applications in the network has grown rapidly and has reached the PB level even higher.In these data,image processing and storage of big data has become a hot topic of research.MapReduce technology is a highly reliable parallel programming framework,commonly used for large amount of data in parallel computing,complex cluster environment has a very good solution.One of the cores of the Hadoop platform,MapReduce,is to use the technology to process massive amounts of data,and it has achieved good results,but the lack of research on image files,especially the small image files.In this paper,we discuss the parallelization of the image under the MapReduce model,and give a new data platform architecture which can be used as a new data platform for processing small data files,and the main idea studied and included are as follows:Firstly,the paper introduced the research background and the state of it.Meanwhile sums up the study meaning of mass image data processing in the big data background,and introduces Hadoop ecosystem,including its core HDFS system and MapReduce framework technology.Secondly,the design of how to improve the Hadoop system,put forward a combination of fragmented methods to improve the processing efficiency of small images,the expansion of the MapReduce software framework,so that it can be a good support and image files.Finally,this paper designs and implements the parallel K-means clustering analysis of the image under the MapReduce model,parallel Sobel edge detection of the image,and extends the parallel histogram extraction of the image under the MapReduce model.Through the experimental verification,the feasibility of the extended MapReduce model and the efficiency of the image files.Through the index performance analysis,the effectiveness of image parallel processing under MapReduce model is verified,which provides a feasible solution to the application of massive data image file in Hadoop.