| With the rapid development of information technology, Internet has become a complex network containing huge amounts of information and users, and played an important role on transmitting and sharing information for the world. However, a good deal of information resources led to the "information lost" and "information overload" and other issues, so users often disorientate in the sea of information. It is more and more difficult to accurately and effectively obtain the information they need. Then, personalized service technology emerges as the times require. Recommendation system as a kind of mature personalized service technology has been widely used in such fields as date finder, e-commerce, and scientific research and so on.The common recommendation technologies include:collaborative filtering recommendation technology, the recommendation based on model, the recommendation based on content, and hybrid recommendation. Among them, the collaborative filtering algorithm is a higher precision, simple recommendation one and easy to use in an actual system. So it is very popular. However, in addition to the sparse and cold start problems, the classic collaborative filtering algorithm’s another essential problem is scalability problem, namely, as the increase of the number of users and commodities, the time complexity increases rapidly. It will affect the real-time recommendation.To solve the scalability problem of the collaborative filtering recommendation algorithm, from the project or user vector representation method, this thesis analyzes the defects of the object properties strictly matching, and the collaborative filtering recommendation algorithm based on mean model is proposed. Also, this thesis focuses on the operation mode of system, then design and realizes a new collaborative filtering recommendation algorithm based on mapreduce.There are two main innovation points in this thesis. The first one, an item vector representation is proposed based on mean model. The model appropriately compresses the length of vector through extracting project’s or user’s features. It gives support to efficient and real-time recommendation in large-scale data set.The other one, this thesis find the improving method based on mapreduce by evaluating the key steps of the collaborative filtering recommendation algorithm. Then, according to the principle of Hadoop, this thesis chooses the right map or reduces function to solve problem. The same time, this thesis transforms the three key steps of the collaborative filtering recommendation algorithm based on mean model to Map or Reduce operations, involving the data expression, the calculation and so on. It is help for achieving the parallelization of the algorithm.Experiments used Movie Lens100k datasets, Movie Lens1M datasets, and Movie Lens lOMdatasets, provide by Group lens. The classic collaborative filtering recommendation algorithm based on item, the commendation algorithm based on cloud model and the collaborative filtering algorithm based on mean model had carried on these contrast experiments. These experiments paid close attention to the precision and running time of algorithm. Experiments showed that the proposed method was efficient to reduce the computational time with higher precision through contrasting with two baseline algorithms. Furthermore, the above method based on mapreduce had made great leap in performance, scalability, and fault tolerance for recommendation system. Finally, to attest the practicability of the proposed algorithm, this thesis designed and realized a prototype system based on the above method. |
