Research And Implementation Of The Distributed Video Recommendation System Based On Mahout

In recent years, due to the rapid development of computer technology and Internet technology, the increasing amount of data results in the information overload problem. A solution called recommendation system is proposed to solve this problem. Recommendation is a way to extract the information that may enable users to generate interest from mass of information. The stand-alone version of the recommendation system cannot handle this massive amount of data nowadays, therefore the system need resort to the ability to handle massive data of cloud computing to meet current needs. Through research on existing video sites, most of which were found existing the problem of information overload, it is very difficult for users to find their favorite video to meet the needs of users, the video sites need a valid recommendation system urgently.Based on the background above, combining with the existing research results and actual demand, this paper carried out the following tasks:(1) According to the actual needs, a kind of system architecture for processing massive data is designed, this system is divided into three parts:recommendation engine, log system and display interface. These three parts combined offline data processing with online video recommending, so can respond to user needs as soon as possible and recommend videos to users that they may interest.(2) Studied the distributed collaborative filtering algorithm based on objects that Mahout has achieved and the source code, learned the principles and steps in its implementation. On basis of this, found its shortcomings and improved it, making the accuracy of the results better. Then through the calculation of open source data sets proved its feasibility.(3) Through comparing the algorithm of this paper used with random recommendation algorithm and the most popular video recommending algorithm, the results demonstrated that this algorithm had higher accuracy, and verified that selecting the appropriate length of the list in TopN recommendation is conductive to provide more accurate results to users.Finally, the accuracy and feasibility of the system is proved through a series of tests, and proved that the system is an efficient model to solve the current problem.