Research And Implementation Of Hybrid Movie Recommendation System Based On Spark Technology

The recommender system helps people get interesting information in a huge amount of data.However,the recommendation algorithm used in the traditional recommendation system has some defects.Therefore,in most of the actual recommendation system,they use hybrid recommendation techniques which mixed with a variety of recommendation algorithms,which perform better than the single recommendation algorithm in the recommendation effect.But with the development of the Internet lead to a sharp increase data size,single recommendation system performance worse and worse,recommendation system is an urgent need to find new solutions to make it to the data quantity of explosive growth environment can still maintain a good performance.At the same time,the development of information makes people have more choices.The recommendation system needs to provide the most suitable hybrid recommendation technology in different application scenarios to improve the recommendation quality of the recommendation system.According to the above problem,this thesis designed and implemented a mixed mode movie recommendation system based on Spark.The recommendation engine uses the hybrid recommendation technique that mixed with content-based and the alternate least squares method based on the co-clustering,and implement it on the Spark platform,so that the recommendation system in dealing with a huge amounts of data can still provide a good user experience.The main work of this thesis is:(1)Researched recommendation system,Spark platform and related technology,and put forward the main research contents of this topic in this thesis,elaborate on research significance,research background and research status quo of related technologies in this thesis.(2)In view of the cold start and sparse data of traditional recommendation algorithm,this thesis proposes a hybrid model recommendation algorithm which mixed with content-based recommendation algorithm and the Alternating Least Squares algorithm based on co-clustering.When facing new users or projects of the system,the supported recommendation engine by the hybrid recommendation algorithm will use content-based recommendation,which can alleviate the problem of system cold startup caused by collaborative filtering algorithm.For a user with a certain amount of data,the system will use ALS algorithm based on co-clustering,this algorithm can alleviate the data sparseness problem,using feedback and implicit feedback information at the same time,so that it can better simulate the user's interests.(3)Through the simulation experiment on the MovieLens data set,the recommendation effect of the hybrid model recommendation algorithm proposed in this thesis is tested by comparison test.So that to verify the effectiveness and availability.Experimental results show that the proposed algorithm performs better in recommendation performance and recommendation effect.(4)According to the proposed hybrid model recommendation algorithm,the design and implementation plan of the film recommendation system based on Spark platform is proposed,and the function,process and framework of the system are designed and explained in detail.The system is a Web service,the user reviews the movie information on the website,and the system recommends according to the user's classification,updating the data according to the user's score,so as to update the recommendation list,and also displaying the main interface of the system.