Design And Implementation Of Movie Recommendation System Based On Spark

With the rapid development of Internet technology,people are surrounded by a variety of data information,but among these massive data information,only a very small amount of data information is of interest to users,so how to find the useful information in the massive data information and push it to the targeted user is a problem to be studied by the recommendation system.Through the analysis of the recommendation system,it is found that the traditional recommendation algorithm has some shortcomings,and the emergence of the hybrid recommendation algorithm effectively alleviates these problems.Firstly,the cold start and data sparse problems of the traditional recommendation algorithm are mainly through the combination of statistical recommendation algorithm,collaborative filtering recommendation algorithm based on ALS(alternating least squares method)and content recommendation algorithm based on ElasticSearch.Algorithm to improve.Among them,the content-based recommendation algorithm can well alleviate the problem of cold start of the system,and the statistical recommendation algorithm has a good mitigation effect on the data sparse problem.Secondly,the problem that the recommendation system cannot update the recommendation results in real time or in near real time is mainly through real-time recommendation by adopting a model-based recommendation algorithm.Finally,through the Spark distributed platform to achieve the combination of offline and real-time hybrid recommendation algorithm,it can better deal with the problem of massive data recommendation.Finally,based on the design idea of the hybrid recommendation algorithm,a Sparkbased film recommendation system is designed and implemented.The system verifies the feasibility of the hybrid recommendation algorithm.