Design And Implementation Movie System Of Hybrid Recommendation Algorithm Based On Spark Platform

At present,we have entered the era of big data.The popularization and application of the network have also generated a large amount of data information.It is an urgent problem that Internet practitioners need to solve how to screen information and quickly find the valuable information they need.In order to screen out valuable and correct information,recommendation system began to develop gradually.The recommendation system records and collects the historical behavior information of users,analyzes the similarity between users and users,items and items,and then screens out items that users may be interested in,which has become a kind of information screening technology with strong initiative and high intelligence.At present,recommendation system is increasingly mature and has been popularized in mobile network applications,such as the most popular social software,e-commerce,audio and video fields.Recommendation system should be able to fully understand what the user is most interested in,so as to grasp his needs,and can quickly find valuable information from a large amount of data.If the recommendation system is combined with the distributed computing platform based on big data,the data analysis ability will be more powerful and the processing efficiency will be higher.This is also the functional positioning of the recommendation system,but also the specific application of big data technology in practice.After Hadoop platform,Spark platform emerged,which is a memory-based distributed computing system.Compared with Map Reduce model,the design idea is more advanced,and the concept of iterative parallelization is introduced,which is superior in performance and speed.The research of this paper is divided into the following parts:(1)Build data warehouse based on Spark environment to be applied in recommendation application engine,provide interface for the call of distributed data,and store data,items,user information and related evaluation information obtained by offline calculation of recommendation system.(2)The implementation of three recommendation algorithms based on Spark platform.The recommendation system can be divided into online and offline computing methods according to its computing time,difficulty,update frequency,etc.The accuracy and response speed of the Movie Lens data set are used to test the accuracy and response speed of the recommendation engine.This paper makes a comprehensive theoretical analysis of Spark distributed computing platform,several mainstream recommendation algorithms and related application scenarios.Based on the application methods and usage scenarios of various algorithms,a film recommendation system based on hybrid recommendation algorithm is designed and specific functions are realized.In order to solve the cold start problem between users and items,the system will use different recommendation algorithms to recommend interested items to users according to the current situation of information stock.In addition,the hybrid algorithm is able to deal with huge amounts of data and information system to meet the requirements of the users of the system response speed,provides from front-end applications and back-end services building,algorithm design and implementation,deployment platform all-round closed-loop business implementation,such as basic to achieve the expected goal,for the design of other recommendation system based on the Spark platform to provide a theoretical basis and technical support.