Design And Implementation Of Distributed Recommendation Algorithm Based On Implicit Feedback

With the rapid development of the Internet,more and more channels of information are generated and the information is generated faster and faster,which leads to an overall explosive growth of information.Recommendation System accords to the user's historical behavior information to mine the user's potential interest and quickly help users get the information they need from the vast amount of information.It has become an effective way to solve the information overload.In addition,as the amount of data dramatically increases,the computational efficiency of single computational unit is limited.As a result,the actual calculation time of the recommendation process is too long,which becomes a major bottleneck that limits the recommendation effect.Therefore,massive data takes new requirements for the implementation of the recommendation system.This paper focuses on the problem of dealing with the cold start of recommendation system and the distributed computing to solve the system operating efficiency.We aim to solve this problem in the situation where we can only access to a set of ratings of items by users and no contextual information is available neither about users nor items.Most of the existing studies usually construct static features of users and items,and then recommend with multi-armed bandits strategy.The disadvantage of them is no further extraction and updating of features,thus limiting the performance of recommendation.In this paper,we introduce a Multi-Armed Bandit Recommender Algorithm with Matrix Factorization approach to solve the feature extraction problems.More specifically,we use matrix factorization algorithm to update features based on the error between the real ratings and the predicted ratings of items by users,which is followed by recommending with multi-armed bandits strategy based on the new features.The new algorithm conveniently combine multi-armed bandits strategy and matrix factorization and it has high generality and scalability.In order to improve the computational efficiency of the system and increase its scalability,we use Spark distributed computing framework to parallelize the algorithm and solve the computational bottleneck of a single processing unit.The experiments of the proposed algorithm are carried out under different number of nodes using Spark clusters.This paper has analyzed the recommendation accuracy using proposed algorithm and compared the efficiency between distributed processing and single machine processing.The experimental results show that the algorithm proposed in this paper improve the deal with cold start problems in terms of cumulative error and click-through rate.At the same time,the Spark computing framework is used to parallelize the algorithm.Experiments on the dataset used in this paper show that the computational efficiency of the algorithm is greatly improved without reducing the accuracy of the algorithm.