Research And Application Of Collaborative Filtering Algorithm Based On Second Order Hidden Markov Model

With the development of Internet in modern society,information appears in large quantities,which makes it more difficult for users to choose information.At this time,collaborative filtering as the first recommendation technology has gradually attracted people's attention.However,the existing algorithms inevitably have some limitations,such as data sparsity and over-dependence on scoring matrices.Based on the existing algorithms,this paper simulates user's scoring trajectory by using the double randomness between observation value and state in the second-order hidden Markov model,so as to improve the data sparsity of collaborative filtering algorithm.However,because of the no aftereffect of the hidden Markov model,it cannot express the dependence relationship between non-adjacent states.This model inherits the advantages of the first-order hidden Markov model.Therefore,this paper proposes an improved collaborative filtering algorithm based on the second-order hidden Markov model,as well as a clustering optimization algorithm for the efficiency of the algorithm and the scalability of users.Finally,a prototype system of information recommendation for college students is designed based on the proposed algorithm.The main research work of this paper is as follows:(1)An improved collaborative filtering algorithm based on second-order hidden Markov model(CF-2HMM)is proposed to solve the problems of data sparsity and over-dependence on scoring matrix in existing collaborative filtering algorithms.This algorithm uses the randomness of state transitions in the second-order hidden Markov model to simulate the change of users' interests.According to the users' scoring trajectory,it finds the candidate set with the highest scoring probability at the next moment,alleviates the data sparsity.The obtained probability and cosine similarity are weighted fused,and a new method of similarity calculation is proposed,which weakens the influence of scoring matrix on similarity calculation.The experimental results on MovieLens dataset show that the accuracy of CF-2HMM algorithm is 4.7 % higher than that of the improved collaborative filtering algorithm based on first-order hidden Markov model(CF-HMM),6.2% higher than that of the classical matrix decomposition-based collaborative filtering algorithm(SVD),and 8.9% higher than that of traditional collaborative filtering algorithm(CF).In order to achieve a balance between accuracy and recall rate.Hence,considering the F1 index,the CF-2HMM algorithm improves the F1 index by 5.9% compared with the CF-HMM algorithm,5.6% compared with the SVD algorithm,and 9.2% compared with the CF algorithm(2)Aiming at the scalability problem of CF-2HMM algorithm,which needs to train model parameters for a single user while users accumulate constantly,this paper proposes a user clustering algorithm(UCST)that fuse scoring trajectories.In this algorithm,users are clustered by combining the scoring trajectories of users,which optimizes the distance measurement of cluster samples and the selection of the initial cluster center.Then CF-2HMM algorithm is used for recommendation,which improves the scalability and computational efficiency of the recommendation algorithm.The experimental results on MovieLens data set show that the CF-2MHM algorithm after clustering is significantly shorter than the original CF-2HMM algorithm in running time,and considering the accuracy and efficiency of the algorithm,the optimal size of the user group is 20.(3)This paper designs and implements an information recommendation system.Aiming at the problem that it is difficult for college students to get the information they want to know quickly and accurately at present,a set of information recommendation system is designed and implemented based on the algorithm proposed in this paper.The system fully caters to the preferences and characteristics of College students,besides the recommendation hot spots tailored for users.In addition to the classification includes many information is closely related to college students.