Design And Implementation Of A Real-Time Music Recommender System Based On Collaborative Online Topic Model

In the bigdata and cloud era,where the enormous online music libraries are updated quickly,and it’s often hard for users to describe their interests with specific keywords,users of music platforms are increasingly dependent on music recommendation systems.A good music recommendation system can greatly improve the experience and loyalty of users,help music sites and APPs to occupy the market.Due to the principle of the algorithm,most of the traditional offline recommendation system constructs the initial model from static data,and then must retrain the model on all data periodically with the arrival of new data blocks,resulting in huge computational resource consumption and cannot capture recent changes of user’s interest and give them latest recommendation results.With the increasing speed of interaction data in music websites,the problem of offline recommendation algorithms is becoming more and more prominent,and their effect is no longer comparable to that of simple online algorithms.This paper designs and implements a system that can recommend and play music and monitor trending of tracks,and uses Flume and Kafka to collect logs of user behaviors in real time,and use Spark Streaming and Redis to maintain statistical metrics like play counts of tracks in time window,the number of playing users of tracks in one day,and personal history of music listening for users and other important statistics,can efficiently support the large-amount of query.The recommendation module of this paper uses Spark Streaming to implement the incremental matrix decomposition algorithm on the data stream,the Collaborative Online Topic Modeling recommendation algorithm and the improved item-based collaborative filtering and application in the system,and use the user behavior log collected by Kafka as the input source of the data stream.Online Collaborative Topic model can use the implicit feedback data of listening records of users,and the label of the song and other auxiliary information,with the new listening record and the arrival of the new song label,update the recommendation model in real-time,to ensure the accuracy and efficiency of recommendation.For collaborative filtering based on items,this paper introduces the item label information and song lyrics information when calculating the similarity,and dynamically maintains the similarity of the items based on the user,which improves the recommendation accuracy and the efficiency of the algorithm.This paper conducted a comprehensive functional test and performance test on the system,and tested the recommendation engine with the real-world data set,analyzed the topics found by the collaborative topic model,and illustrated the effectiveness of tags in describing music as auxiliary information for recommendation.The test results show that the functional modules of the system are running normally with good performance.Compare to traditional offline recommendation systems,This system can capture the change of user’s interest in real time,gives the user a better recommendation result,at the same time can monitor the music popularity trend in real time,makes it easier for user to follow the trend or the website managers to carry out targeted music promotion.