Research On Interactive Recommendation Method Based On Reinforcement Learning

In recent years,with the popularity of smart interactive devices such as cell phones and tablets,Interactive Recommender System(IRS)with better user experience has been more and more widely used.Due to its nature of learning from dynamic interactions and planning for long-run performance,Reinforcement Learning(RL)recently has received much attention in IRSs.However,most of these methods face the large-scale discrete action space problem and data sparsity problem.For a typical Top-K list recommendation based RL,the size of its action space is an arrangement of K items selected from all items.When the number of items is large,the scale of action space is huge which will make the decision inefficient.Moreover,when the interaction data is sparse,the recommendation model cannot be trained well.The performance of most recommendation systems is affected by the data sparsity problem,and RL-based IRS is no exception.To solve the abovementioned two problems,we focus on the methods of interactive recommendation with knowledge enhancement.The efficiency of the RL-based model is improved by constructing an action candidate set and designing the conversion mode from continuous action to discrete action.Through probabilistic matrix factorization knowledge enhancement,simple and effective text knowledge enhancement,and light self-supervised graph representation learning enhancement,the goal of alleviating the problem of data sparsity is achieved.Specifically:Firstly,based on the assumption that user preferences can be represented by a group of items,an algorithm for constructing action candidate sets is designed.Combined with the action candidate set,environment simulator,and other main components,we propose a Reinforcement learning framework for Interactive Recommendation(RIR)considering efficiency.Correspondingly modifying the Double Deep Q-network(DDQN)algorithm,we implement this framework.Meanwhile,we use the representation learned by probabilistic matrix factorization to enhance the candidate set construction and model training,and propose RMIR-DDQN(Reinforcement learning framework using Matrix factorization representation for Interactive Recommendation,RMIR).The decision complexity of this method is greatly reduced.The performance of RMIR-DDQN on some public datasets is better than the mainstream RL-based recommendations.Secondly,based on the assumption that user interest,user characteristics and item characteristics can be expressed in a same feature space,we design a more reasonable conversion mode from continuous action to discrete action,and implement an IRS method with Deep Deterministic Policy Gradient(DDPG)algorithm.Compared with the RMIR-DDQN method,which needs to calculate multiple Q values,this method only calculates one policy vector,which further improves the efficiency of the RL-based method.At the same time,to alleviate the problem of data sparsity,this study designs a simple and effective method to combine text information for RIR,and proposes the TRSIR-DDPG(Text-based deep Reinforcement learning framework using Sum-average word representation for Interactive Recommendation,TRSIR)method.Experiments on some public datasets containing text information show that TRGIR-DDPG achieves state-of-the-art performance over several baselines,in a time-efficient manner.Finally,considering that the TRSIR only makes use of the first-order relationship and ignores the high-order relationship in the process of introducing text information.In this paper,interactive and textual entities are modeled as a heterogeneous graph,and a light graph convolution text combination method is proposed to mine high-order relationships between nodes.To verify its effectiveness,we further propose a recommendation method that combines graph representation learning and neural matching function learning.Experiments show that the performance of this method is superior to other existing methods.Aiming at the goal of distinguishing between individual representations in RIR framework,combining the above graph learning method,self-supervised learning and triplet loss function,we propose a Text-based deep Reinforcement learning framework using self-supervised Graph representation for Interactive Recommendation(TRGIR),and implements this framework using DDQN and DDPG algorithms.Experiments on public datasets show that the TRGIR-DDPG method has significant performance improvement over the SOTA methods.