Sequential Recommendation Based On Knowledge Distillation And Multi-head Attention

With the development of information technology,people have entered the age of information overload.The recommendation system is an important tool for solving information overload,which matches users and information reasonably,helps users find valuable information,and shows the information in front of users who are interested in it.The recommendation system uses data to infer current interest in users,generate the corresponding list of recommended items to users,provide personalized recommendation pages for different users,improve user experience,help them find content that they may be interested in when the user doesn’t have a definite purpose,and apply widely to various types of apps.The effectiveness of feature extraction and representation of recommendation model plays a important role in the final recommendation results.The feature extraction method of previous recommended models ignored the time sequence between features and was limited to two-dimensional feature space,and the feature fusion method couldn’t distinguish the importance of different features.In this paper,a sequential recommendation model based on semantic perception is proposed,and the semantic sensory neural network(SPNN)is based on semantic perception.SPNN uses sequential coding to represent the time information of different input data,and to distinguish the semantic information between different temporal features through the semantic perceptual layer,to capture the semantic information between different temporal features,and to obtain the corresponding semantic encoding,and finally make the recommendation prediction based on the semantic encoding.Aiming at the problems that knowledge distillation method ignores the features knowledge of intermediate network layer,knowledge loss caused by knowledge dimension transformation,and the complexity of distillation process is too expensive.In this paper,a kind of intermediate layer features distillation method is proposed,and the key knowledge of the intermediate metwork distillation layer is obtained through the knowledge detector,and then the knowledge is embedded in it,and finally,the knowledge distillation loss is calculated according to the knowledge embedded vector.Combining feature distillation and SPNN,a sequential recommendation method integration semantic awareness is proposed to solve the feature mismatch problem in cold startup.The effectiveness of the method is verified by comparing experiments in real data sets.