Research On Deep Point Process Modeling Based On Self-attention Mechanism

This paper studies the modeling of deep point process under self-attention mechanism.The occurrence regularity hidden in the asynchronous event data generated by human behavior and society can be used for internal cause analysis and prediction of future behavior,which has important research and practical value.Realistic scenarios include monitoring of criminal activities,dissemination of social behaviors,and behavioral interaction of students in online courses.Benefiting from the breakthrough of computer data warehouse technology and computer hardware,deep learning for mining human heterogeneous behavior data has gradually been included in the research category.As the main mathematical tool for modeling asynchronous event data,point process needs high fitting and prediction accuracy in practical application.The traditional statistical point process is limited by the limitation of expert knowledge and generalization ability.However,the current deep point process based on recurrent neural network and self-attention mechanism has its own deviation in fitting due to the characteristics of network structure,and the fitting result and prediction error are not satisfactory.Aiming at the above problems,this paper improves the network structure of deep point process to match the internal law of point process.Meanwhile,deep learning technology is used to correct the predicted value to improve the prediction accuracy.The model and prediction method presented in this paper show good results in simulation data and real data.The main work and specific innovations are as follows:(1)To solve the problem that the recurrent neural network lacks the ability of long-term memory and the self-attention mechanism does not match the recursive generation pattern of point process,Convolutional Enhancing Universal self-attention Hawkes Process with Time Relative(CUHP-TR)is proposed by adopting the universal self-attention mechanism as the core of network.The universal self-attention mechanism improves the self-attention mechanism by over consecutive revisions of the vector representations of each position(i.e.,over “depth”),so that the universal self-attention mechanism can take into account the global receptive field of the self-attention mechanism and the cyclic recursive characteristics of the recurrent neural network.This construction can better match the internal law of point process,so as to obtain better fitting performance.Furthermore,in order to make the universal selfattention mechanism better adapt to the intrinsic characteristics of point process,two adjustments are made to the weight matrix of the attention.First,the relative position coding on the time step is used in position coding to enhance the dependence of each position on the time step.Secondly,the attention weight matrix obtained by the self-attention mechanism is coupled with the weight of adjacent positions by a convolution kernel to adaptively adjust the interdependence relation of adjacent positions.Experiments show that these two mechanisms can improve the generalization ability of the universal self-attention mechanism on different data.(2)In view of the time prediction of deep point process caused by numerical calculation,the problem of large prediction deviation.In this paper,the conditional generative adversarial network is used to predict the first predicted value and modify its value to improve the prediction accuracy.The loss function uses Wasserstein distance of point process to constrain the training of conditional generation adversarial network.In addition,due to the simple network structure and small number of parameters,the initial prediction can be corrected in real time.The experimental results are satisfactory.