Wasserstein Learning Method For Self-attention Temporal Point Process Generation Model

In reality,there are a large number of asynchronous event sequence data in different fields,and the temporal point process is an effective mathematical tool to model the event sequence.At present,most researches describe the temporal point process by modeling the intensity function,but this method needs to consider the parameter form of the specific intensity function,which can only accurately describe the temporal point process in a specific field,which limits the generalization ability of the model to the actual data;In addition,due to the application of the deep learning method in the temporal point process,it is possible for the deep temporal point process method to explore the potential law of actual data.However,at present,the deep temporal point process mostly uses the recurrent neural network to model the intensity function.This method limits the ability of the model to express the real data,and the information of the intensity function is not necessary in many tasks of point process generation or event prediction,Moreover,the research on point process generation based on recurrent neural network can not capture the long-range dependence between event sequences,which will lead to unstable prediction performance.In order to solve these problems,this paper proposes a point process generation model(self-attention WGAN for Temporal Point Process,for short SGT)using multi-head self-attention mechanism.The model SGT can capture the long-range dependence between event sequences and improve the calculation efficiency of the model.Because the self-attention matrix describes the influence of historical events on current events,the model is more interpretable than the point process generation model based on recurrent neural network.The simulation data and real data experiments of point process show that the point process generation model based on self-attention mechanism has better performance than the generation model based on recurrent neural network.The innovations and research contents are as follows:1.In this paper,an intensity-free temporal point process generation model is proposed,which is also a time sequence point process learning method without likelihood estimation.No intensity function makes the model more generalized;No likelihood estimation makes the model reduce parameter iteration and improve the calculation efficiency of the model.2.This paper extends the self-attention mechanism to the research of asynchronous event sequence and point process generation mode,and optimizes the deep learning method of temporal point process combined with Wasserstein distance.The model uses Wasserstein distance to construct the loss function,which is convenient to measure the deviation between the model distribution and the real distribution,and uses the self-attention mechanism to describe the impact of historical events on the current events,so that the model is interpretable and has stronger generalization ability.3.The proposed model has expansibility.The comparative experiments on three simulation data sets and two real data sets show that in the absence of a priori information of intensity function,the deviation of QQ graph slope and empirical intensity deviation of this method are reduced by 35.125% and 24.200%respectively compared with the generation model of recurrent neural network and maximum likelihood model,which proves the effectiveness of the proposed model.