Text Generation Algorithm Based On Keyword Semantic Control

With the development and application of deep learning,a lot of natural language processing tasks have made great progress.The study of text generation has made a great achievement,especially in the fields of machine translation,conversation generation and so on.With the application of text generation model,the semantic control of the text generation has become an important problem.At present,the text generation has some good effects on the emotional control and tense control,but it is unsuccessful on the semantic control.The vital problems are the encoding of keywords semantic information and how do keywords semantic information used to the text generated process.Aiming to address the aforementioned problems,this thesis will make some research,including the following:1.The encoder-decoder framework applied for the text generation problem based on the keywords semantic control.The input of this thesis'task was only keywords sets.keywords set was the input of encoder part,then the semantic representation was got from encoder.The text generation process was the decoder process.The semantic information introduced through attention mechanism.At the same time,this thesis had designed the evaluation method K-BLEU for this task,which can both evaluate the statement fluency and keywords semantic relevance.2.FM-encoder model put forward for the problem of keywords semantic representation.Referring to the idea of Factorization Machines(FM),the relationship between keywords should be extracted,thus can represent the semantic information more exactly and can make up for the weakness of general semantic encoding model at present.Recurrent Neural Network(RNN)model is suitable to represent the semantics of time series data,but the keywords set is not time series data.RNN will make the relationship of close keywords exaggerate and make the relationship between keywords far apart from each other weaken.Convolutional Neural Network(CNN)model has the advantages of capturing local features,but it cannot directly express the relationship features of keywords far apart.Therefore,FM-encoder model was chosen as the keyword semantic encoding model.3.Aiming at the problem of how to use keywords semantics in text generation,this thesis put forward the Weighted Additive Attention(WA-Attention).The attention model improved to introduce semantic information into the decoder process of text generation.The model was similar to the encoder-decoder model which applied on machine translation.The result of encoder was the semantic vector that obtained by FM-encoder model.And the decoder process was the process of generation text through the attention mechanism word-by-word.A new attention implementation method,WA-Attention,was designed,which is similar to the traditional concat attention and the weighted addition was used instead of connection.The two parts that added up were the encoding of the generated text and the semantic encoding of keywords.Adjusting the weight can be understood as adjusting the control intensity of semantic information.The practice shown that the system in this thesis is able to generate smooth news title,and it is related with keywords semantics.The system achieved the goals.