Research On Chinese Classical Poetry Generation From Images Based On Deep Learning

Chinese classical poetry is an artistic expression of the Chinese nation's thought,culture,spirit and emotion for more than two thousand years.With the development of artificial intelligence and deep learning technology,the research on Chinese classical poetry automatic generation has received considerable attention in recent years.And a large number of online computing systems have emerged.Most of the existing studies focus on the poem generation with given keywords or text sequences.Inspired by the latest advances in image caption,it is an interesting and challenging task to generate poetry from image input to realize the digital experience of traditional culture.As a cross-modal problem,this task involves multiple challenges: obtaining poetic information from images,generating poems based on image information,and also considering the literary effects of the output poems.Based on the above challenges,this thesis proposes two methods for generating poems based on the complexity of image input.At the same time,the problem of weakening the semantic coherence of verses in the existing generation model is optimized.The main work of this thesis is as follows:(1)Propose a Chinese classical poetry generation method suitable for singleobjective image input,and establish a single-objective image-poetry generation model.The model connects images with poems by controlling the first word of the line.There are two modules: image keyword extraction and keyword-based poetry generation.The model obtains the target information of the image through the image feature extractor,uses the target information as the writing outline of the poem,and then generates the ancient poem line by line based on the char-RNN.Experiments were designed to analyze the effects of different layers of models on the validity of the experimental results.The results show that the classical poems generated by the three-layer RNN structure in this paper have higher recognition.At the same time,the model performs better on the five-character poems than the seven-character poems.(2)Aiming at the problem that the single-objective image-poetry generation model can only recognize single objective image,an image-poetry generation method suitable for complex image input is proposed,and a scene-based model for image-poetry generation is established.Considering the importance of scene information in classical poems creation,the scene information is identified in the image processing module.The model obtains the target information and scene information of the image through two image feature extractors,and extends the information to generate four subject words to guide the generation of the following poems.In the poetry generation module,the poetry is generated line by line using the Encoding-Decoding model based on the attention mechanism,and the topic words are used as external input.Each topic word generates a line of poems.At the same time,in view of the weak coherence of the poetry in existing research,the current line of the poetry is only related to the first two lines of poetry.The Turing test was designed to evaluate the effectiveness of the model.(3)A comparative experiment was designed to evaluate the two models.The contrast experiments are carried out from two aspects: one is the comparative analysis of the results of poetry generation;the other is the relativity of the image-poetry model.Perplexity and manual scoring were used as evaluation indicators.The poems generated by the two model are compared with the SMT model,the RNNPG model and the PPG model.The results show that compared with char-RNN structure,the Encoding-decoding framework achieves better results in poetry generation.At the same time,compared to the selection of all the previous information,choosing the first two lines of poetry as reference is more conducive to the better poetry generation.Two contrast experiments were designed to evaluate the correlation between the two models.The experimental results show that the first model works better when the input image information is relatively single.When the image is richer,the poems generated by the scene-based model through two image feature extractors are more suitable for image content expression.