Research On Neural Machine Translation Based On Transformer Optimization

The development of science and technology has made cooperation and cultural exchanges among countries more and more frequent.In recent years,with the advent of the era of big data,human translation has long been unable to meet the great demand of the language translation industry.In contrast,machine translation has the advantages of high translation efficiency and low cost.Committing to the research of machine translation has become the current research trend.The mainstream model Transformer in neural machine translation is wholly based on the self-attention mechanism for translation operations,which has dramatically improved both in accuracy and speed,but there are still some problems.For example,without applying linguistic knowledge,it cannot combine the syntactic structure information of natural language for translation,which leads to issues such as mistranslation and omission.The positional embedding obtained by Transformer using a fixed formula cannot contain context information.At the same time,the autoregressive decoder does not fully utilize the context information when decoding from left to right,and there will be exposure bias problem.Therefore,this paper focuses on the existing issues in the neural machine translation model Transformer,and carries out the following three research tasks:First of all,to solve the problems of mistranslation,omission,and over-translation with neural machine translation,this paper introduces dependency syntax to construct a dependency matrix,and adds words with a high degree of dependence with the current word into the sliding window of CBOW(Continuous Bag-of-Words,CBOW)to predict the current word together with the context word.Due to the different degrees of association between words,different words should have different prediction contributions to the current word.This paper proposes two methods to obtain the prediction contribution,path length calculation and self-attention mechanism,to construct a source language sequence containing syntactic structure information so that the model can better understand semantic information.Secondly,aiming at the problem that the positional embedding obtained by the Transformer model using a fixed formula does not contain context information,this paper introduces Bi-LSTM(Bi-Directional Long Short-Term Memory,Bi-LSTM)and Tree-LSTM(Tree-Structured Long Short-Term Memory,Tree-LSTM)obtains source language sequences containing context location information from horizontal and vertical direction training respectively,and introduces the self-attention mechanism into Tree-LSTM to determine the degree of prediction contribution,retaining relative positional information between words.Finally,since Transformer uses the autoregressive decoder,decoding from left to right does not fully use context information,and there will be exposure bias.This paper proposes a syntax-aware bidirectional decoding neural machine translation model,using a forward decoder and a backward decoder for bidirectional decoding,so the generated decoding results can contain context information.At the same time,the dependency syntax is integrated to generate target language sentences with syntax guidance.Finally,the teacher-forcing mechanism is optimized to balance the difference in the decoding environment between the teacher-forcing training stage and the autoregressive test stage to alleviate the exposure bias problem.This paper aims to optimize the Transformer model to improve the translation performance of the neural machine translation model.The experimental results show that the three methods have achieved corresponding performance improvements.