Research On Mongolian Chinese Neural Machine Translation Based On Lightweight Transformer

With the continuous progress of neural machine translation technology,the quality of Mongolian to Chinese translation has been significantly improved,which also makes the communication between nations more convenient and smooth.At present,most of the models used for Mongolian Chinese neural machine translation are Transformer models.Due to the structural characteristics of Transformer,overfitting is easy to occur in the process of low resource corpus training;At the same time,Mongolian language has complex grammatical structure and morphological changes,which makes it have many unknown words in the process of translation.These two reasons together lead to the lower quality of Mongolian Chinese neural machine translation than other languages.To address this issue,this article introduces the concept of lightweight and constructs a Mongolian Chinese translation model based on lightweight Transformer to alleviate model redundancy and improve the translation quality of the model.The main work of this article is as follows:(1)Corpus expansion.By crawling Mongolian language corpora from major news websites,a Mongolian Chinese translation model is used to translate the crawled Mongolian language into Chinese,and a pseudo parallel corpus is constructed.Then,the Chinese language of the pseudo parallel corpus is scored using the N-gram algorithm through a publicly available pre trained model,and the data that exceeds the threshold is retained to construct a parallel corpus.(2)Reduce the number of unlisted words.At the data level,a combination of synonym based replacement and mapping based replacement of unlisted words in the Mongolian Chinese dictionary is adopted to reduce the number of unlisted words in the Mongolian Chinese bilingual language.Through experimental verification,it is found that the use of unlisted word replacement method has better translation quality.(3)Construct and implement Mongolian Chinese neural machine translation model based on lightweight Transformer.To reduce redundancy from the model level,the attention mechanism is divided into two parts.One part uses the attention mechanism to capture global information,and the other part uses dynamic convolution to capture local information.The captured dual branches are fused by means of a gating mechanism to replace the attention mechanism and feedforward neural network,so that the model has fewer parameters and higher ability to capture information.The experimental results show that the lightweight Transformer model is better than the traditional Transformer model in Mongolian Chinese neural machine translation.(4)Integrate cross layer parameter sharing strategies.To further reduce the redundancy of the model,a cross layer parameter sharing strategy is adopted,which involves grouping and sharing parameters between layers.At the same time,based on the characteristics of Transformer model training,a two point grouping strategy is proposed,and three sets of models are designed based on this to verify the effectiveness of the grouping strategy.The experimental results show that all three sets of models are superior to those without using grouping strategies,indicating that cross layer parameter sharing is effective,and the proposed two point grouping strategy is effective.The research shows that the Mongolian Chinese neural machine translation model based on lightweight Transformer constructed in this paper has good translation performance.At the same time,this article also compares the impact of cross layer parameter sharing strategy on the quality of translation models,and the results show that models with cross layer parameter sharing strategy can achieve better translation results.Therefore,the research work in this article has certain application value.