Research On Chinese-English Neural Machine Translation Based On Joint Learning

Based on deep learning,neural machine translation technology has significantly improved the quality of machine translation.Nevertheless,many problems still remain to be solved.First,there is a problem of data sparsity in model training so that it is usually difficult to obtain sufficient parallel corpus for more practical tasks.In this way,the effect will not be as satisfactory as expected when the model is used for several translation tasks in low-resource areas.Secondly,the multi-layer cascaded network structure can transfer the representing information to the end layer but to some extent it may lose some useful messages that the middle layer has captured.Besides,the loss function set by the maximum likelihood estimation is only at the lexical level,therefore,the effect is poor in the task of sentence or text translation,and there will be inconsistencies between the training stage and the testing stage.Thirdly,in evaluation of the machine translation,traditional methods such as BLEU or NIST are mostly used as indicators to evaluate the translation simply from one side and thus are not covering the all-round information.Aiming at the above problems,some relevant researches have been carried out in this thesis as follows:To overcome the difficulty in acquiring the bilingual parallel corpus first,this thesis studies on the technology of corpus expansion and applies a joint learning method in corpus generation based on the EM algorithm.According to this method,the EM algorithm can be applied for training of the neural machine translation model Transformer,in which the corpus generation is the main task,and the Transformer's training is the auxiliary task.Meanwhile,the parallel corpus can be enriched by machine translation.Experiments verifies the validity of this method.Secondly,the neural machine translation model of layer aggregation and the adversarial training algorithm are proposed in this thesis due to the information loss in the middle layer caused by the depth model of multi-layer cascaded network and the shortcomings of the training method based on the maximum likelihood estimation.Along with joint learning,different models are practiced in this training process.On one hand,the Transformer is used as the baseline model to change its inner structure for a new model that strengthens the association between the middle layers by adding merged layers.On the other hand,the new model training is set as the main task,with sentence classification as the auxiliary one.Both of the two task models are adopted for joint training by adversarial training and intensive learning.Experiments verifies the validity of this method.Thirdly,the evaluation method of machine translation is also studied in this thesis to improve the current way of single indicator evaluation.By using some methods of machine learning,this thesis provides a comprehensive view on different aspects of the evaluation reference system in machine translation.Two useful methods are proposed from the idea of model-driven and the method of joint learning.On one hand,it uses the improved Transformer and SVM to build the translation evaluation model;on the other hand,it combines BERT,BiGRU with full interconnection networks to build a translation evaluation model.Through experiments,the validity of this method is verified as well.With Chinese-English machine translation as the research background,this thesis expands the parallel bilingual corpus,optimizes the Transformer model and training algorithms and improves the machine translation evaluation system through a joint training of the main and auxiliary task models.As the core idea of this thesis,joint learning is further expected to be applied to other natural language processing tasks in future research.