A Study Of Neural Machine Translation With Limited Parallel Corpus

In recent years,deep learning has rapidly penetrated into all fields of natural language processing,including the field of machine translation.Deep learning techniques have brought great changes to machine translation,making it no longer necessary to face the problems such as feature design in traditional statistical machine translation.In particular,neural machine translation(NMT)methods which use deep learning techniques have made rapid progress in recent years,and have achieved full improvements in translation performance,thus becoming the core technology in machine translation.However,the performance of NMT models is highly dependent on the amount and quality of parallel corpus.In many practical languages and domains,the amount of high quality parallel corpus is very limited,thus restricting the performance of NMT models.To deal with the problems above,this thesis aims to explore how to use less parallel corpus and make full use of other resources which are easier to obtain to help the training of NMT models.Specifically,under the situation of limited parallel corpus,this thesis focuses on exploiting and utilizing other related resources,and carries out the following three aspects of research work:First of all,from the perspective of effective development and utilization of monolingual corpus,this thesis studies the problem of training NMT models using both parallel corpus and monolingual corpus,and proposes a new semi-supervised NMT method.The main idea of this method is to estimate the likelihood of the target language side monolingual corpus by using the law of total probability and then maximize the likelihood of parallel corpus and monolingual corpus at the same time.Furthermore,in order to solve the problem that the search space is too large when calculating the expected term in the law of total probability,the importance sampling method is adopted to avoid enumerating all possible source language sentences and ensure the validity of the objective function.The experimental results on the two translation tasks of English-to-French and German-to-English confirm the superiority of the proposed method compared with other semi-supervised NMT methods.Then,we propose to introduce a data related regularization term and apply it to monolingual corpus to help the training of NMT model.Specifically,the law of total probability describes the relationship between marginal distribution and conditional distribution.That is,the translation model and the language model are linked by the probability relation.However,in practice,the NMT models trained on parallel corpus can not guarantee that the law of total probability can be satisfied on any data.Therefore,we propose to incorporate the law of total probability into the training objective of the NMT model as a regularization term,thus explicitly emphasizing the probabilistic relationship between the models,so that the process of model learning can proceed in the right direction.Among them,the added regularization term can be applied to any data including monolingual corpus,that is,it is a data-related regularization term.Finally,experimental results on English-to-French and German-to-English translation tasks demonstrate the effectiveness of the proposed method.Finally,we further consider the shortage of parallel corpus in NMT.In this thesis,we study the problem of building NMT systems with no parallel corpus(i.e.,zero-resource NMT).Specifically,we use additional multimodal corpus to construct NMT systems,and transform the zero-resource multimodal NMT task into a reinforcement learning problem.Further,we introduce a sentence-level supervision signal,that is,we estimate the correlation between source language sentence and target language sentence by the corresponding image,so as to evaluate the quality of target language sentence.Then,two different reward functions are designed to guide the learning process of the model.Finally,experimental results on three translation tasks of three datasets verify the effectiveness of the proposed reinforcement learning method.