| The rapid development of neural machine translation technologies has greatly improved the machine translation raw output.In spite of this,current machine translation quality is still not up to the level of professional from-scratch translation.In order to further improve the machine translation output quality,post-editing by means of correcting machine translation has become a popular approach in translation industry.However,post-editing process is quite different from traditional translation from scratch.Post-editors have to work on the machine translation raw output with varying qualities.Although post-editing is widely used in translation industry because of its fast,efficient and cost-saving features,whether post-editing is applicable to translation trainees is still an unresolved problem.Usability is an important concept in human-computer interaction research.It refers to the extent to which a system,product or service can be used by specified users to achieve specified goals with effectiveness,efficiency and satisfaction in a specified context of use.Usability theory emphasizes the user-centered approach.Different users may need different usability evaluation criteria for different purposes with the same technology,product or service.The usability assessment of neural machine translation post-editing in educational settings is to analyze trainees' efficiency,effectiveness and satisfaction in solving problems.However,prior research on machine translation postediting mostly focuses on the comparison between post-editing and translation from scratch,as well as different machine translation systems from the perspective of translation industry.There lacks systematic research on the usability analysis of the post-editing use for translation trainees in educational settings.Several questions need to be further clarified,such as the difference between trainees' cognitive effort and their performance in post editing and translation from scratch,their problem-solving ability in post-editing condition,as well as the curriculum development of post-editing.Based on the theoretical framework of usability assessment,this study proposes a usability assessment framework of neural machine translation post-editing for translation trainees.The proposed framework takes trainees' translation efficiency,translation effectiveness and their satisfaction in using post-editing as the usability assessment parameters.The measurement indicators of translation efficiency include translation time,physical exertion and cognitive effort.The measurement indicators of translation effectiveness include translation accuracy,translation fluency and problemsolving strategies.Satisfaction measures include trainees' perceived ease of use and perceived usefulness.The study takes MTI students and undergraduate language students as the research subjects.Three English-Chinese scientific and technical texts were selected: text H for translation from scratch,text P1 and P2 for post-editing.The translation direction is from English to Chinese.Text reading difficulty index and manual review are combined to control the source text difficulties.Text H and P1 are compared with similar source text complexities,while P2 is selected with more source text complexities than P1.The study adopts a mixed-methods approach to collect the usability data by means of keyboard logging,eye-tracking,questionnaires and interviews.R language statistical packages are used as the analysis tool.In order to clarify whether and how neural machine translation post-editing is useful to translation trainees,under the guidance of usability assessment framework specific questions have been addressed as follows.(1)In terms of translation efficiency,is post-editing more time and effort-efficient than translation from scratch for translation trainees,and what is the efficiency difference between P1 and P2?(2)In terms of translation effectiveness,is translation trainees' post-editing quality higher than from-scratch translation,and what is the error correction distribution difference between P1 and P2?(3)What are translation trainees' problem-solving abilities and how does translation experience work in the problem-solving process of translation from scratch and post-editing?(4)What are translation trainees' attitudes towards using machine translation and post-editing?The obtained findings of the present study demonstrate that:(1)It is more time-and labor-saving,as well as cognitive-relieving for translation trainees in processing post-editing tasks than in processing from-scratch translation;undergraduate language students need more time than MTI students in from-scratch translation while the two groups of trainees demonstrate no significant difference in post-editing tasks irrespective of task complexities;undergraduate language students perceive more mental load and are more in agreement on this perception than MTI students in from-scratch translation.(2)With respect to translation accuracy and fluency,translation trainees' postediting performance significantly outperforms their performance in from-scratch translation;in from-scratch translation the performance of undergraduate language students is significantly poorer than MTI students.However,there is no significant difference between the two groups of trainees in post-editing tasks.MTI students' translation experience helps significantly in from-scratch translation but not as so much as in post-editing mode.In terms of translation error types,mistranslation is the most frequent error type both in from-scratch translation and post-editing task.Cohesion is the significant error type in from-scratch translation,while word order is the significant error type in post-editing task.The error correction number in P1 is lower than that in P2,though P2 is with much longer source text complexity than P1,suggesting error type may be an essential indicator for post-editing difficulty.(3)At the problem-solving phases,planning is observed with a positive and significant relationship with translation from scratch performance,while monitoring and revision are surprisingly proved to have mixed relationships in both translation from scratch and post-editing.In general,translation trainees demonstrate poor metacognitive problem-solving abilities.(4)Translation trainees are generally positive towards using neural machine translation post-editing.They show an eager desire in learning post-editing and relevant knowledge.The results of the study show that although post-editing can improve translation speed,translation quality and reduce cognitive effort,trainees' problem-solving ability is unsatisfactory in translation from scratch and post-editing.In other words,compared with translation from scratch,the improved post-editing quality is mainly attributed to the help of neural machine translation raw output quality rather than trainees' own translation ability.Therefore,to improve trainees' translation competence should still be the most important thing for translation education in the future.Post-editing performance is often affected by many factors,such as post-editors' profiles,machine translation raw output quality,language pairs involved and translation materials.Postediting courses can be offered as an elective course for MTI students under the premise of scientific and rational design.In addition,post-editing training should pay more attention to machine translation error types.In view of the wide use of neural machine translation in translation industry,the adoption and effective use of post-editing have become an unavoidable topic in the field of translation education.Although this study has reached some preliminary conclusions on the usability of neural machine translation post-editing for translation trainees,it has some limitations due to the sample selection,translation texts and experimental ecological validity.Nevertheless,to reasonably use neural machine translation is a rather complex issue which requires more relevant research.Focuses of future research will be the exploration of neural machine translation post-editing usability assessment with different translated text types and post-editing practice in classroom settings.This study is innovative in terms of research content,method and theoretical application.Regarding the research content,neural machine translation is a new concept in translation technological education research.The usability assessment of neural machine translation post-editing is mainly based on three comparisons: comparison between translation from scratch and post-editing with similar source text complexities;comparison between post-editing with different source text complexities;and comparison between MTI students and language undergraduates.Based on the existing studies,this study identifies the perception and performance differences in the process of translation from scratch and post-editing,as well as the quality improvement sources of post-editing(mainly depending on the machine translation raw output quality or trainees' own problem-solving ability).The research results can provide a pedagogical reference for universities and administrative agencies to judge the machine translation post-editing usability.Methodologically,traditional usability assessment is usually based on questionnaires or process recording tools.In this study a mixed method of eye tracking,keyboard recording,questionnaire survey and interview is used to ensure the validity of the results.In terms of theoretical application,on the basis of machine translation characteristics and the features of translation trainees,the proposed usability evaluation framework of neural machine translation post-editing is an exploration of bringing usability assessment theory from human-computer interaction research into educational settings.In the proposed framework,the integration of problem-solving dimension is the usability assessment consideration of emerging technologies in educational scenes.The refined measurement indicators can provide a reference for the usability study of related technologies in the era of artificial intelligence. |
PDF Full Text Request