Oral English Learning And Assessment For SELL-Corpus And VR Environments

English learning has always been a hot topic in China.In addition to the school's setting up English as a compulsory course,various English training institutions are also emerging.However,because China is not a native English-speaking country,and there is a lack of speaking training compared to listening,reading,and writing,Chinese English learners generally have poor spoken proficiency.In recent years,with the popularity of virtual reality devices,more and more research has begun to try to use virtual reality(VR)to create an immersive English learning environment.Therefore,this paper combines virtual reality and artificial intelligence(AI)technology to create an immersive speaking English learning system.Moreover,in order to make the learning system more in line with the needs of Chinese English learners,we constructed a SELL corpus designed for Chinese English learners,and based on the corpus,a speech recognition and pronunciation evaluation system was established.The main research contents of the thesis are as follows:First of all,this paper designs and implements a spoken language training system for Chinese English learners.The system architecture is highly flexible,and learners can easily customize personalized courses by creating profiles to meet the needs of different learners.Secondly,in order to enable the system to support PC,mobile phone and various mainstream VR devices at the same time,make the interface interaction more natural and improve the user's immersive experience,this paper builds a highly scalable,multidevice human-computer interaction system.Moreover,the system supports multiperson and multi-device synchronization,providing a platform for learners of various device needs to interact and communicate.Subsequently,in order to adapt to Chinese English learners and localization,this paper designs and constructs the SELL corpus.Considering the influence of Chinese dialects on English pronunciation,its recorders cover the seven major dialect areas in China.As the basic data of speech recognition,pronunciation evaluation and pronunciation error detection in the system,SELL corpus can better adapt to the needs of Chinese English learners.Moreover,this paper studies the speech recognition of SELL corpus and trains a speech recognition system based on Kaldi.In addition,in view of the lack of corpus in some dialects,a multi-task English accent recognition migration network method is proposed.Try to improve the recognition rate through multi-task migration learning under a small amount of accent corpus.Finally,this paper designs a complete intelligent conversation system driven by voice,including core AI technologies such as speech recognition,speech evaluation and intelligent dialogue.The conversation system allows the learner and the AI virtual character to perform natural session interactions,simulate various application scenarios for training,and improve the learner's oral communication and English application ability.And the system will automatically record the learner's learning habits and progress,and learn to track.