A Study On The Deep Joint Learning For Language Recognition

With the increasing of computing resources and training data,language recognition technologies based on deep neural network have gradually become the mainstream of language recognition system.However,the current academic research on language recognition system based on deep neural network is mostly limited to the method with single acoustic feature or single task training,and the research on joint learning of language recognition system based on deep neural network is inadequate.In addition,language recognition systems often encounter complex language recognition scenarios such as short speech,cross channel,and open-set recognition,which require higher performance of language recognition system.Meanwhile,there are also special language recognition tasks such as dialect recognition in practical usage.To solve the above problems,with the data sets of the Oriental Language Recognition Challenge,this thesis studies the joint learning method of language recognition system based on deep neural network.The main work and contributions of this thesis are listed as follows.(1)Based on the deep joint learning of multiple acoustic features,this thesis proposes a multi-loss constraint of adaptive weights,which makes the joint learning of multiple acoustic features more flexible;in addition,this thesis presents a canonical correlation analysis constraint,which is used to strengthen the correlation between the outputs from multiple networks of acoustic features,so as to improve the performance and robustness of the language recognition system.(2)Based on the multi-task joint learning of language-phoneme,this thesis investigates different implementation strategies of multi-task joint learning,such as multi-task joint learning for strengthening phonetic information,multi-task joint learning for restraining phonetic information and a combined learning structure;this thesis also studies a language-phoneme embedding joint learning network,which can extract embeddings of language and phoneme simultaneously.(3)With data sets of the Oriental Language Recognition Challenge,this thesis systematically analyzes the performance of baseline language recognition system under different testing conditions,and the influence of different testing conditions on the performance of language recognition.This thesis also analyzes the performance of language recognition system based on multiple acoustic feature joint learning methods and multiple speech task joint learning methods.