Research On Controller Speech Fatigue Detection Based On Deep Learning

With the steady progress of China’s civil aviation power cause and the increase of controlled airspace flow,air traffic controllers are responsible for ensuring the safe operation of aircraft.However,the long-term high-intensity and circadian rhythm disorder can easily lead to fatigue of controllers,which poses great safety risks to civil aviation transportation.Therefore,detecting the fatigue state of controllers during their work is of great significance for ensuring air traffic safety and maintaining air traffic order.As controllers rely on radio to communicate instructions when directing aircraft operations,and voice signals carry a lot of human fatigue information.In order to reduce the safety hazards brought by controller fatigue to civil aviation transportation,this thesis combines deep learning methods and speech visualization means to detect the fatigue status of controllers using their working speech.The main work is as follows:Firstly,establish the controller voice database for fatigue.Due to the special nature of the controller’s work,the relevant fatigue speech is difficult to collect,and there is a lack of publicly available speech fatigue data resources.Therefore,in this thesis,after analyzing the fatigue factors of controllers,the fatigue-inducing experiments were conducted on controllers by using the objective fatigue factors of working hours and working intensity,and the voices were manually calibrated by fatigue scale scores and PVT response times,and 1510 normal voice samples and 1506 fatigue voice samples were defined to construct the original fatigue corpus of controllers.Secondly,fatigue speech detection is investigated based on convolutional neural network model.After converting the speech into a two-dimensional speech spectrogram,we found that the speech of two states showed obvious differences in the speech spectrogram.So,using the original spectrograph as the input to the convolution model,and three convolutional models,VGG16,Res Net34,and Conv Ne Xt-T,were selected for speech fatigue detection experiments,and the visualization technology was used to analyze the model’s focus on the region of the speech spectrogram.The feasibility of the method is verified through comparative experiments and visualization analysis.The experimental results show that all three convolutional models can achieve good detection results,among which Conv Ne Xt-T has the highest accuracy of 92.08%,and the model focuses on the low and middle frequency fatigue-related vocal information of the speech spectrogram.Finally,in order to further improve the accuracy of fatigue detection,the Conv Ne Xt-T model is improved and a speech fatigue detection model(D-Alst Net)based on attentional temporal sequencing is established.The network architecture consists of a convolutional neural network,a long and short term memory model(LSTM),a parametric-free attention(Sim-AM)and a channel attention(SENet).On the one hand,the attention mechanism can enhance the model’s focus on important features,and on the other hand,the LSTM can handle part of the temporal task.The effectiveness of the method is further verified by comparing the experimental and visual analysis results.The experimental results show that the accuracy of D-Alst Net model reaches 97.2%,which is 5.12% higher than the accuracy of the original model,and due to the introduction of the attention mechanism,the model pays more attention to the voice print details in the speech spectrogram,while the attention to the silent region is relatively reduced.