Research On Recognition Methods Of Miners’ Unsafe Behavior Based On Deep Learning

Coal mine safety accidents seriously threaten the life safety of miners and the economic benefit of enterprises,and the safety accidents caused by the unsafe behavior of miners account for more than 85% of the total safety accidents.As a result,the research on the recognition methods of the unsafe behavior of miners is of great significance to ensure the safe production of coal mines.With the development of mining Internet of Things and artificial intelligence,the research on mining unsafe behavior recognition has made some progress.However,the coal mine environment has complexity and particularity,and there are still three problems in the current research on the unsafe behavior of miners.Firstly,the unsafe behaviors of miners are sensitive to the environment,and the judgment of some miners’ behaviors requires indepth analysis of the interaction between behaviors and environment to determine whether they are safe or not.Secondly,the problem of insufficient samples exists in the unsafe behaviors of miners.There are many kinds of unsafe behaviors of miners,and the probability of some unsafe behaviors of miners is low,so it is difficult to collect samples.Thirdly,real-time response to unsafe behavior of miners.The existing behavior recognition model has high computational complexity and is usually deployed in cloud server.How to guarantee the real-time response characteristics of unsafe behavior is also one of the issues studied in this dissertation.In view of the above problems,we carries out research on the recognition method of unsafe behavior of miners this dissertation.The main research contents are as follows:(1)To solve the problem of being sensitive to environment of miners’ unsafe behavior,the method of semantic description is adopted to map the video clips containing unsafe behavior,interactive objects and environmental state into semantic space,so as to realize the unified description of unsafe behavior of miners,and at last it can provide semantic support for the recognition of unsafe behavior.The existing video description method usually uses the codec method to convert between the visual mode and the language mode.The feature expression and extraction of video modes are difficult to meet the requirements of fine-grained recognition of unsafe behaviors of miners.In order to fuse the low-level and high-level semantic features of Convolutional Neural Network(CNN),the Convolutional Long and Short Term Memory Network with strong expression ability of spatio-temporal features is adopted to fuse the feature images output by different Convolutional layers of CNN.The attention mechanism is used to enhance the weight of relevant semantic features to model temporal and spatial sequences.The experimental results show that the proposed method is superior to the current mainstream video semantic description methods.(2)As for the problem of insufficient samples of unsafe behaviors of miners,a meta-learning method based on adaptive feature fusion was proposed to realize the recognition of unsafe behaviors of miners under the condition of small samples.The main feature of this method is to put forward an adaptive CNN feature graph fusion method,which effectively integrates the texture,color and other information at the bottom with the semantic information at the top.The quantum genetic algorithm(QGA)is used to optimize the embedding model and classifier,and the feature embedding model with stronger feature expression ability is selected effectively.Experimental results show that the proposed method is superior to the current mainstream small sample learning methods in mining unsafe behavior recognition and a variety of small sample image recognition tasks.(3)In order to solve the problem of real-time response of miners’ unsafe behaviors,a lightweight CNN behavior recognition model based on dynamic convolution kernel was proposed.The model was deployed in resource-constrained edge equipment to realize rapid recognition of miners’ unsafe behaviors.The main feature of this model is to propose a dynamic convolution accounting algorithm based on channel information redundancy.The operator preserves part of the spatial information of the selected channel so as to enhance the self-adaptability of the spatio-temporal information of the dynamic convolution kernel.Then,the convolution operator is used to improve the twodimensional and three-dimensional lightweight CNN and identify the unsafe behavior of miners,so as to improve the feature representation and learning performance of the lightweight CNN behavior recognition model.Through static images and video on the miners’ unsafe behavior recognition database respectively to verify the improved 2D and 3D lightweight CNN unsafe behavior recognition model,the results show that the method can not only reduce dynamic convolution kernels based lightweight CNN’s parameters and the amount of calculation,also effectively improved the convolution of lightweight network layer of the characteristics of the polymerization and extract performance.(4)A Hierarchical Adaptive Loss Transferring Framework(HALTF)is proposed for loss constraint and optimization of CNN behavior recognition model.HALTF consists of multiple training stages.In each stage,branch loss function is selectively added in shallow layers to train the lightweight model,and quantum genetic algorithm is adopted to optimize the weight of loss at different layers to integrate the different knowledge learned from the hierarchical branch loss function of CNN model,so as to improve the feature learning performance of CNN.Through experiments on mining unsafe behavior recognition database and a variety of public databases,this method can effectively improve the performance of the model’s feature expression.The dissertation contains 54 figures,37 tables and 192 references.