Research On Workflow Recognition System Based On Human Skeleton

With the advent of the era of Industry 4.0,the intelligent manufacturing model is gradually gaining popularity,in which video surveillance plays a pivotal role.At present,the vast majority of manufacturers are deploying monitoring systems on the product production line to realize the traceability of the product production process in the future.However,the current traceability work is mainly done manually,which is not only inefficient,but also requires a lot of human work,which is time-consuming and laborintensive.At the same time,the resource utilization of the monitoring device is low.In view of the above problems,this thesis takes a display manufacturer as the background,introduces machine vision technology into it,and proposes a workflow recognition system based on human skeleton.The main research contents are as follows:First,analyze the needs of the panel assembly workshop of the display manufacturer,and complete the following two aspects based on this: selecting the hardware resources involved in the workflow recognition system,and building the hardware foundation;analyzing the advantages and disadvantages of different behavior recognition algorithms,aiming at the panel For the problems of people walking around and small movements in the assembly workshop,the skeleton-based behavior recognition method is used as the algorithm scheme of the workflow recognition system.Secondly,aiming at the problems of complex background and interfering information caused by people walking in the panel assembly workshop,based on the YOLOv3-Tiny target detection model,a fast human detection model in the workflow recognition system is designed,and the output head in the model is improved.;Aiming at the occlusion of key points in the lower body of the assembler and the lack of key point features in the hand,based on the Hrnet key point detection model,redefine the number and category of skeleton key points,design a skeleton detection model for the workflow recognition system,and use it in the human body.The calculation method of the coordinates of the key points of the bones and the weight ratio of the key points of the bones have been improved.Then,the traditional machine learning method is used to extract and analyze the key points of the skeleton output by the skeleton recognition network in the time and space dimensions,so as to realize the classification of the factory behavior.Aiming at the weak robustness of traditional machine learning methods and easy false detection of delicate movements,based on the spatiotemporal graph product neural network,the workflow recognition task is converted into the classification of the spatial position information of skeleton key points in consecutive frames.The experimental results show that the spatiotemporal graph convolutional neural network can better complete the skeleton-based action recognition task.Subsequently,in view of the large number of parameters of the workflow recognition model,slow inference speed,and difficulty in deployment,the compression optimization of the workflow recognition model is completed based on the Tensor Rt inference engine.Finally,the overall parameters of the model are reduced by 67.1%,and the inference speed is increased by 189%.The inference rate on Nvidia Xavier is16 Fps,which meets the basic deployment requirements.Finally,based on Py Qt5,the interface control management system of the workflow identification system is developed to realize the visual monitoring and efficient management of the workflow identification system.The thesis has 55 figures,14 tables,and 66 references.