Tractor Assembly Operation State Recognition For Digital Twin Workshop

At present,the domestic tractor assembly production workshop automation degree is low,when the introduction of digital twin technology applied to the production control,the existing research for personnel is less,resulting in the tractor assembly production workshop digital twin is difficult.On the other hand,in the tractor mixed-flow production line,the assembly operation is mainly done by the personnel,and the proficiency of the personnel has a great impact on the production efficiency and even causes the production bottleneck.To address the above problems,this paper is oriented towards the digital twin process of tractor production workshop,through computer vision to identify personnel and obtain the operating process posture data;and establishing the posture mapping relationship of the human virtual model and the real human body,which realizes the 3D posture-based human virtual model drive;and through the twin data action analysis to achieve the analysis of personnel operating state,which providing basic data for the digital twin-based intelligent control of tractor assembly production line.The main research contents and results of this paper are as follows:(1)Workshop personnel identification and 3D posture detection based on computer visionAiming at the complex environment of agricultural equipment assembly workshop,all assembly stations have personnel involved,and by acquiring the video information of on-site operation,a workshop complex environment operation personnel recognition model is constructed based on Yolo-v4 CNN.The results show that for assembly line operations the correct detection rate is 90.14% in the case of a single person operation and 88.32% in the case of a pair.Based on the operator recognition model,the human 3D posture detection is realized based on the Videopose3 d recognition model.For the problem of human morphological differences caused by video detection,the personnel posture is standardized according to the standard human model.The test results showed that the error of the key points in the three directions is between 3 and 4 cm when the person was at a line change from the camera,so the posture standardization results met the accuracy requirements for human posture recognition in real space.(2)Digital twin of human model based on 3D posture dataThe establishment of the digital twin model of the personnel is the key of digital twin applied in tractor assembly,combined with the digital twin workshop framework to establish the virtual human model;the model is lightened and imported into the digital twin workshop to complete the model skeletal mapping and static position mapping relationship establishment;the establishment of the twin data transmission channel based on Rabbit MQ,which realizes the data collaboration of the digital twin workshop;human virtual model is drove by the inverse kinematics,the driving results showed that the virtual human model action matched with the actual assembly operation action,which realizes the collaboration between the actual workshop personnel and the model,providing a feasible technical solution for serving the digital twin of tractor assembly line.(3)Posture feature analysis based on twin dataBy analyzing the posture data during the assembly operation,the posture feature of continuous operation of personnel assembly is proposed as the elbow joint grip angle,and the feature data set of each process of the assembly operation is constructed by combining the time series of the grip angle change process;the process work time analysis method based on dynamic time window and feature similarity matching is constructed.The results of different process work time analysis showed that the analysis method can accurately analyze the work time of each process,and the average error of work time identification is 3.3s,which met the work time calculation requirements of the process.(4)Development of personnel operation status recognition system based on virtual-real fusion technology.In order to meet the demand of providing personnel operation twin data for digital twin workshops,we use graphical interface development technology and network communication technology to build a system platform that can be used for personnel operation process state identification.Through local video or remote video stream analysis,the system realizes functions such as personnel identification,3D pose detection,Rabbit MQ-based data transmission,personnel digital twin and visualization presentation,and operation status analysis.