Research On Assembling Operation Monitoring Based On Deep Learning

With the increase of personalized demand in the manufacturing industry,mass customization production arises at the historic moment.Mass customization production mode often needs to reorganize the assembly line according to personalized demand.However,assembly workers are limited by knowledge and ability and are difficult to adapt to this changeable production mode.Problems such as omission of key assembly processes,incorrect use of assembly too ls and substandard assembly operations often occur.As for assembly monitoring,the previous research focused on product quality monitoring,which could not avoid workers' operation errors in the assembly process in time,resulting in lower product qualification rate,increased production cost and longer production cycle,which is difficult to meet the development needs of today's assembly manufacturing industry.In view of these problems,this paper studies the action recognition technology,object detection technology and pose estimation technology based on depth learning,and realizes assembly operation monitoring from three levels of assembly action monitoring,assembly tool monitoring and assembly action repetition judgment.The specific research contents are as follows:(1)The assembly action monitoring method based on action recognition technology is studied,and a new 3D convolution neural network model with batch normalization layer is proposed.Firstly,307 video samples including 9 types of assembly actions are established,and all video frames are processed to form assembly action data sets of RGB,gray,binary and depth patterns.Then,a new type of three-dimensional convolution neural network with batch normalization layer is established,which is trained and tested on the data sets of four modes respectively,and the results are compared and analyzed.Experiments show that the new 3D convolution neural network model established in this paper greatly improves the convergence speed of the model during training,and the recognition accuracy reaches 81.89%,thus realizing the monitoring of assembly actions.(2)The assembly tool monitoring method based on YOLOv3 object detection algorithm is studied.Firstly,an assembly action data set containing tool information is established,each image in the data set is marked by corresponding marking tools,and the marked tool category and coordinate position information are taken as training labels;Then,YO LOv3 object detection algorithm is used for training and testing on self-built data sets,and the results are compared with R-CNN,Fast R-CNN and Faster-RCNN.Experiments show that YO LOv3 object detection algorithm has a detection accuracy of 92.5% and a detection speed of 32 fps,which realizes the monitoring of the use of assembly tools.(3)This paper studies the method of judging the number of repetitions of assembly actions based on Open Pose pose estimation algorithm,and proposes a new method of judging the number of repetitions of assembly actio ns by replacing the conventional action recognition algorithm with the object detection algorithm to judge the action category and the occurrence time point,and combining the joint coordinate information obtained by pose estimation.First of all,three typical assembly actions of hammer,file and nut were selected.Each action was recorded by ten experimenters and 30 video samples were obtained.Then,YO LOv3 object detection algorithm is used instead of action recognition algorithm to detect the category and occurrence time of assembly action.Subsequently,Open Pose is used to estimate the pose,obtain joint coordinates,clean and analyze the coordinate data,and judge the number of repetitions of assembly actions according to the data analysis results.The results show that the method proposed in this paper is simple and feasible to monitor the completion of assembly operations.