Part Segmentation In Cluttered Scenes Based On Convolutional Neural Network

In order to adapt to the situation of industrial production automation,industrial robots are widely used in sorting operations.Aiming at the research problems of serious stacking of target objects in sorting operations and low accuracy of traditional detection and segmentation algorithms,a method of parts segmentation in cluttered scenes based on convolutional neural network is proposed.In-depth study of semantic segmentation and instance segmentation algorithms,building and training a network model for image segmentation,segmentation of parts in a cluttered scene,and laying the foundation for subsequent parts capture work.At the same time,in view of the problem that the segmentation algorithm based on convolutional neural network requires high-quality data sets,the three-dimensional simulation technology and physics engine are studied,and a virtual data set construction method based on Open Scene Graph(OSG)and Bullet is proposed.The main work of this paper is as follows:(1)The construction scheme of the virtual data set of parts in the chaotic scene is designed.Image segmentation based on deep learning requires high-quality data sets.Considering the lack of existing public data sets in the field of mechanical parts,and at the same time,in order to avoid manual and cumbersome annotations,a method for constructing virtual data sets based on Open Scene Graph and Bullet is proposed.Establish a collection of chaotic scene data into the system to generate virtual images for training.Firstly,use Solidworks to build the part model,and assign different material information to each part model through 3Dmax,then use the 3D graphics library Open Scene Graph and the physics engine Bullet to create a virtual messy scene,and finally generate a virtual image through viewport rendering.(2)The semantic segmentation method of cluttered scene parts based on deep learning is studied.First,the current mainstream semantic segmentation models Deep Lab V3,Deep Lab V3+,PSPNet and Refine Net are studied and analyzed;then a virtual data set of cluttered scene parts is established to train and test the semantic segmentation model,and the test results are compared and analyzed;finally,in view of the loss of feature information caused by the up-sampling operation of the semantic segmentation model,the fully connected conditional random field(CRF)image post-processing module is used to refine the results of image segmentation.(3)The segmentation algorithm of parts in cluttered scenes based on deep learning is studied.Aiming at the problem that it is difficult for the semantic segmentation model to segment stacked parts of the same type,a method for segmentation of cluttered scene parts based on Mask RCNN and virtual data set is proposed.First,the model was built,and the instance segmentation model Mask RCNN was divided into three modules:backbone network,regional candidate network,and network output;then three virtual data sets of the same type of parts were established,and the method of fine-tuning in migration learning was used to The model is trained;secondly,the rationality of the model division modules is visually analyzed using virtual images;finally,the real images collected by the Kinect camera are used to test the model,and the impact of different data sets on model training is analyzed.(4)Research on the panoramic segmentation algorithm of cluttered scene based on deep learning.Combine the semantic segmentation model and the instance segmentation model to build a panoramic segmentation network.First,use the chaotic scene training set to form a system to build a virtual data set,and the Kinect camera collects and builds a real data set.Aiming at the problem of holes and noise points in the depth image collected by the Kinect camera,a method of using the fast marching algorithm FFM to repair the depth image;Build a panoramic segmentation network model,use virtual data sets for pre-training,and a small amount of real data sets for fine-tuning training strategies;finally use real data sets for testing.Experimental results show that the model built in this paper can segment the target object and background in a cluttered scene.