Research On The Measurement Of Grasping Pose For Scattered Workpieces Based On Instance Segmentation

Robot automatic grasping technology is widely used in production workshops that need workpiece loading,unloading and assembly.However,there are often positioning and grasping operations for scattered workpieces in the actual scene.Due to the arbitrary position and pose,occlusion,and stacking of scattered workpieces,traditional monocular vision technology is challenging to measure position and pose accurately.At present,it still uses manual mode or vibration screening mechanism to achieve the flat placement of the scattered workpieces and then performs positioning and grasping operations,which seriously affects the production efficiency.Therefore,to improve industrial production efficiency,reduce the manufacturing cost,and solve the problem of automatic grasping of scattered workpieces,this paper realizes the pose measurement of scattered workpieces by combining deep learning-based instance segmentation neural network and point cloud registration technology.The main contents are as follows:(1)A grasping system of the scattered workpiece is designed.Firstly,the definition of the pose of the grasping system of the scattered workpiece is described,the transformation relationship between each coordinate system is analyzed,and the pose relationship to be solved is determined.Secondly,the requirements of the grasping task are analyzed emphatically.By summarizing the existing technical difficulties,the overall design scheme of the system is put forward,and the hardware selection and design is completed according to the overall design scheme.Then,according to the design of the end fixture,the grasping position is analyzed,and the pose relationship between the end tool coordinate system and the workpiece coordinate system is obtained.Finally,the camera calibration parameters are analyzed in detail,and the internal parameters of the selected depth camera are solved by designing the camera calibration experiment,which provides accurate parameters for the subsequent point cloud reconstruction of the workpiece surface.(2)The instance segmentation network model based on deep learning is studied.Firstly,the basic layer of the neural network and image data transmission process is described,and the composition framework of Mask Region-Based Convolutional Neural Network(Mask R-CNN)network and the functions of each sub-network are emphatically analyzed.Secondly,the scattered workpiece images are collected by depth camera,and the workpiece dataset is made by the annotation tool,Labelme.Finally,the self-made dataset is expanded by data augmentation,and the Mask R-CNN is trained under the Tensor Flow framework combined with the pre-training weight of the Common Objects in Context(COCO)dataset.The trained Mask R-CNN is tested with the test set image.The test results show that Mask R-CNN network model can achieve accurate instance segmentation for scattered workpieces.(3)The point cloud registration algorithm based on keypoint extraction is studied.Firstly,the point cloud on the workpiece surface is obtained through the prediction mask segmentation depth map of the neural network,and the template point cloud is obtained by sampling the workpiece Computer Aided Design(CAD)model,and the obtained point cloud data is downsampled reduce the number of point cloud.Secondly,in order to improve the speed and accuracy of pose registration,by comparing the three key point extraction effects of 3D Conner(3D Harris),3D Scale Invariant Feature Transform(3D SIFT)and 3D Intrinsic Shape Signatures(3D ISS),a pose measurement algorithm using different keypoint extraction algorithms for template point cloud and workpiece surface point cloud is proposed.3D Harris keypoint extraction is used for template point cloud,and 3D ISS keypoint extraction is used for workpiece surface point cloud,and the extracted key points are used as the input data of the registration algorithm.Finally,the Sample Consensus Initial Alignment(SAC-IA)algorithm is used for rough registration,and the Iterative Closest Point(ICP)algorithm is used for pose correction.The pose relationship between the workpiece coordinate system and camera coordinate system is quickly and accurately obtained,and simulation experiments verify the feasibility of the proposed algorithm.(4)The experimental verification of the scattered workpiece grasping system is carried out.Firstly,a hand-eye calibration experiment obtained the transformation relationship between the camera coordinate system and the robot base coordinate system.Secondly,the grasping experimental hardware platform is built,and the grasping system software is designed.Taking the tee and elbow pipe as the experimental objects,the pose measurement algorithm proposed in this paper is used to grasp the scattered pipe workpieces.Finally,various scattered workpiece placement scenarios are designed,and the robot grasping experiment is repeated to verify the system’s reliability developed in this paper.