Space Weld Extraction And Multi-robot Task Planning For Complex Welding Robot Applications

With the wide application of robot technology in welding,the demand of intelligent welding without teaching for 3D space weld is gradually increasing.The accurate perception and extraction of space weld is the key to realize intelligent robot welding,and its precision directly affects the final welding quality.In addition,in the face of complex and diverse welding scenes,it is often necessary for multiple robots to cooperate in welding at the same time,so task allocation and planning among multiple robots is also the difficulty to improve the working efficiency.Therefore,in this paper,an in-depth study is conducted on 3D space weld feature extraction and multi-robot task planning in complex welding operations.The specific contents are as follows:Aiming at the problem of 3D space weld extraction,a weld extraction method based on3 D point cloud projection mapping combined with image processing technology is proposed.Firstly,the linear structured light sensor was combined with the robot to realize 3D perception,and the 3D point cloud of the workpiece is obtained and preprocessed by filtering,de-noising and ROI extraction.Secondly,a new projection mapping method of 3D point cloud is constructed.On the basis of determining the projection direction through PCA principal component analysis,the precise mapping relationship between the point cloud and the depth image is generated by the raster regular compression method.Therefore,the representation ability of depth image to point cloud spatial information is improved.Then the weld features are extracted based on Canny edge detection algorithm.Finally the weld feature points are back projected to the point cloud.The experimental results of many kinds of space weld extraction show that the weld extraction method based on 3D point cloud projection mapping proposed in this paper can achieve the extraction effect of about 1Hz and submillimeter.In order to improve the generalization learning ability and robustness of the weld extraction algorithm,the weld feature extraction method based on 3D point cloud deep learning mechanism is explored and studied in this paper.Firstly,Weld3 D data sets are constructed according to the geometric characteristics of different welds.Secondly,two welding seam extraction methods combined with deep learning are proposed.1)Depth projection image weld extraction method based on FCN,on the basis of 3D point cloud projection mapping,the FCN model is used to extract the features of the projection mapping depth map by semantic segmentation.Then the feature points of the weld are extracted by back projection into the 3D space.2)3D point cloud weld extraction method based on Point Net,the semantic segmentation of point cloud is directly used to extract the weld feature points through Point Net network.The experimental results show that the above method has a certain generalization ability and good real-time performance for the extraction of space weld.Aiming at the problem of multi-robot task planning for complex welding work,a multi-robot multi-station task allocation algorithm based on stepwise optimization(SO-MRMSTA)is proposed.Firstly,the optimization model of the problem is established as a whole.Then,based on the stepwise optimization algorithm,the problem is divided into three sub-problems from top to bottom: single-robot trajectory planning problem,multi-robot task planning problem and multi-station task assignment problem.1)The single robot trajectory planning problem is similar to the TSP problem,which takes the robot operation time as the optimization objective and is solved by the LKH solver.2)A region assignment method is proposed for multi-robot task planning problem,and the problem is transformed into an optimization problem of selecting region dividing lines,which is solved based on genetic algorithm.3)The problem of multi-station task assignment is solved by the principle of task balance.Finally,several cases are used to compare the proposed method with the expert-tuning method and the two cutting-edge methods,and the results verify the effectiveness of the proposed method.