Research And Implementation Of Transparent Object Visual Detection And Grasping System Based On Segmentation And Depth Completion

In recent years,with the rapid development of technologies such as artificial intelligence,the intersection of robots and AI has entered a critical period of development.The application fields of robots continue to expand,from traditional industrial scenes to various industries,and have been widely used.Laboratory robots,medical robots,and logistics robots have also appeared and gradually entered people’s vision.Unmanned laboratories are intelligent laboratories that can automatically complete various laboratory operations and management tasks,improve the accuracy of experimental data and the reliability of experimental results.As an important equipment in unmanned laboratories,robots can complete many tasks in the laboratory.However,traditional visual detection technology cannot correctly identify transparent objects,which brings great difficulties to robots in experiments in unmanned laboratories.Therefore,a new visual detection technology is needed to detect and grasp transparent objects in unmanned laboratories.The research content of this paper mainly includes:This paper proposes a three-stage method to further improve the visual detection and depth data acquisition of transparent objects.The method uses RGB information and RGB-D information for semantic segmentation and depth completion of transparent objects.In the first stage,this paper uses a neural network mixed with CNN-Transformer and boundary optimization to accurately perform semantic segmentation of transparent objects.In the second stage,surface normal prediction and boundary occlusion detection are performed for the subsequent reconstruction of depth data for transparent objects.In the third stage,based on the semantic segmentation results of the first stage,the depth data of the transparent object region is removed,and the removed depth data is reconstructed using the results obtained from the second stage to obtain more reliable depth data.After obtaining the depth data,using the depth data and camera intrinsic parameters,the spatial position of the target relative to the camera is derived.Then,using the obtained spatial position and camera extrinsic parameters,the spatial position of the target relative to the robotic arm base is derived.Using the calculated position information,the Move It algorithm in the ROS robot operating system is used to complete the path planning of the robotic arm when grabbing the object,and finally,the closed-loop grasping operation of the robotic arm in the laboratory is realized under visual guidance for transparent objects.In addition to that,functional and non-functional requirements analysis,as well as feasibility analysis,were conducted for each module of the system.Based on the system design,various programming techniques were utilized to construct the system interface.The implementation of visual algorithms and robotic arm algorithms was carried out to develop a visual detection and grasping system specifically designed for transparent objects.Following the system implementation,thorough functional testing was conducted to ensure the system’s stable performance.