Research On The Chemistry Laboratory Apparatus Detection And Segmentation Based On Computer Vision

The safety of chemical laboratories has always been a key problem of concern for the majority of research institutions and chemical companies.In the past years,safety accidents such as explosions,poisoning and fires in chemical laboratories have been commonplace.In order to avoid safety problems caused by operational errors,it is very important to automate the chemical experiment process.In the future chemical laboratory scenario,machine for human is an important initiative to promote industrial upgrading,and replacing manual testing of chemical laboratory instruments and conducting dangerous chemical experiments with robots will become the future development trend.Currently,robots that can conduct chemical experiments autonomously have been realized,and they use the same chemical laboratory instruments as human chemists to conduct chemical experiments unmanaged.Chemical laboratory apparatus is an important part of chemical experiments,and being able to accurately identify and detect chemical laboratory apparatus is a key part of the process of conducting chemical experiments.Computer vision technology based on deep learning learns by inputting massive chemical laboratory apparatus image data into a multilayer convolutional neural network to analyze and learn,simulating a biological neural brain,from which experience is gained to finally identify and locate the apparatus.Therefore,deep learning and computer vision can be used to add a pair of human "eyes" to chemical experimental robots,which can adapt to the changing reality.In this thesis,we propose a deep learning and computer vision-based method for detecting and segmenting chemical laboratory apparatus,which aims to accurately identify and locate chemical laboratory apparatus objects and outline the boundaries of chemical laboratory apparatus objects,and further improve the accuracy and speed of detecting and segmenting chemical laboratory apparatus.The research in this thesis focuses on:(1)This thesis first investigates the technical characteristics of computer vision and deep learning and the construction of chemical laboratory image datasets.Since there is no publicly available dataset about chemical laboratory apparatuses,in order to restore the real environment when conducting chemical experiments as much as possible,we visited several chemical laboratories in the School of Energy Materials and Chemical Engineering of Hefei University and collected a large amount of original chemical laboratory apparatus image data by taking photographs.For different kinds of chemical laboratory apparatus,the original images were collected by photographing around different backgrounds,perspectives and brightness.Data cleaning is performed manually,and image annotation is also performed.The dataset is augmented by data augmentation techniques such as rotation and flip.Finally,the chemical laboratory apparatus image dataset(CLAD)is established with 13,476 chemical laboratory apparatus images and 21 chemical laboratory apparatus categories,which are labeled with chemical laboratory apparatus image recognition dataset,chemical laboratory apparatus object detection dataset,chemical laboratory apparatus semantic segmentation dataset and chemical laboratory apparatus instance segmentation dataset,respectively.(2)Chemical laboratory apparatus classification was evaluated using popular image recognition models in the chemical laboratory apparatus image recognition dataset,and the performance of each classification model was compared,while the image recognition model with the best classification performance was used as the backbone network part of the chemical laboratory apparatus detection model for feature extraction.In practice,chemical experimental robots usually use robotic arms and mechanical grippers to grasp experimental apparatus,so the location of chemical experimental apparatus needs to be framed out from the image,and the object detection model is used to learn and train in the chemical laboratory apparatus object detection dataset,and the feature pyramid network is used in the Faster R-CNN model to increase the model computation by internally changing the network connections in a small way The detection model performance is further improved by substantially improving the accuracy of small chemical laboratory apparatus detection.(3)A semantic segmentation model based on deep learning is adopted for training on the semantic segmentation dataset of chemical laboratory apparatus,so as to exclude the interference of background information and identify the contours of chemical laboratory apparatus more accurately.In order to further distinguish the overlapping interference of different instances of the same class of chemical laboratory instruments on the semantic segmentation results,the object detection results of the above Faster RCNN model are combined with the FCN semantic segmentation model for segmentation to obtain the location and contour of each chemical laboratory apparatus instance.The experimental results perform well,and the visualization verification about the detection and segmentation of chemical laboratory apparatus is also performed.