Deep Learning Based Binocular Machine Vision System And Its Application In Elevator Inspection

As the demand for elevator safety inspection grows,the use of robots instead of traditional manual inspection has become an inevitable trend for reasons such as saving human resources in society.Therefore,this paper designs and implements a robot system that can ride the elevator autonomously.Considering the need for the robot to identify and locate the elevator buttons,based on the completion of the overall robot system design,the position information of the target button in the camera coordinate system is obtained through a binocular machine vision system based on deep learning.After processing this information through coordinate system conversion and so on,it is of great practical significance to manipulate the robot arm to complete the button action,realize the autonomous elevator ride and carry the elevator measuring instrument to complete the inspection of the elevator,so as to solve the boring and repetitive work brought by these demands.The main work of this paper is as follows:(1)The overall design of the autonomous elevator riding robot system is carried out,and the research and analysis are carried out in terms of software and hardware respectively.In order to reduce the influence of camera distortion parameters on subsequent target detection,the calibration method of Zhang Zhengyou’s camera and the calibration tool in Matlab are used to complete the calibration and correction of the distortion parameters of the binocular camera.(2)To address the needs of elevator button recognition in the local highlight area and button on state,the study is solved based on the use of YOLOv4 algorithm.First,in order to solve the problem of low recognition accuracy in the local highlight area of elevator buttons,a light compensation algorithm is proposed,which improves the recognition accuracy in the highlight area by 12.1% compared with the original YOLOv4 algorithm;second,a method based on luminance feature extraction is proposed for the recognition of elevator buttons’ on status,which can fully solve the demand for detection of buttons’ on status.(3)To further optimize the algorithm on the basis of satisfying the detection requirements,the YOLOv4 algorithm is improved in terms of real-time and leakage detection rate.Experiments comparing the YOLO series algorithms longitudinally show that,compared with the original YOLOv4 algorithm,the average detection time is shortened by 16.9% and the leakage rate is reduced by 2.2%,while the detection accuracy of elevator buttons is only reduced by 2.4%.After that,several elevator button detection algorithms in recent years were analyzed in cross-sectional comparison experiments,which showed that this algorithm has significant advantages in detection time and leakage rate,with the values of 18.7% and 4.1%,respectively.At the same time,the accuracy change curve is relatively smoother and more stable.(4)The problem of obtaining the 3D position of the target button and how to guide the robot arm to reach the specified position accurately is addressed.Firstly,the spatial coordinates of the button on the camera coordinate system are calculated by triangulation;secondly,forward and inverse kinematic modeling is performed based on the D-H parameters of the robotic arm,so that hand-eye calibration can be performed to obtain the spatial coordinates of the target button in the robotic arm coordinate system;finally,the obtained coordinates are sent to the robotic arm for execution through the industrial control host,and experimental verification is conducted for the accuracy of button recognition and the accuracy of position calculation.