Research On Key Technologies For Multi-sensor Based Intelligent Sewer Pipelines Defect Inspection

The health of sewer pipelines is directly related to the normal running of cities and the daily life of residents.Routine inspections to prevent the accumulation of defects that can cause major safety hazards and economic losses have become an important part of urban development.At present,the industry generally adopts the visual method to collect the internal video of the sewer pipeline and then manually interpret it and generate a defect inspection report.This is not only time-consuming and laborious but also prone to false inspections and missed inspections.Although the current research related to intelligent sewer defect detection has been carried out for many years,it still cannot solve the problem completely.This dissertation firstly states the four most essential problems of pipeline defect detection: 1)Are there defects? 2)What are the types of defects? 3)Where are the defects?4)What are the levels of the defects? According to the survey,the current research only revolves around the first two problems to solve;and the latter two problems,especially the third problem,have become the pain point hindering the fully intelligent detection of sewer pipeline defects.After analysis,it is found that 3D mapping of the inner surface of the pipeline and fusing the 2D image detection results and 3D mapping results together are necessary means to fundamentally solve the four essential problems.Therefore,an intelligent defect detection scheme for sewer pipelines based on the integration of multibeam Li DAR(Light Detection and Ranging),single-beam Li DAR,CCTV(Closed Circuit Television)camera,and IMU(Inertial Measurement Unit)is proposed,and the following key technical researches are carried out on this basis:(1)One of the premises of multi-sensor fusion: extrinsic parameter calibration of multiLi DAR and IMU.A one-station self-calibration scheme for Li DAR and IMU extrinsic parameters is designed without a special calibration target but only relying on the common rectangular-shaped corridor in daily life.The solution first uses the geometric constraints formed by the corridor to convert the calibration of the extrinsic parameters of the lidar into a nonlinear optimization problem;then,on the basis of it,multiple Li DAR sensors are used as a whole and the corridor is used as a geometric reference to calculate the body posture in the calibration process,which can be used to the hand-eye based extrinsic calibration of the IMU.Experiments based on simulation data and real data verify the feasibility and calibration accuracy of the method.(2)The key to push-broom 3D mapping inside sewer pipelines: pose estimation based on multi-beam Li DAR.Multi-beam Li DAR will not be affected by factors such as fog,uneven fill light,and weak texture,but it is still difficult to solve the problem of pose estimation in weakly structured scenes such as sewer pipelines that lack three-dimensional reference targets.This paper proposes a multi-beam Li DAR odometry scheme based on CAEs(Convolutional Auto-Encoders)to realize the pose estimation inside the pipeline.This method first maps the multi-beam Li DAR point cloud to the spherical ring model and then uses 2D CNN(Convolutional Neural Network)to detect interest points;then maps the point cloud to the multi-resolution 3D voxel model and uses 3D CNN to extract the corresponding features.By using feature matching,the initial odometry based on inter-frame registration can be generated;finally,the pose estimation accuracy in the pipeline based on the multibeam Li DAR can be further improved through the pose optimization for the keyframes.The CNNs used are all trained through the unsupervised method of CAEs,and experiments have shown that this method performs better in sewer-like environments than existing methods.(3)The key to connecting 2D inspection and 3D mapping to fully solve the problem of defect location and level rating: sewer defect detection based on CCTV image recognition and the localization of multiple defects in the image.The designed neural network based on CNN class-wise attention can realize the detection of multiple defects in a single image and the location of each defect in the image based on the deep convolutional neural network based on weakly-supervised training only with image labels.At the same time,based on the output structure of the hierarchical classification network,the activation function selection scheme was modified,and combined with the selected suitable loss function for class imbalance,the single-image multi-defect detection problem was optimized.A total of 20,865 pipeline defect data sets were collected from sewer pipeline inspection videos covering about500 kilometers.Experiments show that the designed network structure can not only locate defects in the images but also improves the recognition accuracy of the network by about1%-2%.The multi-sensor integration solution proposed in this article aims to solve the pain points that restrict the full intelligent defect detection of sewer pipelines.The researched key technologies solved the calibration problem,pose estimation problem in sewer-like unstructured scenes,defect localization problem in 2D images based on weakly-supervised deep-learning,and at the same time improved defect-recognition accuracy.