Traffic Sign Extraction Based On Panoramic Images And MLS LiDAR Point Clouds

Traffic sign is a part of road traffic facilities,which conveys traffic indication information to people through their words or symbols,so as to ensure road fluency and pedestrian safety.From the establishment to the operation stage of traffic signs,it is necessary to carry out qualified acceptance and uninterrupted monitoring and maintenance,in order to prevent traffic problems from fuzzy and damaged signs.In view of the above problems,image-based traffic sign detection and classification methods emerge constantly.However,the detection results lack geometric information,unable to locate and study the traffic sign in the three-dimensional environment,also cannot monitor and evaluate the rationality of its setting.To sum up,the detection,classification and geometric positioning of traffic signs in the three-dimensional environment are very important.In view of this,this paper proposes a traffic sign extraction method that integrates panoramic image and MLS LIDAR point clouds.The advantage of this method is that traffic signs with accurate semantic information and complete geometric information can be obtained.The major study content of this paper is stated in detail as follows:(1)The great significance of traffic signs to road traffic fluency and pedestrians' safety in modern society is introduced,and then the background and meaning of traffic sign detection and recognition is drawn forth.Then,according to the characteristics of various traffic sign data acquisition,the advantages and disadvantages of current traffic sign extraction methods based on image or point clouds are summarized.And based on this,the research objectives,content and structure of this paper are determined.(2)This paper introduces the setting specifications of traffic signs in China's national standards,and classifies the common traffic signs in urban scenes.Then,it emphasizes several types of traffic signs in the research range of this paper.Afterwards,according to the two data sources,the traffic sign detection and recognition methods and the shortcomings are summarized,and the improvement is discussed.(3)Image detection and recognition are often operated by deep learning framework.This paper briefly introduces the network structure of YOLOv3 algorithm.Based on YOLOV3 algorithm,traffic signs in panoramic images are detected and recognized,and then the results are analyzed to summarize the current problems.Subsequently,the point clouds is preprocessed,combined the above test results and the initial registration model of point clouds with panoramic images,the region of interest in the point clouds greatly reduces the search space of the traffic signs,by clustering and the point clouds filtering based on reflection strength to extract traffic signs coarsely,the extraction efficiency and precision are improved greatly.(4)The limitation of perspective,target reflection intensity and scanning time window lead to partial missing of laser scanning point clouds,which has a great impact on the integrity of traffic sign collection.Meanwhile,due to the fusion error of the two data sources,the ROI of the point clouds contains redundant points.Therefore,based on the center of the image in the target detection results and the corresponding template of the sign we can accurately extract the sign points in the projection of the local coordinate system and get a complete geometric boundary of the traffic signs,improving the traffic signs' geometrical integrity.Finally,the method of point clouds symbol extraction accuracy is up to 97.9% on average.