Research On Vehicle-mounted Cross-modal Binocular Stereo Matching Syste

With the increasing development of intelligent driving technology,driverless cars have become an inevitable direction for the development of future transportation methods.Unmanned driving technology involves multiple research fields,including route plan,behavior prediction,SLAM,visual navigation and other research directions.Among them,visual navigation is a very important module.This module includes but not limited to image detection,segmentation,enhancement,Depth estimation,etc.The current stereo matching research mainly faces daytime scenes,and mainstream data sets do not include the lack of light at night.In practical applications,there is a great demand for scenes in complex environments such as nights,and it is very important to face stereo matching in such complex environments at night.This paper uses binocular cameras of different modalities to optimize the vehicle scene,combined with the current mainstream binocular stereo matching algorithm for depth estimation,and build a complete multi-modal vehicle 3D reconstruction system.The main work of this paper is as follows:(1)Complete the construction of the vehicle-mounted large-depth-of-field stereo matching system.This paper uses visible light cameras,lidar rangefinders,and long-wave infrared cameras to complete the vehicle-mounted multi-modal stereo matching system.This paper proposes a cross-modal data polar line correction method based on depth information.The depth information is collected by introducing a lidar sensor,and this information is used to effectively correlate the same information in different modalities,and the infrared image is subjected to high-precision correlation through this correlation.Mapping can effectively keep the characteristics of infrared images unaffected.In this paper,the true value of parallax is obtained by combining the lidar depth information with the polar line correction calibration result to complete the dataset.At the same time,the frame synchronization of the visible light camera and the long-wave infrared camera is realized through the soft trigger method,and the system is finally completed.(2)Research on cross-modal stereo matching algorithm.This article designs a new multimodal feature consistency constraint method by analyzing the way the stereo matching network processes the feature map and combining the true value of the disparity.Through this constraint method,the feature extraction structure can effectively extract the consistent features between different modalities.This article combines the limitations of end-to-end training,designs the acquisition method of the pre-training model,and combines the characteristics of the encoder and the decoder,designs a new content retention method,completes the training of the pretraining model,and strengthens the end-to-end through the pre-training model Training convergence ability.Through the above work,this paper has completed the construction of the system and the design of the algorithm.Finally,the average pixel error of the disparity map is 3.2862 in the binocular stereo matching of the car scene at night.Compared with other algorithms,it has a significant performance improvement.