Research On Campus Environment Scene Understanding Based On Laser Point Cloud And Visual Image Fusion

High-quality scene understanding technology is the key to achieving highly unmanned intelligent driving.Road users are complex and constantly changing in the park environment,which brings great challenges to the understanding of scenes in urban parks,especially the scene understanding images of road participants and their surroundings.In this regard,this paper proposes an improved E-Net scene understanding algorithm that introduces laser point cloud to optimize the area growth,and realizes the scene understanding of the park environment.The specific research work of this article is as follows:(1)Build a multi-sensor hardware platform and software system.In order to improve the system’s ability to understand the scene,build a lidar and camera hardware platform and develop a software system for data collection and processing that integrates laser point clouds and visual images.The lidar and camera sensor data collection and processing interactive system based on the Qt platform realizes the collection and processing of image and point cloud data,reducing the time spent on many repetitive tasks in the multi-sensor independent data collection and processing process,and can directly output the laser point cloud and The result of visual image fusion reduces a lot of repetitive work when collecting and processing data.(2)Realize data fusion and data processing of laser point cloud and visual image.Relying on a single sensor to achieve perception and failing to achieve the optimal scene understanding,so in this paper,through the calibration of lidar and camera internal and external parameters,the sensors are jointly calibrated and fused to achieve the fusion of data of different dimensions.And use binary filtering to remove noise from the image data;perform post-processing operations such as bilateral filtering,ground segmentation,and European clustering on the point cloud data to effectively distinguish the ground point and ground obstacle point cloud data,and optimize it for later The regional growth algorithm provides the basis.(3)An improved E-Net scene understanding algorithm based on optimized region growth is proposed.Since the scene segmentation accuracy of the semantic segmentation network for large scenes has a significant order of magnitude difference,this paper proposes an improved E-Net scene understanding algorithm that introduces laser point clouds to optimize the growth of the area,and realizes the scene understanding of the park environment.Each point is achieved by mapping the point cloud group after the European clustering as a seed to the E-Net semantic segmentation output and extracting the Canny edge pixel coordinates of the original image as the region growth boundary on the E-Net semantic segmentation output image.The detection of the exact image area of the scene effectively improves the segmentation accuracy of E-Net semantic segmentation for small scenes.(4)Fully verify the improved E-Net scene understanding algorithm.This paper selects different data sets for experimental verification,including the actual park collection data set and intelligent driving public data set KITTI.First,the scene understanding algorithm before and after the improvement is applied to process the actually collected park data.Through visual comparison,it is proved that the improved E-Net scene understanding algorithm can detect more accurate small scene areas.At the same time,the public data set KITTI is used for experimental verification.The experimental results show that the improved E-Net has a significant effect on the average segmentation accuracy of small scenes.In summary,this paper uses multi-sensor fusion,area growth algorithm and other methods to improve the E-Net scene understanding algorithm,realize the scene understanding of the park environment,and effectively improve the accuracy of small scene image area detection in the park environment,focusing on solving The semantic description and segmentation accuracy of various road participants in the park are provided,which provides a useful reference for intelligent driving in the park environment.