Object Detection And Localization System Based On Image And Point Cloud Fusion In Unmanned Scenario

With the rapid and vigorous development of artificial intelligence in recent years,the topic "unmanned driving" has aroused extensive attention from enterprises,scholars and even ordinary citizens.However,as for how to realize unmanned driving,there are two distinct approaches.The first approach is based on the current driver assistance system,which is a progressive method adopted by traditional automobile enterprises.With functions like automatic steering and active collisions prevention gradually added,conditional unmanned driving is available and when the cost and relevant technologies satisfy certain requirements,unmanned driving can be further achieved.The other method pursues one-step realization of unmanned driving,and it is favored by high-tech IT enterprises.As the latter method faces more challenges and risks,innovative algorithms and efficient robust system are needed to support it.Besides,with such strong demand,target detection and positioning are particularly important as they can make the car observant and alert to all directions and provide massive useful information for the following decisions and planning.With regard to the reasons above,this article conducts research on the perception system used in the scene of unmanned driving.Today's self-driving cars are often equipped with a variety of sensors that work together to perceive their surroundings,and cameras and LiDAR are among the most commonly used.After analyzing existing studies,we propose a target detection and positioning system which integrates images and point cloud data.The core of the system is an end-to-end network framework based on deep learning.This system can detect objects of multiple categories simultaneously and predict their center points,length,width,height,and orientation,thereby enabling unmanned vehicles to detect the surrounding objects in actual driving scenarios.This article probes into how images and point cloud are integrated through technical analysis and architectural design.Firstly,the system would obtain the position of target objects in the image;then,it obtains corresponding point cloud data through internal and external parameters between sensors and conversion matrix of spatial position.Subsequently,the system would perform multiple convolutions on corresponding image data and point cloud data and integrate their characteristics after every convolution.Based on the integration of images and point cloud data,this method proves effective in system testing.Compared with other systems,it can achieve an increase in operation speed besides guaranteeing high accuracy and robustness.In addition,this system provides a visualization module of test results,which greatly facilitates the troubleshooting and software daily debugging in the system.