Real-time Monocular Multi-object Perception In Complex Driving Situation

Accurate and comprehensive environmental perception is the basis of intelligent vehicles to achieve autonomous driving.Environment perception algorithms for intelligent vehicle are mostly designed for a single task or single object,and cannot fulfill the requirements in complex traffic scenes.However,for existing multi-task and multi-object environment perception algorithms,the accuracy and real-time performance of are often difficult to balance.To this end,this paper proposes a real-time multi-object joint perception framework based on vehicle monocular vision,designing a convolution neural network structure with shared feature extractor and multiple sub-task branches,which can simultaneously perform real-time online inference of 2D object detection,3D object information estimation and road segmentation,and then achieve 3D vehicles,pedestrians and cyclists detection and road detection as well.Firstly,the convolution neural network structure of the multi-task joint perception framework is designed.The full convolution neural network is used as shared feature extractor to encode the 2D image features.Multiple independent decoding branches are used for the predictive output of each sub-task.Secondly,improve the existing singlestage object detection method to achieve fast 2D object detection.Aiming to address the problem that pedestrians and cyclists are easy to misdetect each other,a hierarchical classification strategy is proposed.For the problem of mutual occlusion of objects,a soft non-maximum suppression post-processing algorithm is adopted.Thirdly,in order to achieve the 3D reconstruction of the object,a plurality of 3D information estimation methods are designed to obtain the 3D position,posture and size of objects.A method for indirect acquisition of object heading angle by regression object observation angle is proposed.A method of order regression and compensation term regression based on spatial incremental discretization is proposed to estimate the longitudinal distance of the targeting 3D bounding box's center.To retrieve the lateral position,the 2D projection of the 3D object center of is estimated,and then the projected center is extruded into real3 D space with camera calibration parameters and aforementioned estimated longitudinal distance.To obtain the 3D size of the object,a method of estimating the deviation of the true object size and the default off-line statistics of datasets is proposed.Finally,in order to achieve accurate road segmentation,a road segmentation method based on edge optimization and method of road geometry deformation data augmentation is proposed.In order to verify the effectiveness of the on-board monocular real-time multi-object joint perception method proposed in this paper,several experiments are carried out on the KITTI dataset and Tsinghua-Daimler Cyclist Benchmark.The experimental results show that the joint perception framework can perform accurate vehicles,pedestrians,cyclists detection and road segmentation at the same time.The produced results rank first in the KITTI dataset public rankings over several evaluation metrics.Besides,on the platform of GPU,when the image is input with 1242 × 375 pixels,the average processing time of a single image is about 65 milliseconds,which demonstrates the real-time perception ability of this approach.