Research On Obstacle Detection And Distance Estimation In Surrounding Fisheye Camera Based On Deep Learning

With the increasing number of cars in the country,the traffic accidents of vehicles are also increasing.The results of the study show that the autonomous driving technology of vehicles can effectively reduce the number of traffic accidents.As the first step of autonomous driving technology,environmental perception plays a crucial role in subsequent behavioral decision-making and motion control.With the popularization of the panoramic surround view system,some breakthroughs have been made in the environment perception technology based on the fisheye camera and the panoramic view.However,there are still some problems in this field,such as poor target detection effect based on fisheye image,cumbersome process of distance estimation algorithm,slow running speed of model algorithm on embedded development board and so on.Aiming at these problems,this paper proposes a series of models and methods.Specifically,the main research contents and results of this paper are as follows:(1)A dataset of vehicle fisheye images(FE＿V)suitable for multiple tasks is created.In view of the problems of low quantity and low quality of the current vehicle fisheye image dataset,this paper collects and produces the FE＿V dataset based on various scenes through real vehicles.The FE＿V dataset has five categories such as cars and pedestrians,and has various labels such as rectangular boxes,rotated rectangular boxes,and semantics.The dataset can be applied to single-task or multi-task of object detection and semantic segmentation,such as vehicle and pedestrian detection,traffic light detection,and vehicle and pedestrian semantic segmentation.(2)Two improved fisheye image object detection models are proposed.In view of the relatively poor effect of fisheye image target(obstacle)detection,a rotation model R-YOLOv5-reg based on angle regression is proposed based on rotation bounding box and yolov5.Since the angle regression method may have the problem of abnormal angle boundary,R-YOLOv5m-cls based on angle classification and Circular Smooth Label is proposed.The experimental results on the Woodscape and FE＿V datasets show that:compared with YOLOv5 m,the m IOU and instance m AP of R-YOLOv5m-reg are increased by 4.55% and 6.84%,respectively,the m IOU and instance m AP of RYOLOv5m-cls are improved by 4.76% and 10.82%,respectively.The experimental results show that the two proposed models achieve excellent detection results on different datasets,and R-YOLOv5m-cls successfully solves the boundary anomaly problem of angle regression.(3)A distance estimation method for targets or obstacles is proposed based on a surround-view fisheye camera and panoramic surround-view images.The distance estimation of obstacles is performed by combining the lower edge of the rotated bounding box in the improved R-YOLOv5m-cls model with the panoramic look-around image.First,the lower edge of the rotated prediction frame is extracted and mapped to the panoramic view image,and then the distance estimation of the target is performed in the panoramic view image.The verification and error analysis are carried out through experiments,and finally the error between the estimated distance of the target and the actual distance is only 4.749%.(4)The proposed model algorithm was transplanted to the embedded development board Jetson TX2,and the inference was accelerated by the TensorRT inference engine.The improved R-YOLOv5-cls model was ported to the TX2 development board,and the detection of the model was accelerated by TensorRT for the slow running speed of the model.Finally,the detection speed accelerated by TensorRT can reach 25 frames per second,which can basically meet the requirements of practical applications.