A Real-Time Traffic Light Recognition Method Based On Convolutional Neural Network

Traffic light recognition which consists of detection and state recognition plays an important role in intelligent traffic system and is an important perception module of self-driving cars.In the actual road environment,there are various types of traffic lights,which are small in scale compared with vehicles,pedestrians and other objects,thus increase difficulties to the traffic light detection and recognition.This paper proposes a traffic light recognition method based on convolutional neural network,which has the following characteristics: it can detect far away and small-scale traffic lights,identify multiple types of traffic lights,and can track traffic light.The work of this paper is as follows.In terms of traffic detection,this paper is based on YOLOv3 algorithm,by improving the way of feature extraction and feature fusion adjusting and scale,optimizing the loss function to improve the traffic detection effect.First,we reduce the down-sampling rate of backbone network to increase the feature expression ability of small-scale objects.At the same time,feature pyramid pooling is introduced to integrate local features and global features,which increases the expression ability of network feature extraction.The experiment result shows that feature pyramid pooling can improve mAP value by about 2%.Second,the dual feature fusion which add a set of feature maps can promote the effective fusion of early feature and later feature.Early feature has high location information and later feature have high semantic information.The final result shows that the dual-feature fusion can bring about an improvement of mAP value of about 4%.At the same time,increasing the size of fusion features is helpful for the detection of small-scale traffic lights.Last,GIoU is introduced as the loss function of the detection task.Compared with the mean square error of the center point coordinates of bounding box or the IoU,GIoU pays more attention to the degree of non-overlap between objects.In addition,rectangle similarity combined with GIoU is proposed as a new loss function to improve the regression effect of the bounding box.And result show that new loss function can improve the mAP value of traffic light detection by 1%～2%.In order to meet the real-time requirements of the automatic driving perception module,we use some tricks to make an attempt on improvement of the inference time,eg: the number of detection heads is increased or decreased,the resolution of the input image is changed,and introduce pruning.The experiment shows that small backbone networks can also achieve remarkable results in accuracy and inference speed with high input resolution and multiple detection branch.The more detection branch lead to the slower the inference speed and the higher the detection accuracy and higher input resolution can also improve the accuracy of detection.Pruning can significantly reduce all of parameters,FLOPs and inference speed of the networks.The degree of decreasing become more and more with the increase of pruning rate,and the accuracy rate is basically unchanged.In the aspect of traffic light state recognition,the method of color and shape constraint is used to identify the state and verify the category of the traffic light and its act on the detection results as the post-processing algorithm.Ift can effectively improve the recognition accuracy of red,yellow and green traffic lights.In addition,a classified CNN model is designed,which is trained separately and added to the object detection algorithm.The CNN classification model achieved more than 99% accuracy in the self-built train set and test set.The experiment result shows that the state verification after the object detection is effective for the recognition of traffic lights.In order to stabilize the traffic light detection and recognition results,we use a motion model to estimate the movement of traffic lights relative to the ego car to make stable detection of bounding box,so as to predict the position of the traffic light in the next frame image.And a neural network is used to correct the box’s coordinates.This tracking method has the ability to track traffic lights more than 10-pixels width.Figure [35] table [7] reference [58].