Automatic Driving Scene Prediction And Semantic Understanding At Night

An automatic driving car is a wheeled robot that can sense its surroundings and implement navigation without human operation.In recent years,with the continuous innovation and development of robotics technology,automatic driving cars have not been fully commercialized,but the idea of self-driving cars has gradually become a reality,which has led to a lot of discussions on vehicle safety.At night,due to insufficient light,people's ability to recognize surrounding scenes is greatly reduced,and the night driving safety of automatic driving cars is also attracting more and more attention.The ability of a automatic driving car to recognize surrounding scenes at night is the key to its ability of driving safely at night.Therefore,research on how to improve the ability of automatic driving cars to sense night vision images is of great significance to the driving safety of night-time automatic driving cars.During the day,ordinary cameras can easily capture the pictures of scenes.However,at night,due to insufficient light,ordinary visible light imaging cameras cannot perform this task.Therefore,an infrared camera is usually used to capture images at night.Compared with ordinary cameras,it can work in dark,and form an image with different gray values using temperature differences of different objects.Due to its special imaging mechanism,it has been used widely with good performances in the military,medical,and industrial applications.But it has some disadvantages compared with visible light image,such as lack of color information,low contrast,and low signal-to-noise ratio,which make the understanding of night vision images very challenging.In order to enhance the automatic driving car's understanding of the scenes around at night,We aim to predict the changes of the surrounding environment more effectively and accurately and express it through fluent language so that the car can make timely adjustments.In this paper,we study the scene prediction and semantic understanding of automatic driving cars at night based on the imaging properties of night-vision images.The main research contents of this paper includetwo parts: the first part is the prediction algorithm of night-vision image scene based on predictive coding network;the second part is the semantic recognition algorithm of the night-vision image based on deep convolutional neural-long-term and short-term memory network.The main innovations of the dissertation include the following two points:1.Deep learning is applied to scene prediction of night-vision images.The scene changes in night-vision images are predicted using a predictive coding network.We make some adjustments to the network structure based on the traditional deep convolution-recurrent neural network.The error of the predicted image and the actual image is forwarded in the network,and the prediction error is updated constantly to adjust the prediction result.The trained scene prediction model can predict the reasonable future of the night driving scenario after 0.4s,and improve the poor performance for long-term prediction tasks,which allows enough time for driverless cars to make appropriate decisions in time.The experimental results demonstrate the accuracy and real-time performance of the method of night-vision image scene prediction in this paper.2.In order to enhance the automatic driving car's understanding of the predicted night-vision image during night driving,if we can use the machine to read the night-vision image and express the content of this image through fluent language in the process of driving the cars,it will help the car or the passengers in the car to understand the information transmitted better and faster by the predicted image.We propose a semantic understanding method of night-vision images based on convolutional neural network-long-term and short-term memory networks.Firstly,features are extracted by the convolutional neural network,and corresponding semantic understanding statement is generated by long-term and short-term memory networks.The network uses an end-to-end training mode to output the words that have the highest probability of matching the characteristics of the original night-vision image to form a semantically interpreted sentence.Experimental results show that the method has good accuracy and real-time performance.