Locating And Splicing Method Of Traffic Monitoring Video Based On Image Semantic Segmentation

The objective world is multi-dimensional,and one of the most important dimensions is the dimension of time,which can be reflected in smart cities is real-time data,including real-time perception data of the Internet and real-time detection data of the Internet of Things.Without the support of real-time data,the decisions made on this basis are often not scientific and real-time enough,which leads to the decision-making can not effectively guide the actual life and work.Traffic surveillance video is an important dynamic real-time data in smart city data.Through positioning and splicing of traffic surveillance video,it can restore the objective world dynamically in three-dimensional GIS platform,which has important application and research significance.The positioning of traffic surveillance video,which essentially adds geographic coordinates to the video image,is a key link in integrating into three-dimensional GIS scene.Traditional video image mapping models require camera equipment to have positioning function,such as GPS,gyroscope,etc.The application of data without positioning equipment such as traffic monitoring video is limited.Video splicing is the extension of image splicing and plays an important role in scene monitoring,target recognition and other applications.Traditional video splicing algorithms require large overlap areas between videos and only consider image geometry features in feature point matching process.When dealing with traffic surveillance videos,the overlap area between different cameras is very small or the angle between main optical axes is large,which will result in large splicing or image distortion.In order to solve these two problems,this paper presents a traffic monitoring system based on image semantics segmentation by using the theory of Digital Orthophoto Image data and deep learning semantics segmentation,which utilizes the position information of each pixel of Orthophoto Image and has a wide coverage.Based on the research of Orthophoto Image Search theory,deep learning semantics segmentation theory,SIFT feature extraction and matching theory,a traffic monitoring system based on image semantics segmentation is proposed.Video positioning and splicing methods.First,the second-order difference histogram algorithm of edge angle is used to automatically identify the ortho-image in the multi-video intersection area as a splicing background image;then,the ortho-image and video image are semantically segmented based on the multi-feature semantics segmentation method composed of full convolution neural network and BP neural network to extract the traffic topic semantics in the image;next,the traffic topic is taken as the traffic topic.Semantics are used as constraints to match feature points,and each traffic monitoring video is matched to background orthophoto achieve video splicing in monitoring area while locating.The actual video data of a city in Shandong Province is used for experimental verification.The results show that for monitoring video with small overlap area,the method in this paper can achieve a good splicing image,at the same time,it can effectively improve the accuracy of feature point matching and better complete the positioning task.This paper mainly does the following work:(1)Research the orthophoto search method based on edge angle second-order difference histogram,and summarize the orthophoto position search method suitable for retrieving corresponding areas of traffic monitoring video.(2)Research on image semantics segmentation method based on BP neural network and full convolution neural network,improve a multi-feature depth learning video image semantics segmentation frame,and determine a multi-feature depth learning semantics image segmentation method suitable for the video by combining the characteristics of traffic monitoring video.(3)Similarity measurement of SIFT feature matching based on video image semantics is studied,and feature point matching is performed by combining image semantics and SIFT features to optimize the accuracy of feature point matching for traffic monitoring video and orthophoto image.(4)In order to verify the validity and rationality of this method,locating and splicing experiments are carried out using real data such as partial traffic monitoring videos and orthophoto images in Linyi City,Shandong Province.Semantic segmentation accuracy verification experiments,feature point matching reliability verification experiments and video location and splicing results validity verification experiments are designed and compared with traditional methods.