Low-latency Video Semantic Segmentation Algorithm Based On Deep Learning

In recent years,due to the development of deep learning in the field of image,semantic segmentation has made remarkable progress.However,how to guarantee the real-time performance of semantic segmentation algorithm under video scene task,such as automatic driving,is still a challenging problem.Most of the existing image semantic segmentation methods have high algorithm delay.Generally,it is difficult to meet the requirements of high throughput video data in practical applications.By using the timing and redundancy of video data,we can provide more feature information for the task of semantic segmentation,and reduce the computational overhead,which is conducive to improving the accuracy and real-time of video semantic segmentation.Its core is efficient feature information extraction,accurate key frame matching,and flexible inter frame feature propagation.In view of these problems,this paper has carried out theoretical analysis,method research and data validation,and other work.The main innovations are as follows:1.This paper proposes a high-performance method of building basic partition network.Using resnet-34 combined with the spatial attention mechanism module(SAM)and channel fusion module(CFM)proposed in this paper,it solves the problem that it is difficult to capture rich context information in ordinary segmented networks,and can effectively combine the characteristics of different levels.While improving the ability of the network to deal with the details,the reasoning delay of the algorithm is effectively reduced.2.A key frame scheduling strategy that can adapt to the rapid change of video content is proposed.By modeling the similarity relation of adjacent frames at pixel level,the optimal key frame judgment criterion is derived.The process of fitting this similarity measure with small CNN network is used to make it efficient and easy to realize.It solves the problem that the update rate of key frame in the previous keyframe recognition method is difficult to be consistent with the change of video content.3.A non-keyframe adaptive feature propagation method is proposed.By constructing a small network with attention mechanism,the spatial feature correlation between non-key frame and key frame is established,and global weight adjustment is carried out on the basis of the feature of key frame,so as to obtain the high-level semantic feature of the current frame.It solves the problem that the current feature propagation method can not flexibly conduct the subtle changes of inter-frame features and realizes the more accurate and efficient utilization of the existing features.The methods proposed in this paper have been verified by experimental comparison,and the results show that these methods can better solve the problem that accuracy and low latency cannot be combined in semantic segmentation,and achieve an average of 76.8% m Io U and 26.8 fps performance on Cityscapes dataset.