Research On Coverage Summarization Of Unmmaned Aerial Vehicle Based On Convolutional Neural Network

Unmanned aerial vehicle(UAV)video image is widely used in more and more fields.The technology used for processing UAV video images includs stabilization,matching and stitching,target detection and tracking,event understanding and so on.This thesis focuses on the topic of summarizing UAV video image to realize the overall description of spatial coverage in the sensing area.This technique is also commonly referred to as spatial coverage summarization.The theoretical basis of the spatial coverage summarization is image panoramic stitching,which is made up stitching images,and the key technique is image matching.The main methods of current image matching are to design and extract specific local features,and then get the matching results by comparing the local features extracted from two images.This kind of common methods have obtained very good matching result.But it also faces the obvious negative impacts from the parameters of the used algorithm and the limited ability to tolerat the uncertainties of the complex UAV video imaging conditions.The negative impacts will decrease the performance of local feature extraction.In addition,the matching of feature points usually adopts the linear distance metric as the similarity measure,and can not adapt to the complex transformation relationship between images.To solve these problems,this thesis focuses on the CNN-based image representation and comparison method,and then proposes the technique of UAV video spatial summarization based on the CNN feature representation and comparison.The thesis mainly consist of the following three aspects of research.Firstly,a CNN-based UAV video image comparing system is proposed.A new CNN-based feature extraction and comparing model is constructed,and the image comparing task is deemed as a two-class classification problem.The model structure is composed of two-channel CNN feature extraction network,similarity metric network,classification layer and so on.The constructed model allow all network layers to be learned end-to-end through the usual back-propagation algorithm.The proposed model has the following advantages:(1)It extract the image feature through the CNN network and the model parameters can be automatically determined by the learning method.These characteristics mean the model is suitable for the uncertainty of complex UAV imaging condition.(2)The joint learning method simultaneously trains CNN representation network,similartiy metric network and classification network,making full use of the correlation between the various parts to improve the performance of the model.In addition,the thesis proposes a transfer learning method,which can capture the useful information in different databases to improve the image matching performance of the proposed model.Secondly,an image summarization method is proposed based on the CNN representation and matching method.Since the matching of the CNN based UAV video image comparing system is performed over the image patches,it is difficult to get the stitching results with the pixel or sub-pixel accuracy through directly using the CNN based matching result.To this end,this thesis proposes a local salient feature points guided image stitching method based on the CNN image representation and matching.By using the localization performance of the local salient feature points and the excellent representation performance of the constructed CNN-based comparing model,an accurate spatial coverage summarization of the UAV video images will be obtained.Finally,a system for UAV spatial coverage summarization based on CNN is constructed.Using the previous proposed methods,the thesis develops a preliminary demonstration system using the Visual Studio C++ program language.Through the reasonable planning,the main functions of the developed system include video playback and display of the spatial coverage summarization.The developed system can demonstrate fully the research methods and their potential.