Multi-Feature Extraction And Fast Image Matching Based On CNN

Image matching is the process of finding the same regions of different images through feature detection and description,to complete tasks such as recognition,registration and retrieval.This thesis mainly focuses on the degradation of SIFT,a traditional image matching method,in visual navigation for unmanned aerial vehicle under difficult conditions such as image blur.Considering that the learning-based descriptors using convolutional neural network(CNN)have larger description regions and strong adaptability to difficult conditions,thus complement the performance of hand-designed SIFT descriptor,this thesis combines traditional and deep features via the SIFT-HardNet method,and propose to solve its three problems,consisting of the lack of scale invariance and rotation invariance,as well as the slow calculation speed of the HardNet floating-point descriptor.The main work is in the following:(1)Two methods are proposed to enhance scale invariance of the HardNet network.The first is the SIFT-HardNet method combined with SIFT Difference of Gaussian(DOG)pyramid,referred to as DOG+SIFT-HardNet.This method calculates the size of input patch of HardNet through the DOG scale feature,so that the scale of image can be effectively described by HardNet.Experimental results show that DOG+SIFT-HardNet achieves the same scale invariance as SIFT,and the matching performance is better than SIFT.The second method is multi-scale dataset augmentation of HardNet.In this thesis,aerial images are taken as sampling objects to make positive and negative sample sets of different scales.In order to ensure the dissimilarity of the sample sets,a feature point selection method based on coordinate distance and response value is proposed to remove redundant feature points.Experimental results show that the augmented dataset method can enhance the adaptability of HardNet to scale change.(2)Two methods are proposed to enhance rotation invariance of the HardNet network.The first is the SIFT-HardNet method combined with the SIFT rotation angle,referred to as rotation+SIFT-HardNet.In this method,the input patch of HardNet is rotated and then cut according to the feature of rotation angle.Experimental results show that rotation+SIFT-HardNet acquires rotation invariance and the matching performance is better than SIFT.The second method is the multi-rotation angle augmentation dataset of HardNet,which produces positive and negative sample sets with rotation relation.The experimental results show that the HardNet trained by the multi-rotation angle augmentation dataset can adapt to rotation change better.In addition,the DOG+Rot+SIFT-HardNet method and the multi-scale and multi-rotation angle dataset augmentation of HardNet are proposed,which combine scale invariance with rotation invariance.The experimental results show that the combination methods enhance both scale invariance and rotation invariance.(3)Two binary quantization methods of floating-point descriptor of HardNet are proposed.The first is a median quantization method that generates 128-dimensional binary descriptor.Floating-point descriptor is quantized as ’0’ or ’1’ with the median value as the boundary value.The second is a three-quantile quantization method that generates 256-dimensional binary descriptor.the three-quantile points are used as cut-off points to quantify the floating-point descriptor as "00","01" or "11" according to the size.Experimental results show that the matching accuracy rate of the 256-dimensional binary descriptor is closest to that of the floating-point descriptor(only 1.69%less),and the speed is increased by 4.17 times.In addition,the matching accuracy of 128-dimensional binary descriptor is low,but the matching speed is the highest,which can reach 4.41 times that of floating-point descriptor.Therefore,the 256-dimensional binary descriptor is an effective solution to ensure the matching accuracy and improve the matching speed.