The Application Of Regional Convolutional Networks And Generative Adversarial Models To The Interpretation Of Remote Sensing Image

Deep neural networks are booming again with the support of big data.It has made great progress in solving computer vision tasks for which labeled data is plentiful.However,there is a lack of labeled datasets in the field of remote sensing image.How to use deep neural network to learn representation and interpret it with lack of labeled data becomes one of research focuses in the field of remote sensing.SAR image and optical remote sensing image are two common remote sensing data.The segmentation of SAR image contains several spatially disconnected heterogeneous regions whose segmentation suffers the imbalance of samples.Besides,in the dataset of optical remote sensing detection,some objects’ backgrounds are monotonous for the lack of data,which causes the poor performance in test set for a number of false alarms.Finally,the common single-stage object detection model faces the dilemma of classification accuracy and localization accuracy because little attention is paid to the global shape of object.Based on the discussion above,we investigate the application of generative adversarial networks(GANs)in the generation of heterogeneous regions in SAR image,the application of GANs in the generation of object detection samples in optical remote sensing image and the application of neural network in object detection of optical remote sensing image.The main contents are as follows:1.We propose sketch-and-structure-constrained GANs for the generation of heterogeneous regions in SAR image towards the imbalance of heterogeneous regions’ samples.In this thesis,a GAN is built to learn the joint distribution between SAR image and SAR sketch for heterogeneous regions in SAR image.In order to avoid distortion of generated SAR image,we propose a sketch loss function to constrain the global structure of the generated image,and use the geometry window loss function to constrain the local structure of the image.The experimental results show that the method can generate realistic SAR image with the same structure of SAR sketch.2.We propose an object pasting neural network for the generation of object detection samples in optical remote sensing images to improve detection model’s robustness on different backgrounds.Based on the existing object detection dataset of optical remote sensing image,we build a object pasting network inspired by the additive noise model,and implement the generation of objects with variable scale at the specified positions of the background image under the premise of only objects’ classes and bounding boxes.A training method for the double discriminator is proposed to ensure that the object in the generated mixed image does not lose diversity.The experimental results show that the sample generation method can realize the generation of multi-scale objects with position customization under the premise of given background.3.We propose an optical remote sensing image object detection method based on regional convolutional neural network towards the little focus on global shape of object for detection model.In order to make the convolutional neural networks sensitive to the boundary of objects,we build a Do G ridgelet branch using a Difference of Gaussians(Do G)ridgelet basis function to generate edge-and ridge-sensitive Do G ridgelet filters as convolution kernels.The Do G ridgelet branch learns features together with the general convolution kernel.At the same time,in order to enhance the response of the convolutional feature map at the object’s structural region,we use the primal sketch of original image to obtain the structural region of original image.The convolution kernel can be convolved separately to the structural region to obtain an enhanced feature map.The experimental results show that the proposed method can effectively improve the localization accuracy of the optical remote sensing object detection model.