Multi-source Remote Sensing Image Object Detection Based On Transfer Learning

Remote sensing image object detection aims to locate and distinguish typical ground objects,and plays an important role in national defense,military,intelligent surveillance.With the development of high-resolution earth observation technology,the increasing mass of high-resolution remote sensing image（HRRSI）data has brought new opportunities and challenges for remote sensing image object detection.Currently,the HRRSI fully supervised object detection methods represented by deep learning is booming,but there is a limitation that a large amount of real data needs to be labeled as training data.Compared with time-consuming manual annotation,3D rendering technology has the advantages of high generation efficiency and accurate annotation.It provides a promising method for automatically obtaining synthetic images and labels.However,due to the domain gap between synthetic data and real data,the object detection model trained with synthetic data performs not very well in real scenarios.In view of the above background,this paper explores the generation and use of synthetic data for HRRSI object detection task,and carries out cross domain object detection research based on transfer learning at the pixel level and feature level respectively,which improves the performance of object detection models on labelscarce remote sensing images.The main innovations of this paper include:（1）This paper proposes a 3D rendering-based HRRSI object detection sample automatic generation framework.We build rendering systems using the Three.js and GTA V engines,respectively,to generate synthetic samples with rich content and a wide distribution of object sizes.Simulating the unique imaging perspective of HRRSI,the Synthetic N and Synthetic U synthetic aircraft datasets and the GTAV10 K synthetic vehicle dataset are constructed to provide training data for cross-domain object detection experiments.（2）This paper proposes an image translation method with a multi-scale attention（MSA）mechanism for generative adversarial networks.This paper analyzes the texture distortion and noise phenomena in the output images of Cycle GAN（Cycle-Consistent Adversarial Networks）.Assuming that the problem of texture conflict is caused by the limitation of network receptive field,we design a multi-scale spatial and channel attention mechanism to enhance the global semantic feature representation.Experiments on three datasets show that the proposed image translation method can refine the style of synthetic images,thereby improving the performance of the object detection model.（3）This paper proposes a domain adaptive object detection method based on domain specific channel recalibration（DSCR）module.Different from the way of fully sharing the backbone convolutional network in existing methods,for the source and target domain data with large distribution differences,this paper uses a domain-specific network branch to process the domain-specific information and convert it into domainshared features.To obtain the respective channel attention of the source and target domains separately,we try to insert the DSCR modules in different depths of adversarial feature alignment network.Experimental results on synthetic-to-real and cross-dataset adaptation confirm the effectiveness and generality of our method.