Research On Deep Learning Object Interpretation Method In High-Resolution Remote Sensing Images With Complex Scenes

With the development of high-resolution remote sensing satellites,high-resolution remote sensing images have become easier to acquire and have contained a large amount of ground object information.Thus,it is essential to interpret and make full use of this information.High-resolution remote sensing image object interpretation aims to automatically obtain the category and location information of interested objects from high-resolution remote sensing images,which has been widely applied in traffic monitoring,urban planning,national security,and other fields.However,influenced by the shooting angle and spatial resolution of high-resolution remote sensing images,the remote sensing objects are typically embedded in complex scenes with intricate object spatial patterns and massive background noise and exhibit significant differences in scale,angle,and appearance,which dramatically increases the challenge of object interpretation of high-resolution remote sensing images.Therefore,studies on effective methods to address the challenges in the high-resolution remote sensing object interpretation field are the top priority.Based on the above analysis,this thesis focuses on challenges in high-resolution object interpretation,including high background complexity,tiny objects detection,few-shot object detection,and object refinement interpretation,and carries out research on deep learning object interpretation in high-resolution remote sensing images with complex constrained scenes.Benefiting from the deep learning techniques,we aim to build deep detectors to improve detection accuracy and adaptability.The main contents and contributions of this work are given as follows:(1)Considering the high background complexity in high-resolution remote sensing object detection,this work proposes a refined rotation detector,namely,the foreground refinement network,to alleviate the above problem by leveraging the information of foreground regions from the perspectives of feature representation and network optimization.From the perspective of feature representation,we propose a foreground relation module that aggregates the foreground-contextual representations from the coarse stage and improves the discrimination of foreground regions on feature maps in the refined stage.From the perspective of network optimization,we design a foreground anchor reweighting loss that integrates the classification confidence and localization accuracy of each foreground anchor from the coarse stage to dynamically regulate their contributions in the refined stage,highlighting the potential foreground anchors.Extensive experiments demonstrate that the proposed method can effectively increase the response of the foreground regions and suppress the background noise interference.(2)Considering the unsatisfactory performance of tiny object detection in high-resolution remote sensing images,this work proposes a multi-stage enhancement network that achieves the instance-level and feature-level enhancement of tiny objects from different stages of the detector.Since the Io U-based label assignment drastically deteriorates the positive samples for tiny objects,a central region-based label assignment is proposed to substitute the Io U-based label assignment in RPN,which provides more positive samples for tiny objects.Then,we design a gated context aggregation module that selectively aggregates valuable context information to enhance the feature representation of tiny objects.Additionally,we devise a positive Ro I feature generator to generate a rich diversity of high-quality positive Ro I features for tiny objects.Extensive experiments on remote sensing tiny object detection datasets verify that the proposed method can effectively improve the performance of tiny object detection.(3)The current state-of-the-art remote sensing object detectors typically need sufficient welllabeled samples to achieve good performance and are prone to overfitting with limited samples.To address the above problem,this work proposes a generalized few-shot object detection method for remote sensing images.Firstly,based on the comprehensive analysis of each component in the detector,we present an efficient transfer-learning framework as the foundation of our method,which is more suitable for few-shot object detection in remote sensing scenes.Then,considering the greater intra-class diversity and lower inter-class separability of geospatial objects,we design a metric-based discriminative loss to learn a more discriminative classifier in the few-shot fine-tuning stage.Furthermore,a representation compensation module is proposed to alleviate the catastrophic forgetting problem by decoupling the representation learning of previous and novel knowledge.Comprehensive experiments on different few-shot settings demonstrate that our proposed method achieves competitive novel class performance with minor degradation in the base class.(4)Considering the class confusion and class sample imbalance problems in fine-grained remote sensing object detection,this work proposes a dual indicator network that leverages the learning state of the network for different fine-grained classes as the indicators to strengthen the network’s attention on the confused classes and the under-sampled classes.Since the class confusion problem usually occurs between more similar classes,a gradient boost loss function is devised.The gradient boost loss takes the prediction probabilities at the current moment as a short-term indicator to mine the confused classes with the ground truth class and strengthens the learning of these confused classes.To address the underfitting problem of under-sampled classes due to the class sample imbalance problem,a memory-based feature generator is proposed,which takes the cumulative learning quality of the network for each class in the training stage as a long-time indicator to guide the generation of features for each class,which enhances the feature representation of the network for under-sampled classes.Experimental results show that the proposed method can improve fine-grained object detection performance.(5)Since remote sensing object detection can only acquire region-level location information of objects,this work concentrates on a new and challenging problem of instance segmentation in remote sensing images,which can provide pixel-level object location information.Considering the complex background interference and huge scale variances of instances in remote sensing image instance segmentation,this work proposes a semantic attention and scale complementary network to tackle these problems.Firstly,a semantic attention module is proposed,which introduces semantic segmentation labels to guide the generation of semantic attention maps.The generated semantic attention maps are then fused with multilevel feature maps to strengthen the activation of interest instances on the feature map and reduce the background noise interference.Besides,we propose a scale complementary mask branch to learn different scale instance features with a multi-branch parallel structure and to fuse the multi-scale complementary information through a feature fusion module.Extensive experiments verify the effectiveness of the proposed method in remote sensing image instance segmentation.