| Deep learning-based object detection algorithms can achieve excellent performance.Such high-performance detection algorithms,when deployed on board,can carry out fast and efficient analysis of visible satellite images on-orbit.This would significantly increase the timeline of observation tasks and improve the utilization of satellite-ground communication bandwidth,which has important research and application values.Considering the limitation of computing resources and the impact of spatial environment,running deep learning-based object detection algorithms directly on board faces many new challenges.In this paper we research on the on-orbit deep learning-based object detection technology of visible satellite images from three perspectives: algorithm design,software implementation and hardware development.The specific research contents of this paper are listed as follows:(1)Object detection algorithm design for raw data of remote sensing payloadThe raw images of remote sensing payloads that have not been preprocessed could have potential defects such as noise or geometric distortion.In this paper we propose an improved YOLOv5 algorithm,which achieves an over 35% recall rate increase of airplane and ships targets in raw images through modification of data input and optimization of feature extraction layer in YOLOv5.We adopt JPEG2000 compression as a preprocessing step,which allows detection algorithm for handling huge amount of payload raw data.With the combination of compression and detection algorithms,we can realize fast object detection of payload raw data.(2)On-board object detection algorithm improvement through satelliteground collaborative computingTo enhance the generalization performance of the on-board object detection algorithm,we borrow the idea of semi-supervised learning to realize the domain adaptation of on-board algorithm,and propose an automatic improvement method for deep learning-based object detection model through cloud-device collaborative computing.This novel method consists of data filtering on-board and domain adaptation on the ground,and constructs a closed loop for on-board algorithm selfupdating through the collaborative computing of on-board computer and cloud server on the ground.The detector’s cross domain adaptation experiments based on DOTA and DIOR datasets show that our method can significantly improve the performance of on-board detection algorithm with an over 6.1% m AP increase.(3)Fault-tolerant deep learning-based object detection softwareDeep learning algorithms directly deployed on board are easily affected by spatial environment induced transient effects such as SEU,which could result in silent data corruption.In this paper we conduct large amount of fault injection experiments to analyze the fault-tolerant performance of commonly used CNN-based object detection algorithms from multiple perspectives.We further propose a model-layer selective hardening method AMHR,which realizes the searching and hardening of most error sensitive kernels in a CNN model through the training of a reinforcement learning agent.The output corruption rate for AMHR hardened object detection model is only 30%compared to original model,and it outperforms other model selective hardening methods.(4)Development and verification of a spaceborne AI computing platformTo verify the feasibility of on-board application of commonly used AI processors,we developed a heterogeneous multi-core AI computing platform and installed it on a scientific satellite SATech-01.This computing platform integrates multiple AI processors with various architectures such as GPU and ASIC.After the successful launch of SATech-01,we’ve carried out preliminary tests for the processing platform where AI processors are required to inference on pre-stored images with pre-trained deep learning models.The results of the tests show integrated AI processors can correctly execute the algorithm and output expected results,which proves the basic feasibility of on-orbit application for those AI processors. |
