Hardware Implementation Of Lightweight Model For Semantic Segmentation Of Remote Sensing Images

Micro-nano satellite remote sensing technology has become a research hotspot in recent years because of its low cost,fast deployment,and the ability to achieve large-area and refined distributed collaborative remote sensing detection tasks through cluster network observations.Image semantic segmentation,as one of the hot research directions in the field of computer vision,has been used more and more in the field of remote sensing image processing,and has been gradually deployed on micro-nano satellites to provide users with low-latency,highefficiency edge End-to-end semantic segmentation service.Traditional semantic segmentation methods have low segmentation accuracy and poor algorithm adaptability,which cannot meet the application requirements of high-performance and refined image semantic segmentation.In recent years,semantic segmentation technology based on deep learning has developed rapidly,which has solved the problems of low precision and poor adaptability in traditional algorithms.However,with the continuous improvement of semantic segmentation accuracy,the amount of parameters and calculation of the algorithm model is also increasing.However,the size and power consumption of micro-nano satellites are strictly limited,which makes it difficult to deploy semantic segmentation models with excellent performance on micro-nano satellites.To address this problem,this paper proposes a lightweight semantic segmentation convolutional neural model that combines multi-scale feature fusion,grouping downsampling,and super-resolution auxiliary subnets.The model is evaluated in terms of segmentation accuracy,computational complexity and detection speed on the i SAID remote sensing data set,and the hardware acceleration of the algorithm inference network is implemented on the edge FPGA suitable for the micro-nano satellite platform.The main contributions of this paper are as follows:(1)In view of the large amount of parameters and calculation of the existing high-precision semantic segmentation models such as UNet and Deep Lab,which lead to the problems of high resource consumption and poor real-time detection in hardware implementation,this paper proposes a light-weight algorithm based on multi-scale features.The magnitude semantic segmentation model is called Sub-GDUNet.The method adopts the classical encoderdecoder structure in the field of semantic segmentation,and performs multi-scale feature fusion by means of skip connections.In addition,inspired by the grouping convolution method,this paper also proposes a grouping downsampling module,which performs channel grouping differential downsampling on the feature map,using maximum pooling and stride channel-by-channel convolution,respectively,through channel splicing.This enables the model to extract richer spatial location information.The experimental results show that compared with the UNet network,when the pixel accuracy is only reduced by 0.32%,the amount of parameters is reduced by 66.51 times,and the amount of calculation is reduced by 67.95 times.(2)Aiming at the problem of spatial feature loss due to linear interpolation at the decoding end of the model,a super-resolution auxiliary subnet structure is proposed in this paper.The structure uses the classical super-resolution method to constrain the feature layer of the model’s sub-high resolution,and guides it to learn finer spatial features,thereby making up for the defect of insufficient spatial information when restoring feature maps with larger resolutions.The super-resolution auxiliary subnet only guides the learning of the backbone network during training,and does not participate in the operation of the inference stage.Therefore,the false alarm rate of the Sub-GDUNet network can be reduced by 8.95% without increasing the complexity of the network inference,the segmentation accuracy is improved by 0.11%.(3)The hardware acceleration of the lightweight semantic segmentation algorithm proposed in this paper is implemented on FPGA.A high-throughput hardware processing system based on the deep learning processing unit is designed,and the designed model is quickly deployed on the selected hardware platform suitable for micro-nano satellite applications through quantization and compilation.Under the condition of making full use of hardware resources,the inference speed of the semantic segmentation model deployed on the edge FPGA device reaches 13.54 FPS on an image with a resolution of 512 × 512,which is 1.2 times higher than that of the CPU side,and the segmentation accuracy is only Decrease 0.05%.