Design And Implementation Of FPGA Image High-Speed Stitching Heterogeneous Processing System

Digital orthophoto is one of the basic geospatial data to study and solve national sustainable development and major social problems such as population,resources,and environment.With the continuous launch of new high-resolution satellite remote sensing platforms,the rapid development of UAV technology and the maturity of image sensor technology,the amount of image data has increased geometrically.The traditional CPU serial processing method is difficult to meet the requirements of orthorectification and timely processing of massive remote sensing image data.On the other hand,surveillance cameras as an effective security technology,playing a huge role in many applications such as urban security,smart transportation,and intelligent buildings.However,since a single camera can only monitor areas with limited views,the collected video(image)data has the disadvantages of spatial information fragmentation,it’s not conducive to a comprehensive understanding of the target’s behavior in the monitoring area,which greatly limits the role of the monitoring system.In addition,the image size,brightness,and color of each surveillance camera are variable,which also puts forward higher requirements on the real-time nature and processing efficiency of the image stitching technology.In view of this,this paper studies high-speed remote image ortho-rectification and image stitching technology based on programmable logic device FPGA.The main research contents are as follows:(1)Building a high-speed orthorectification and stitch system of remote sensing images coordinated by CPU-FPGA,the two processors interact with each other through PCIE communication protocol.The CPU is responsible for the overall scheduling of the system and the loading and block sending of image data and DEM digital elevation model data.FPGA parallelizes bilinear interpolation of images and digital elevation,and realizes coordinate transformation of collinear equations to complete high-speed ortho-rectification of images.The CPU realizes the final image stitching by receiving each orthophoto block and mapping it to the corresponding storage space.(2)According to the characteristics of the fixed spatial position of each monitoring camera,the coordinate mapping relationship of multiple video / image to mosaic images is unchanged.Creating a look-up table file from the coordinate mapping relationship of multi-images to stitched images and the weight coefficient of stitching line pixel fusion.Online image matching is not performed during actual splicing,only image fusion processing needs to be completed by looking up the table,thereby improving real-time and image splicing efficiency.(3)Building an multi-images real-time stitching heterogeneous processing system based on ARM-FPGA,the two processors exchange data through AXI communication protocol.ARM is responsible for the system’s overall scheduling and the loading and block sending of images and lookup table data.FPGA realizes parallel acceleration of adjacent image fusion process.