A Real-time Visible Infrared Fusion And Object Detection System Based On FPGA

In recent years,UAVs have developed rapidly and gradually become an indispensable and important equipment.They have been widely used in various fields such as national security,economic construction and so on.With the improvement of technology and expansion of social needs,more and more civil and military UAVs begin to appear in our visual field.They provide an efficient and convenient service for mankind and greatly promote social economic development and scientific technological progress.The wide use of UAV has also brought a wealth of aerial images.Capturing regions or targets of interest in these images has become a research hotspot.Therefore,it is urgent to realize an automation and intelligence of object detection under the vision of UAV.In this thesis,The current situation of object detection under the vision of UAV is investigated,and the existing challenges of it are summarized as follows: It need the ability to work all day.However,the detection effect is greatly affected by light intensity especially at night;Multi-scale objcets exist depending on the various flight altitude;The imaging background is complex with diverse interference factors;The airborne equipment has limited computing capacity and power consumption.Aiming at the problem of poor detection effect in complex scenes such as insufficient visibility or illumination intensity and high temperature,this thesis proposes a backbone network with feature level image fusion which improves the adaptability and accuracy in our task.Aiming at multi-scale objects and complex background,this thesis proposes a light weight object detection algorithm with several modifications such as attention mechanism,data enhancement and loss function.Based on the above unified model of image fusion and object detection,our model suppresses the interference of background information and improves both the accurarcy and speed of detection.In view of the limited performance of airborne processing equipment,this thesis proposes a dual FPGA architecture,which divides the interface,control,communication processor and edge computing processor into two processors to give full play to the advantages of edge computing processor.This thesisdesigns sensor image acquisition based on FPGA,SRIO high-speed data bus and VDMA cache mechanism.In this thesis,we use hardware acceleration to process a real-tiem image denoising and registration algorithmin.We optimizes,quantifies and compiles the network model and deploys it to the edge platform which uses DPU for computing.This paper realized the unmanned airborne real-time image fusion object detection system,which effectively improves the detection accuracy under difficult conditions and achieved the detection speed of 30 fps.Compared with other UAV target detection systems,this thesis proves the advantages of our design in detection performance.