Design And Implementation Of Embedded Real-time Computing System For On-board Target Perception

With the more and more extensive and in-depth application of UAVs in civilian and military,airborne intelligence technology gradually arouses general interest in the competition of various technological and military powers.The technology of intelligent drones is usually based on various perception computing.The target perception combining remote sensing and computer vision technology is the most effective intelligent means in a variety of scenarios such as current UAV autonomous flight,automatic search and rescue,automatic monitoring,military reconnaissance,etc.To perceive the environment on a larger scale,UAVs often use multiple visual perceptions such as hyperspectral,infrared,visible light,and radar,which significantly increase the scale of airborne data while obtaining more original information.Constrained by ground communication bandwidth,transmission delay,and real-time performance,as well as the requirements for independent and controllable intellectual property rights on special occasions,the airborne perception computing of domestic UAVs often becomes an essential bottleneck in applicationsTo address the problem of real-time computation of UAV airborne target perception,considering real-time requirements,environmental resource constraints,and independent and controllable intellectual property rights,this thesis investigates the visual perception computing problem from two aspects: distributed co-processing and hardware acceleration,and designs and implements a heterogeneous multi-processor airborne embedded system for visual real-time perception computing.The main work is as follows:(1)A distributed co-processing scheme for airborne perception computing is designed.The parallel acceleration of the target detection workflow needs to solve each work’s mapping problem to the processor.This thesis studies this issue in three steps,firstly the computing characteristics of the workflow are clarified,such as target detection preprocessing,inference,and post-processing,and the computing characteristics of various mainstream processors are discussed;Then,according to the analysis the mapping relationship from each workflow part to CPU,ASIC,and FPGA cores is determined;Finally,inspired by the cross-device end-toend parallel inference method,a collaborative target detection scheme is determined.To expand computing resources and improve the system’s reliability,this thesis independently maps each workflow part to multiple different pieces of equipment.(2)An airborne perception computing embedded system composed of heterogeneous multiprocessors with independent and controllable intellectual property rights is designed and implemented.The system includes a real-time target detection system and a GNSS real-time positioning system.The former is the specific implementation of this paper’s distributed collaborative processing scheme,including the hardware circuit design and implementation of three boards and one backplane corresponding to the three workflow parts.Among them,the inference board integrates 4 ASIC acceleration cores;As a part of the flight control,the latter provides prerequisites for UAV navigation and remote sensing image correction,including GNSS and IMU modules for solving satellite positioning data and pose positioning,as well as communication modules of the DGNSS data link.(3)A target detection application with location information completed by the NNInference method on the designed and implemented embedded system is realized.Aiming at the real-time bottleneck of the inference part in the target detection workflow,this thesis extends the data-parallel way from the end-to-end level of the deep neural network to the workflow level.First,the data division strategy for dividing the input video stream by frame is determined.The OLB static scheduling method is used to realize the mapping process of the split data to the ASIC inference processor.For the GNSS positioning issue,firstly the data link of the DGNSS system is accomplished;secondly,raw GNSS data and RTK differential data are combined to calculate the accurate position.Finally,this work also verifies the effectiveness of the design scheme and system.