High-Speed Computing Of Lk Optical Flow Based On Heterogeneous Computing

With the booming of autonomous driving industry,its application scenarios have become more complex and diverse.Simultaneous localization and mapping(SLAM)based on GPS and laser lidar is increasingly difficult to meet the requirements of autonomous driving scenarios.Therefore,visual simultaneous localization and map construction(VSLAM)system based on optical flow algorithm has become one of the research hotspots in this field.VSLAM based on optical flow mainly calculates the matching relation between two consecutive images by optical flow,and constructs the visual odometer based on this.Optical flow-based VSLAM builds a visual odometer by calculating the matching relation between two consecutive frames of images through optical flow.In order to achieve real-time positioning and 3D point cloud reconstruction in vehicle-mounted scenarios,VSLAM is usually supplemented with corresponding back-end optimizations.Most of the existing optical flow algorithms are optimized and implemented based on servers,which cannot be directly deployed in vehicle scenarios.In order to solve the time-consuming problem of optical flow calculation in VSLAM system,this paper studied and proposed a high-speed LK optical flow calculation method based on two low-power heterogeneous computing platforms.Firstly,this paper designed a LK optical flow accelerator based on FPGA heterogeneous computing platform.Based on the parallelism analysis of the sparse and dense computing modes of LK optical flow,different parallel optimization strategies and caching mechanisms were designed.At the same time,the global quantization of optical flow calculation was realized by fixed-point quantization of bilinear interpolation.Experimental results show that dense optical flow calculation is more suitable for FPGA architecture than sparse optical flow calculation.The implementation of optical flow dense computing on FPGA is 1.8 times better than the multi-threaded version of Open CV on ARM Cortex-A57.In addition,this paper proposed a high-speed calculation method of LK optical flow based on embedded GPU.Aiming at the high concurrency of sparse LK optical flow computation,the scheduling method and memory allocation of the algorithm were designed and implemented on GPU.The pyramid model of dynamic window is designed to solve the problem of load imbalance in optical flow calculation.After that,the hardware performance of GPU is better developed by partial semi-precision quantization.Finally,a new bidirectional constraint method is proposed to reduce redundant computation effectively.Experimental results show that on TX2 and Xavier embedded GPU platforms,the performance of the proposed method is3.9 times and 3.3 times higher than Open CV GPU version,respectively,and the frame rate of720 P image processing can reach 30 frames per second.Finally,the VSLAM system based on high speed computation of LK optical flow is implemented and verified in real vehicle environment.Based on the optical flow calculation optimization scheme on GPU,a complete VSLAM system is designed and implemented.Then the parallel modules and data flow between VSLAM multi-nodes are designed to give full play to the computing performance of multi-nodes.Experimental results show that the optimization of optical flow algorithm improves the accuracy of the whole system effectively.In real vehiclemounted scenarios,the system can output accurate trajectory and sparse point cloud.This paper mainly solves the problem of insufficient optical flow computing performance on low-power devices through optical flow acceleration based on FPGA and embedded GPU,and verifies the feasibility of this scheme on the vehicle VSLAM system.