Research On Real-time Optical Flow Estimation Method Based On FPGA Embedded System

The theory of estimating optical flow in image sequences has been developing for many years in machine vision field,but the high computational complexity of the algorithm limits its application in the industrial field.In order to meet the real-time requirements of applications,it is very popular to calculate the optical flow on FPGA using the L&K method.However,most of the research is devoted to the design and implementation of calculating optical flow using L&K method on FPGA,and there is no further research on optimization of the design.Especially,due to the fixed-point calculation using FPGA,the accuracy of computation is reduced.And the convolution complexity of L&K method is high.In order to improve the accuracy of computation and solve the problem of excessive utilization of FPGA resources caused by high computational complexity,we improves the L&K method and optimizes the design method.The main tasks involved include:(1)In order to meet the real-time requirement of calculating optical flow in application,we use the FPGA platform to design and implement the L&K method.The input image is denoised by median filter.Design the controller to read data from external memory and write data to external memory.Optical flow is computed in real time using the L&K algorithm by pipeline design.And a display controller is designed for displaying the image.(2)The accuracy of the design is optimized.Aiming at the problem that the calculation precision decreases for fixed-point calculation in FPGA,we design a two-layer architecture to improve the precision of optical flow.High-level images are used to roughly estimate the optical flow field,and low-level images are used to further refine the estimated optical flow vector.The calculation method of two-layer architecture increases the number of read and write data while improving the accuracy of the algorithm.Therefore,we propose that when the image is read,which will be sampled and cached to two spaces for subsequent computation.Considering the correlation of iterative computation in calculating the optical flow of two levels images,we have designed the readout order of data stored in external memory.Byincreasing the utilization ratio of image data,the requirement of high data rate of external memory is reduced.(3)Resource optimization.In view of the inherent complexity of convolution computation in L&K algorithm,we analyze the convolution calculation process.It is found that there is a data overlap area in the two convolution operations of two adjacent pixels.In order to make the overlapping data have the same calculation,we designed a new weight function for convolution computation.It is only 4 addition to do convolution operation using the new weighting function,and the convolution computation is independent of the size of convolution window.This method can reduce the convolution computation by 73%,thus saving a lot of logical resources.In addition,the gradient calculation and least square operation are optimized by reducing the width of data bits.The experimental results show that the proposed method is more accurate than the traditional L&K method running on PC platform,and the resource consumption about convolution computation is significantly reduced.The designed system can process 1280×1024 pixels images at 60 fps,and meet the real-time requirements of optical flow calculation.