Design And Implementation Of Stereo Visual Inertial Odometry Based On FPGA

Stereo visual inertial odometry is a real-time localization technology combining binocular stereo vision and inertial information,which is one of the important technologies in mobile robot localization and navigation.The stereo visual inertial odometry makes full use of the environment information to solve the scale uncertainty of monocular visual odometry and improve the robustness,but the computing power requirements for the computing platform are also improved.To solve this problem,this thesis designs Stereo visual inertial odometry system which uses FPGA for image acquisition and processing.It uses high-speed parallel computing ability and flexible hardware programming of FPGA to complete the feature point acquisition process,which reduces the CPU burden and greatly improves the real-time performance of the system.In addition,in order to improve the robustness of the system further,this thesis designs a set of PTZ stabilization system driven by exposure pulse based on FPGA,and realizes the function of motion blur suppression by controlling the three-axis PTZ based on optimization method.In this thesis,FPGA is used to directly drive the stereo camera and IMU sensor to complete the multi-sensor data acquisition and processing.In terms of hardware driver,the functions of unified automatic exposure of binocular camera,synchronous acquisition of binocular image,hardware synchronization between IMU and binocular camera are realized.The collected image is transmitted to the CPU side line by line through Ethernet or USB3.1 interface.At the same time of image transmission,the process of feature point extraction in FPGA is also executed in parallel.After the transmission of image frame,the feature point extraction can be finished immediately.The way of image transmission and image processing are executed simultaneously reduces the time from camera image acquisition to image processing.The well designed hardware improves the real-time performance of the system compared with the conventional scheme.The stereo image processing module of FPGA lightens the burden of CPU,and makes CPU have more computing power to run more consuming algorithm.The work of this thesis refers to the implementation idea of the fusion of stereo vision and inertial information in orb-slam3 system,and completes the stereo matching,inter frame pose optimization,local inertial bundle adjustment,etc.through the parallel cooperation of pose tracking thread and local map management thread,the system can complete high-precision real-time localization in a larger scene.When visual inertial odometry is used for robot localization and navigation,the changing of the robot’s motion state will affect the image.The author thinks that the localization system should have the ability to suppress the motion blur effect caused by the bump,vibration,pure rotation and high-speed movement of the robot body.Therefore,this thesis analyzes the causes of camera motion blur,establishes the model,and uses the optimization method to calculate the pan tilt motion control value based on the map points in the camera cone area and the body motion state.The combination of FPGA and STM32 microcontroller is used to realize the high-performance three-axis pan tilt based on BLDC motor.By adjusting the relative motion between camera and map points,the motion blur suppression guided by feature points is realized,and the robustness and positioning accuracy of the system are improved.To sum up,this thesis integrates circuit design,mechanical design and software design,and implements a complete stereo visual inertial odometry device.