Research On The Key Techniques Of Panoramic Vision System For Autonomous Navigation Agricultural Vehicles

Vision technology is a key technology for autonomous navigation agriculture vehicles,and it is the hot issue of research fields at home and abroad. Compared with the traditional vision technology, panoramic vision system can effectively obtain 360-degree non-blind area of environmental information, and it can better realize autonomous navigation of agriculture vehicles based on vision.The panoramic vision system for navigation was designed, and key techniques of panoramic vision system included stitching panoramic images, light self-adapting,detecting and tracking moving objects, and panoramic vision for simultaneous localization and mapping (PV-SLAM). The relative experiments were carried out on the experiment platform which was transformed by a Dongfanghong SG250 tractor. The major research contents and conclusions were summarized as below:1. The improved stitching method of multi-vision panoramic images was proposed by doing research on panoramic vision system. Firstly, the structure of panoramic vision system was researched and analyzed, and hardware platform of system was designed and realized. Secondly, the improved stitching algorithm of multi-vision panoramic images was researched and analyzed specifically. The improved stitching algorithm used multi-threaded to increase speed of obtaining multi-camera images, used camera calibration to reduce distortion of camera images, and used improved RANSAC-SIFT algorithm to improve the effect of matching feature points and stitching multi-vision panoramic images. The improved RANSAC-SIFT algorithm could avoid global operation to reduce time consuming, and optimize the feature points. Experiments showed that the improved stitching method of multi-vision panoramic images could effectively complete image acquisition, camera calibration, feature extraction and optimization, the feature point matching and panoramic images stitching. Using the improved method, the accuracy rate of feature points matching was 91.6% and operation time was 0.27s under the 512 x 256 pixels. Compared with the traditional algorithm, the accuracy rate of feature points matching was increased by 25.6% and the arithmetic speed was increased by 25.0%.2. The light self-adapting method of panoramic vision was proposed by doing research on the problem of image quality which was affected by ambient light in practical application for autonomous navigation agricultural vehicles. Firstly, hardware modules including light intensity acquisition, control and processing and data wireless transmission were designed and realized. Secondly, vision imaging system was analyzed, and the concrete research content was digital imaging principle, imaging exposure control and image quality evaluation. Thirdly, the light self-adapting algorithm implementation process of panoramic vision was researched and analyzed. Experiments showed that the improved light self-adapting algorithm could obviously increase the panoramic image quality in normal light, high light, and low light. Under the condition of high light, compared with the original algorithm, the two-dimensional image information entropy was increased by 47.1%, and image gradient value was increased by 60.9% using the improved light self-adapting algorithm. Under the condition of low light, compared with the original algorithm, the two-dimensional image information entropy was increased by 30.3%, and image gradient value was increased by 76.40% using the improved light self-adapting algorithm. Under the different light conditions, the improved light self-adapting algorithm could averagely take 0.36s. Compared with the traditional multiple-exposure algorithm, the arithmetic speed of the improved algorithm was increased by 75.5% under the almost same image quality.3. The method of moving objects detecting and tracking was proposed by doing research on the driving safety problem of autonomous navigation agriculture vehicles based on panoramic vision. Panoramic vision possessed the advantages of non-blind area detection and the improved algorithm solved the problem of the overlap in multiple moving objects tracking. Firstly, moving object was detected by the improved CLG (Combined Local-GJobal) optical flow algorithm. Secondly, the improved kernel function algorithm based on segmented image was used to detect and track the moving object automatically and rapidly. Thirdly, the particle filter algorithm based on path prediction was used to track multiple moving objects and solved the overlap problem. Experiments showed that the improved CLG optical flow algorithm could averagely take 1.55s to detect moving obstacles, and the accuracy was 95.0%. Compared with the HS optical flow algorithm, the accuracy rate of detecting was increased by 22.3%, and the arithmetic speed was increased by 47.8% using the improved CLG optical flow algorithm. Compared with the traditional kernel function algorithm, using the improved kernel function algorithm, the memory consumption was reduced by 66.8% and the success rate of tracking was increased by 24.4% at the same time arithmetic speed was increased by 35.63%. Multiple moving objects detection using the particle filter algorithm based on path prediction could averagely take 0.78s, compared with the traditional algorithm, the arithmetic speed was increased by 37.3%, and the success rate of tracking was increased by 33.1%. Under the condition of overlap in multiple moving objects, the arithmetic speed was increased by 46.8%, and the success rate of tracking was increased by 39.5%.4. A method named PV-SLAM was proposed by doing research on simultaneous localization and mapping for autonomous navigation agriculture vehicles. Firstly, the principle of Inertial Navigation System (INS) was researched and analyzed, and hardware circuit module of Inertial Measurement Unit (IMU) was designed and realized. Secondly,the motion model of agricultural vehicle and the observation model of panoramic vision system were researched. Thirdly, the implementation process and steps of PV-SLAM algorithm was analyzed, panoramic vision (PV) was combined with Inertial Measurement Unit (IMU) and Extended Kalman Filtering (EKF) to realize the PV-SLAM process for autonomous navigation agriculture vehicles. Compared with the traditional visual SLAM(vSLAM), experiments showed that the number of landmarks was averagely increased by 80.2%, and the success rate was increased by 15.8% using PV-SLAM method, under the condition of less or no fixed landmarks. Using PV-SLAM method, the average error on x and y direction was 0.065m and 0.062m respectively, and average error of location was 0.108m. C.ompared with the traditional visual SLAM (vSLAM), the average accuracy on x and y direction was 35.3% and 37.8% respectively, and average accuracy of location was 36.2% using PV-SLAM method. Under the condition of open loop path, the average accuracy on x and y direction was 27.4% and 29.5% respectively, and average accuracy of location was 28.3%. Under the condition of closed loop path, the average accuracy on x and y direction was 43.1% and 46.1% respectively, and average accuracy of location was 44.1%. PV-SLAM method could accurately and completely obtain the landmarks. This method could less depend on fixed landmarks, and could work well under the condition of closed loop path for agricultural work.