Method For Kiwi Trunk Detection And Navigation Line Fitting Based On Deep Learning

In order to realize the visual navigation of the mobile platform in the environment of kiwi fruit orchard,the scheme of the visual navigation system was determined through the comparison of feature selection and visual detection methods.The effect of convolutional neural network depth on the feature extraction of kiwi trunk was explored by using convolution layer feature visualization technology.By optimizing the Yolo v3 Tiny model,it can achieve the balance between detection accuracy and detection speed on embedded devices with limited computing resources,and verify the adaptability of the object detection model under different working conditions,and then use the detected kiwi trunk object to fit navigation lines.Vision navigation system was built through hardware integration and software design,and navigation performance was verified through tests.(1)Visual inspection method and visual navigation system design.Field research was conducted on the scaffold-type kiwi fruit orchard.The GPS signal under the scaffold was weakened due to the dense canopy on the top,so a visual detection method was selected to implement navigation.Use small drones to fly under the scaffold to collect image data.Annotate the dataset using the features of the edge of the ridge and the characteristics of the kiwi trunk,and the semantic segmentation Segnet and object detection Faster R-CNN models were trained.The results show that the precision of Segnet semantic segmentation was 80.14%,and the inference time was 2.5s.Faster R-CNN object detection accuracy was 79.56%,and the inference time was 1s.Finally,the object detection method was selected due to real-time and environmental adaptability factors,and the overall plan of the visual navigation system was determined.(2)Feature extraction of kiwi trunk based on convolution layer feature visualization.In order to explore the effect of the depth of the convolution layer on the feature extraction of kiwi trunk images,a visualization method was proposed to analyze the extracted features.First,the collected data set was classified into positive and negative samples.Take the area where the trunk and the water pipe intersect in the dataset as positive samples and the remaining areas as negative samples.Input the samples into Le Net,Alexnet,Vgg-16 and the defined 3 types of shallow structures for training.Then,by extracting the activation map,normalization,and bicubic interpolation visualization methods,the visualization results of the last convolution layer of each classification model are obtained.The comparison can be obtained: Alexnet and Vgg-16 extract trunk features in the test image,while Le Net and 3 types of shallow models extract the trunk,ridge and other features together while extracting the trunk.Finally,the image classification and object detection models of the above 6 types of network structures as feature extraction layers are used to verify the flowering period and fruiting period data sets,and the accuracy drop caused by changes in the characteristics of the data sets in different seasons was used as the evaluation criterion: the accuracy of image classification shallow model decreases by more than 15.90 percentage,Alexnet and Vgg-16 decrease by 6.94 and 2.08 percentage respectively,the accuracy of object detection shallow model decreases by more than 49.77 percentage,Alexnet and Vgg-16 decrease by 22.53 and 20.54 percentage respectively.The accuracy of all shallow models was greatly reduced due to changes in the extracted features.(3)Kiwi trunk detection and navigation line fitting method based on Yolo v3 Tiny-3p.The Yolo v3 Tiny model was optimized to achieve a balance between detection accuracy and speed in the Jetson TX2 with limited computing resources.A lightweight convolutional neural network was selected as the feature extraction layer,and prediction was performed on 3 scales.The test results show that the optimized Yolo v3 Tiny-3p model has a detection accuracy of 87.63% and a detection speed of 12 FPS.The detection accuracy was 90.40% under black ridge conditions,87.23% under interrow grass conditions,and 85.23% under mulching conditions.The large number of small objects in mulching conditions results in the lowest detection accuracy.A navigation line fitting method based on quadratic polynomial extreme points was proposed.The detected objects are divided into left and right categories by finding extreme points,and the midpoint of the lower part of the bounding box was used to fit the straight lines on both sides.The centerline of the side straight line gets the final navigation line.The test results show that the fitting accuracy of the navigation line was 92.15%.The fitting accuracy of the navigation line in the black ridge condition was 92.03%,the fitting accuracy of the navigation line was 93.13% in the interrow grass condition,and the fitting accuracy of the navigation line was 91.28% in the mulching condition.The deviation of the object detection results under different working conditions has a lower degree of influence on the results of the navigation line fitting,and the environmental adaptability of the navigation line fitting method was better.(4)Integration and verification test of visual navigation system.Build a visual navigation system through hardware integration and software design.The navigation speed test was set with different travel speeds,and the lateral deviation was used as the evaluation standard.The final travel speed was 0.2m/s.The initial position deviation test places the initial position of the chassis at an angle.The navigation line can be traced to a distance of 5m.The heading deviation fluctuates around 0° and the lateral deviation range was within 5cm.The navigation deviation test under different working conditions was tested under three conditions: black ridge,interrow grass and mulch.The average lateral deviation was 7.15 cm under the black ridge condition,and the average lateral deviation was 6.29 cm under the interrow grass condition,and the average lateral deviation was 7.36 cm under mulching conditions.The visual navigation system can adapt to 3 different working conditions,but the lateral deviation fluctuates widely.