| The development of unmanned agricultural machinery can effectively promote the progress of agricultural science and technology,so that China’s agriculture continues to present a new appearance.Unmanned agricultural machinery can replace farmers work,improve the efficiency and quality of agricultural machinery operation.However,there are inevitably some dynamic and static obstacles on the path of agricultural machinery,such as trees,haystacks,houses,telegraph poles,people,cattle,sheep and other tractors.If these obstacles are not detected and identified,the collision between unmanned agricultural aircraft and obstacles will cause serious economic losses and even casualties.Therefore,the unmanned agricultural machinery should have strong ability of environmental perception.The purpose of this paper is to build a robust agricultural obstacle detection method,which can effectively improve the use of single vision sensor to detect agricultural obstacles encountered blocking problems,unknown obstacles,sensor failure can not detect the situation;In addition,the accuracy and comprehensiveness of obstacle information obtained by agricultural machinery can be improved,laying a foundation for accurate obstacle avoidance in the future.Therefore,an obstacle detection method based on multi-sensor information fusion is proposed in this paper.This method combines the millimeter-wave radar in distance measurement speed and camera in type identification and transverse location advantage,will be successful data correlation decision level fusion method of millimeterwave radar target sequence effectively and the camera effectively target sequence weighted output,output information including more precise target bearing,the longitudinal velocity,category.For the unassociated sequences,the extended Kalman filter algorithm is used to track the new targets,and target management and output are carried out based on the effective life cycle.The main contents and work of this paper are as follows:(1)Research on obstacle detection based on millimeter-wave radar.The structure and principle of millimeter-wave radar were introduced.However,the radar selection and radar data analysis;In this paper,a method was proposed to filter the data generated by millimeterwave radar to reduce the calculation cost.False targets ware filtered by effective life cycle theory;Non-threat targets ware filtered according to the set transverse distance threshold and longitudinal distance threshold that ware safe for tractor operation.The millimeter-wave radar data visualization interface was designed.Experiments showed that the proposed filtering algorithm could effectively filter out 82% of invalid targets and reduced a lot of unnecessary calculations for subsequent fusion.At the same time,it can be known from the test that the millimeter-wave radar had a poor ability to identify the type of obstacles,so it was necessary to introduce the camera to identify the obstacles.(2)Research on obstacle detection based on camera.Firstly,the deep learning detection algorithms of Faster R-CNN and YOLOv5 s ware introduced,and the detection algorithm was selected and improved according to the actual detection results.An improved YOLOv5 s detection method was proposed to solve the problems existing in the original YOLOv5 s loss calculation using GIo U.In this method,CIo U was used to calculate the loss and the original anchor frame size was replaced by the anchor frame size obtained by clustering based on Kmeans algorithm.The improved YOLOv5 s,the original YOLOv5 s and the Faster R-CNN were used to detect agricultural obstacles,and the detection effects of the three were compared.The test results showed that the improved YOLOv5 s has a good detection effect,and the inference time of a single image was 0.074 s,which was only one fourth of that of Faster R-CNN.The m AP value of Faster R-CNN was 65.12%,only 1.64% lower than that of Faster R-CNN.The m AP value of the improved YOLOv5 s was 5.80% higher than that of the original YOLOv5 s,and the detection time was not much different.The detection effect of small targets was improved a lot.A two-stage detection method based on improved YOLOv5 s and SSRN-De Blur Net was proposed to solve the problem of missed detection and misdetection when the deep learning detection algorithm encountered fuzzy image input in the inference stage.The test results showed that the improved YOLOv5 s and SSRN-De Blur Net two-stage detection method could effectively reduce the missed detection and false detection caused by fuzzy image input.Compared with the improved YOLOv5 s,the m AP value of this method was increased by9.39%,and the inference time of a single image was increased to 0.172 s,which meet the requirements of tractors.(3)Information fusion strategy of millimeter-wave radar and camera.The calibration of millimeter-wave radar and camera and the alignment of time and space datum ware carried out.In this paper,a vertical distance estimation method based on pixel value fitting was proposed to transform pixel coordinate system to world coordinate system.A data association method based on the GNN method was proposed to match the observed values of millimeterwave radar and camera.A weighted output method was used to output the fused target sequence.The extended Kalman filter(EKF)was used for target tracking to maintain an effective target library.(4)Information fusion test of millimeter-wave radar and camera.In order to verify the effectiveness of the farmland obstacle detection method based on deep learning and multisensor fusion,obstacle detection was carried out based on the real tractor after completing the construction of hardware and software.Experimental results showed that the reliability and accuracy of the proposed method ware higher than that of the single sensor.The multisensor fusion could correctly detect 66.18% of obstacles,which was higher than that of a single camera(52.47%)(camera had a false detection rate of about 10%).Due to the use of millimeter-wave radar,the rate of missed detection was 13.71% lower than that detected only by camera. |
PDF Full Text Request