Research On Deep Reinforcement Learning Based Path Tracking System For Intelligent Agricultural Vehicle

Intelligent agricultural machinery can effectively improve the efficiency and quality of operation,and plays an important role in promoting the development of precision agriculture.It is an inevitable trend for China to realize agricultural modernization.Path tracking control technology is one of the key technologies of intelligent agricultural machinery,which has been widely used in the whole process of agricultural production.Aiming at the problems of complex design,difficult parameter tuning and poor adaptability to complex agricultural roads of existing path tracking control methods,this paper proposes a path tracking control method based on deep reinforcement learning algorithm,and develops a path tracking control system based on the control method.The specific research contents are as follows:(1)The path tracking control algorithm based on deep reinforcement learning is optimized.The five-layer BP neural network is used to construct the network part of the DQN-based(Deep Q-network)path tracking control algorithm to realize the lightweight of the network and the high portability of the control algorithm.The input state of the tracking control algorithm network introduces the average path curvature within the set distance in front of the vehicle along with the current steering angle of the vehicle and the lateral distance between the vehicle and the control point on the target path.The reward function is constructed by the distance error between the vehicle and the target path in the path tracking process.(2)The path tracking control system based on deep reinforcement learning is developed,and the experimental platform of autonomous navigation control system is constructed.The software and hardware of navigation electronic control unit(ECU)and main vehicle control unit(VCU)are developed,and the vehicle navigation control system composed of navigation ECU,main VCU and slave VCU is constructed.The path tracking control algorithm based on deep reinforcement learning designed by C++ is installed on the navigation ECU.The acquisition and analysis functions of vehicle pose information are developed on the main VCU,and the interaction between system components is realized based on serial communication and CAN communication.In addition,the human-computer interaction interface is developed to realize the convenient operation of the tracking control system.(3)The sinusoidal curve is used to train two path tracking control models with path curvature input and without path curvature input based on deep reinforcement learning to verify the feasibility of the algorithm.The results show that both control models can converge to the sinusoidal target path,and the average distance error is 0.008 m and 0.017 m,respectively.The two convergent path tracking control models are respectively installed into the path tracking control system,and the tracking experiment is carried out on the U-shaped path.The average distance errors of the two tracking control methods with and without path curvature input on the whole path are 0.038 m and 0.068 m,respectively,and the average tracking errors on the curve path segment are 0.051 m and 0.133 m,respectively.The performance of the tracking control method with path curvature input is better than that of the tracking control method without path curvature input.Then,the tracking control system with path curvature input is used to carry out field experiments on the S-shaped path under two road conditions.The experimental results show that: By comparing with pure pursuit control method with different forward looking distance,the tracking performance of the path tracking control method based on deep reinforcement learning proposed in this paper is better than that of the pure pursuit control method.The developed path tracking control system has good adaptability and stability,and can track the path under different road conditions and different curvature paths.Under soft and smooth road conditions,the average tracking error on two S-shaped paths with a straight distance of 6 m and 5 m is 0.024 m and0.029 m,respectively.Under solid and uneven road conditions,the average tracking error on two S-shaped paths with a straight distance of 6 m and 5 m is 0.032 m and 0.035 m,respectively.