Research On Fast Obstacle Avoidance Path Planning Method Based On Q-learning Algorithm

The application of robotics technology in all aspects of production and life is becoming more and more extensive.The main task of a robot control system is to complete path planning.When facing high-dimensional systems and systems with complex constraints,traditional path planning algorithms suffer from shortcomings such as high time complexity and tendency of falling into local optimum.The Q-learning algorithm is a typical reinforcement learning method,it does not require complete environmental knowledge;the DQN algorithm combines reinforcement learning and deep learning and gives full play to the decision-making advantage of reinforcement learning and the perception advantage of deep learning.These algorithms enable robot to autonomously learn appropriate behaviors from the environment,and have achieved good results in robot path planning.With the advent of artificial intelligence era,more and more complex environments have put forward higher requirements for the speed and flexibility of path planning algorithms.Research on how to effectively improve the speed of path planning has important application value and practical significance.Based on the Q-learning and DQN path planning algorithms,this paper proposes and improves a series of methods aimed at increasing the speed of the algorithms,including threedimensional Q table,initializing Q function with a one-step reward value,variable ε-greedy strategy,path backtracking method,one-hot encoding state mapping,reducing bit-width of floating-point numbers,invoking SIMD instruction set,etc.The research method is to use Python language and Num Py and Pillow library programming to implement the above algorithms and improved methods,and then use several different grid maps to conduct path planning experiments,analyze and evaluate the experimental results in terms of whether a complete Q function(strategy)is obtained,?q,win rate,the episode value when the complete Q function is obtained for the first time and the execution time of the program and other indicators to verify the feasibility of the algorithms and the effectiveness of the improved methods in improving the running speed.The data obtained in the experiment shows that after introducing various high-speed improvement methods,the running speed of each algorithm has been significantly improved.The Q-learning algorithm can complete the path planning of a 9×9 grid map in only 0.27 s,and the convergence speed of Q function has also been improved several times.The episode value and time spent when the complete Q function is obtained for the first time for the fully connected neural network DQN algorithm is only a fraction of the original.The convolutional neural network DQN algorithm can complete the same work in 79.76% of the original time.Both the Q-learning algorithm based on reinforcement learning and the DQN algorithm based on deep reinforcement learning can successfully solve the path planning problem.The methods proposed and improved in this paper aimed at improving the running speed of the path planning algorithms are effective.