Cloud Computing Based Decision Of Intelligent Model Vehicles Through Deep Reinforcement Learning

In recent years,the rise of artificial intelligence,especially neural networks and reinforcement learning,has brought promise to solve the long-term classic problems of the transportation system.A common approach to improve traffic efficiency is to control traffic lights to ensure the smooth flow of traffic.However,due to the unruly behaviour of vehicles,the actual effect is limited.With the development of intelligent vehicle technology,the cloud-based control of the transportation system is not only able to control the traffic lights but also directly control the vehicles.In this traffic management method with cloud-based control,cloud decision-making ability is the crucial factor to determine the efficiency of the transportation system.The cloud decision algorithm will be the key technology of the future intelligent transportation system.Because cloud-based decisionmaking involves multi-vehicle cooperation,the difficulty,and risk of using real vehicles for research are high.Therefore,this paper will focus on the cloud-based decision-making algorithm based on deep reinforcement learning with the intelligent model vehicle as to the carrier.The research work of this paper can be roughly divided into three parts:Firstly,this research proposes an indoor positioning algorithm based on the fusion between camera and UWB,which solves the accurate acquisition of simulated vehicle position information.At present,the existing indoor positioning method is based on camera detection,which relies too heavily on the brightness and clearness of the object detected as it may make detection becomes unreliable.Furthermore,the localization of the edges of the camera is also inaccurate positioning.Therefore,this paper studies the wireless positioning method UWB which does not rely on the camera,and constructs the UWB positioning system adaptively selected by the base station to solve the multi-base station time synchronization problem and the non-line of sight(NLOS)positioning problem.On this basis,this research further studies the fusion positioning method of realtime positioning information and map prior information and realizes multi-target position detection and tracking.In this paper,the UWB positioning system achieves a good positioning result,but the model vehicle heading angle information cannot be obtained.Therefore,the angular estimation method combining camera detection and UWB positioning is further studied,and the accurate acquisition of complete location information including the heading angle is realized.The second part of the study is deep reinforcement learning where deep Q network(DQN)and asynchronous advantage actor critic(A3C)were explored.These two algorithms are all trained by using RMS prop optimization.DQN algorithm is trained in a batch of thirty-two to remove correlation between episodes.A3 C on the other hand implement asynchronous capability by implementing workers agent to train the global network.Both of these two algorithms were trained on more than a thousand episodes with three thousand steps in each episode.To increase learning,the spawn position of every vehicle is randomized on every episode.Finally,in order to verify the proposed algorithm,a virtual simulation software platform and a sand table simulation platform to tests the cloud decision algorithm are constructed.Although the object of this study is the model car,its control interface is the expected speed and steering wheel angle which is similar to real vehicles.The interaction between cloud server and model cars is through wireless transmission.And to ensure that the deep reinforcement learning algorithm studied in this paper can deal with real traffic surroundings,the simulation test show that the deep reinforcement learning proposed in this paper can ensure that multiple controlled vehicles pass through the intersection safely and efficiently.Based on the test of the miniature traffic platform,it is shown that the multi-sensor fusion indoor attitude positioning method proposed in this paper has high precision and stability.At the same time,in the model of cloud control model car,it is also preliminarily verified that deep reinforcement learning can deal with the decision problem at the intersection.