Research On Connected And Autonomous Vehicle Operation Control In Freeway Work Zone Area Based On Reinforcement Learning

With the development of connected and autonomous vehicle(CAV)technology,traffic control methods on urban roads and freeway have been developed intelligently.Aiming at the limitations of existing local single-point traffic control methods in freeway work zone areas,this paper proposes a cooperative multi-point traffic control method in freeway work zone areas based on multi-agent reinforcement learning,and builds an online simulation platform to visually analyze and verify the control method and control effect.This paper conducts related research on multi-agent reinforcement learning algorithm for connected and autonomous vehicle operation control in freeway work zone area.The main work involves the following aspects: First of all,learning from the advantages and disadvantages of the variable speed limit control method in the active traffic management,this paper proposes the vehicle merging control in the freeway work zone area.By adjusting the headway between multiple vehicles in the freeway work zone area in real time,the vehicle following state between multi-vehicles can be changed.Then,the problem of unbalanced distribution of upstream traffic in the freeway work zone area is analyzed,and the multi-vehicles advance lane-change control is proposed.By identifying the lane change gap and combining the proposed merging control mechanism,the purpose of the upstream traffic flow in the freeway work zone area is allocated in advance.Moreover,the distributed multi-agent online SARSA algorithm in reinforcement learning is used to cooperate the merging control and the advance lane change control,and realize the coordinated control of the vehicle in the multi-control areas before the freeway work zone area.In addition,the coordinated merging and lane-changing actions in the multi-agent are established,the lane occupancy rate and traffic volume for state monitoring are determined,the number of vehicle stops and the constructed density function as the joint reward.The coordination graph(CG)method is applied to seek the global optimum,where adjacent agents are incorporated into a multi-dimensional Q matrix to ensure that each Agent can find the global optimal result in consideration of the interaction of neighboring agents when updating the ontology Q value,which overcomes the problem of exponential growth of the Q-value matrix caused by the increase in the number of agents in centralized multi-agent reinforcement learning.Finally,an online interactive simulation platform was built by combining MATLAB and VISSIM COM,and the proposed control was simulated and evaluated;the iterative process of updating the reward value and Q value during the learning process was recorded,and the convergence and divergence of the algorithm results were verified;The four scenarios were compared and analyzed,and the global vehicle trajectory under the four scenarios were visualized;multiple indexes were selected to verify the optimization effect of the control.The comparison results show that the performance of the proposed control method is better than the existing local variable speed limit theory,and the trajectory of the vehicle is smoother than other scenarios.In addition,under the selected indicators,the optimization effect is significant.The above research results show that the research on freeway intelligent vehicle operation control based on multi-agent reinforcement learning proposed in this paper can significantly improve the operating efficiency in freeway work zone area,which can provide a scientific method for the operation control and technological development of connected and autonomous vehicle.