| To improve the level of safety,low-carbon level and comfort of commercial vehicles,intelligent driving technology has been attached great importance in recent years.As the key to achieve high quality intelligent driving,driving decision-making determines the rationality and effectiveness of automated commercial vehicles.Among them,how to make safe and effective driving decision-makings,especially to achieve comprehensive driving decision-makings including safety,low carbon,economy and comfort,is a key issue in the research of driving decision-makings for automated commercial vehicles.Therefore,this paper combined with the new development of deep reinforcement learning and imitation learning.It conducted an indepth study on the key issues involved in the driving decision-making for automated commercial vehicles.Firstly,a driving decision-making methodology considering both forward and backward anti-collision was proposed.Then,a safe driving decision-making methodology was proposed.It considered forward,backward and lateral collision avoidance and rollover avoidance.On this basis,a comprehensive decision-making methodology considering safety,low carbon and comfort is studied.Finally,a driving decision-making method for platooning is proposed.The main contribution and innovation of this dissertation can be summarized as follows.1)To cope with the problem that existing driving decision methods pay more attention to forward anti-collision and ignore backward anti-collision,an anti-collision driving decisionmaking method based on actor-critic structure was proposed.Firstly,the anti-collision driving decision problem was modeled as a finite Markov decision process.Secondly,based on the multi-objective optimization idea,the forward and backward anti-collision reward functions were designed.When designing the reward function,both adaptive driving safety clearance and obstacle types were considered.Under the guidance of the reward functions,an improved deep deterministic policy gradient algorithm was used to learn anti-collision driving strategies under different driving conditions.Finally,under the conventional driving conditions and edge driving conditions,experiments were conducted to evaluate the anti-collision performance of our proposed method.Experimental results show that our proposed method has better anticollision performance by comparing the minimum time headway and minimum reverse time to collision.In the corner case in which the backward vehicle suddenly accelerates,our proposed method can still avoid collisions.Compared with other comparison methods,the forward,rear anti-collision and anti-rollover performance of our proposal are improved by more than 7.6%,11.3% and 26.7%,respectively.It has effectively solved the problem of forward and backward collision,which occurs most frequently in the highway environment.2)To cope with the problem that existing driving decision methods are difficult to comprehensively consider forward,backward and lateral anti-collision and anti-rollover,a safe driving decision-making method based on cascaded imitation learning network was proposed.Firstly,to minimize the difference between the driving strategy and safe driving maneuver,a decision sub-network based on supervised learning was constructed.Behavior cloning algorithm was introduced to guide the network to learn the anti-collision driving strategy in real driving data.Secondly,to cope with the problem that the difficulty of designing explicit reward function artificially in multi-objective optimization process,a decision sub-network based on imitation learning was constructed.Generative adversarial imitation learning algorithm was used to further learn safe driving strategies considering both anti-collision and anti-rollover.Experimental validation was performed.Compared with the evaluation indicators such as collision time,lateral clearance and lateral acceleration,our proposed method was able to reduce jerky and dangerous driving actions.Under the high-density traffic conditions prone to collision and rollover,our proposed method can still ensure the driving safety of commercial vehicles.Compared with other mainstream comparison models,the anti-collision and antirollover performance of our proposal are improved by over 27.2% and 30%,respectively.It has achieved safe driving decision-making for commercial vehicles considering forward,backward and lateral collisions as well as rollover.3)Aiming at the problem that existing decision-making methods are difficult to take into account safety,low carbon energy saving,comfort and other aspects,a comprehensive driving decision-making method based on cascade strategy learning for automated commercial vehicles was proposed.Firstly,to cope with the problem that the difficulty of designing explicit reward function artificially,a driving decision sub-network based on generative adversarial imitation learning was constructed.To reduce excessive invalid explorations,the behavior cloning algorithm is used to pre-train the generator of the sub-network.Then,a better initial strategy for the subsequent comprehensive driving strategy learning can be provided.Secondly,it is difficult to accurately extract expert data representing low carbon and comfort level.Meanwhile,to break through the limitation that imitation learning only studies conventional driving conditions,a driving strategy learning module based on competitive two-layer deep Q-network is constructed.Meanwhile,a reward function was designed to guide the learning of driving decisions.It considered factors such as vehicle collision,rollover,intensity of carbon dioxide emission,jerk,etc.Experimental validation was conducted.In the traffic environment with high traffic flow density,compared with other mainstream decision-making methods,the proposed method improves the performance of anti-collision and anti-roll by more than 9.2% and 34.2%,respectively.Meanwhile,it improved the performance of low carbon level and driving comfort by over 2.4% and 11.4%,respectively.The driving strategy output by our proposed method has a higher safety,lower carbon emission intensity and better driving comfort.It has realized comprehensive driving decision of commercial vehicles in highway environment.4)Existing driving decision-making methods for platooning pay more attention to the longitudinal driving strategy.The vehicle roll stability has been ignored in the formation lane change process,which is difficult to ensure the safety and comfort effectively.Therefore,a formation driving decision-making method based on semi-supervised learning was proposed for following vehicles during formation.Firstly,a reward function considering anti-collision,anti-rollover,travel consistency,lateral distance offset and comfort was designed.Secondly,to solve the problem that the convergence difficulty because of continuous control quantity in the action space,a sub-network of driving decision based on data aggregation was constructed.By means of supervised learning,it was guided to learn expert experience.Finally,to explore driving strategies including conventional,corner and dangerous driving conditions,a subnetwork of driving decision based on improved soft actor-critic was constructed.By means of unsupervised learning,it was used to further learn formation driving strategies.Under the edge condition where the leader vehicle changes lane frequently,the anti-collision performance and formation driving performance of our proposal have been improved by over 10.7% and 8.1%than other mainstream decision-making models.Our proposal can effectively ensure the safety and comfort of commercial vehicles when platooning,especially in the process of formation lane change.It has also achieved good formation decision-making performance. |
PDF Full Text Request