Research On Road Environment Segmentation Methods In Vision-based Autonomous Vehicle

As it can bring safer driving experience,reduce traffic accident rate and effectively solve urban traffic congestion,autonomous vehicle are receiving more and more attention.The architecture of the driving automation system generally consists of three main parts:the perception system,the decision-making system and controlling system.This topic of this dissertation focuses on the road environment segmentation technology in the perception system.The road environment segmentation technology is mainly aimed at recognizing and understanding the vehicles,roads,pedestrians and etc.in the traffic environment to provide data input for the follow-up system.With the upgrading of automatic driving technology from driving assistance to driverless driving,the demand for the accuracy,scalability and versatility of road environment segmentation technology is also gradually increasing.In terms of accuracy,the main problem is that it is difficult to fuse global and local features in the process of road environment segmentation.In terms of scalability,the road environment segmentation model has the problem of catastrophic forgetting for old data when it is based on new data training.In addition,in terms of universality,road environment segmentation technology still has problems such as that models trained under ideal conditions do not predict well under unfavorable conditions and negative optimization of domain adaptation algorithm.For this reason,this dissertation proposes representation learning based on attention network,incremental learning based on knowledge distillation and domain adaptation based on data enhancement to solve the problem of feature fusion,catastrophic forgetting and negative optimization in automatic driving environment segmentation.The main research contents of this paper are shown as follows:First,addressing the challenge of merging global and local features during the process of road scene feature extraction,a novel road scene representation learning segmentation method based on the attention network is proposed.An interactive process between channel-aware information and spatial-aware information is established,fostering an interdependent relationship between the spatial and channel domains,leading to the formation of a structured attention network.Building upon this,the network is integrated with a Transformer-based neural network.This integration successfully amalgamates the benefits of multi-scale fusion and infinite receptive fields these networks provide.Experimental results,derived from autonomous driving scenario datasets,demonstrate that the proposed method enables the network to effectively balance between global and local features.This addresses the challenge of merging global and local features during road scene feature extraction for autonomous driving scenarios,resulting in a significant improvement in the accuracy of the road scene segmentation process.Second,in response to the issue of catastrophic forgetting by road scene segmentation models when trained on new data,research is conducted on an incremental learning segmentation method for road scenes based on knowledge distillation.The introduction of the attention mechanism into the incremental learning segmentation method allows the road scene segmentation method to focus on previously learned content while acquiring new knowledge.Furthermore,the semantic relationships between different classes of pixels are considered during the knowledge distillation process.Leveraging the complementarity of contrastive learning and uncertainty estimation,an uncertainty-based contrastive distillation method is constructed.Experimental results demonstrate the superiority of the uncertainty-based contrastive distillation method over basic contrastive distillation and pixel-level and semantic-level contrastive distillation.This effectively resolves the issue of catastrophic forgetting that occurs when models learn new knowledge in autonomous driving scenarios,thereby enhancing the extensibility of the road scene segmentation method.Third,in light of the poor predictive performance,or even complete failure,of road scene segmentation methods under adverse conditions caused by domain distance,this paper investigates a road scene domain adaptation segmentation method based on data augmentation.An enhanced algorithm of hidden layer space features via bidirectional classlevel adversarial network replaces the original domain-level feature distribution alignment algorithm.This forces the segmentation method to consider class-level consistency,thus avoiding negative optimization during the domain adaptation process.By mixing source and target domains,as well as blending daytime images with their corresponding nighttime images,the domain distance between the source and target domains is effectively reduced.This achieves the dual effect of preventing overfitting in the segmentation model during training and making full use of time-based weak matching information to train style conversion models.Experimental results reveal that the proposed method is able to effectively utilize class-level consistency information and time-based matching information to reduce domain distance,thereby enhancing the versatility of the road scene segmentation method in autonomous driving scenarios.As it can bring safer driving experience,reduce traffic accident rate and effectively solve urban traffic congestion,autonomous vehicle are receiving more and more attention.The architecture of the driving automation system generally consists of three main parts:the perception system,the decision-making system and controlling system.This topic of this dissertation focuses on the road environment segmentation technology in the perception system.The road environment segmentation technology is mainly aimed at recognizing and understanding the vehicles,roads,pedestrians and etc.in the traffic environment to provide data input for the follow-up system.With the upgrading of automatic driving technology from driving assistance to driverless driving,the demand for the accuracy,scalability and versatility of road environment segmentation technology is also gradually increasing.In terms of accuracy,the main problem is that it is difficult to fuse global and local features in the process of road environment segmentation.In terms of scalability,the road environment segmentation model has the problem of catastrophic forgetting for old data when it is based on new data training.In addition,in terms of universality,road environment segmentation technology still has problems such as that models trained under ideal conditions do not predict well under unfavorable conditions and negative optimization of domain adaptation algorithm.For this reason,this dissertation proposes representation learning based on attention network,incremental learning based on knowledge distillation and domain adaptation based on data enhancement to solve the problem of feature fusion,catastrophic forgetting and negative optimization in automatic driving environment segmentation.The main research contents of this paper are shown as follows:First,addressing the challenge of merging global and local features during the process of road scene feature extraction,a novel road scene representation learning segmentation method based on the attention network is proposed.An interactive process between channel-aware information and spatial-aware information is established,fostering an interdependent relationship between the spatial and channel domains,leading to the formation of a structured attention network.Building upon this,the network is integrated with a Transformer-based neural network.This integration successfully amalgamates the benefits of multi-scale fusion and infinite receptive fields these networks provide.Experimental results,derived from autonomous driving scenario datasets,demonstrate that the proposed method enables the network to effectively balance between global and local features.This addresses the challenge of merging global and local features during road scene feature extraction for autonomous driving scenarios,resulting in a significant improvement in the accuracy of the road scene segmentation process.Second,in response to the issue of catastrophic forgetting by road scene segmentation models when trained on new data,research is conducted on an incremental learning segmentation method for road scenes based on knowledge distillation.The introduction of the attention mechanism into the incremental learning segmentation method allows the road scene segmentation method to focus on previously learned content while acquiring new knowledge.Furthermore,the semantic relationships between different classes of pixels are considered during the knowledge distillation process.Leveraging the complementarity of contrastive learning and uncertainty estimation,an uncertainty-based contrastive distillation method is constructed.Experimental results demonstrate the superiority of the uncertainty-based contrastive distillation method over basic contrastive distillation and pixel-level and semantic-level contrastive distillation.This effectively resolves the issue of catastrophic forgetting that occurs when models learn new knowledge in autonomous driving scenarios,thereby enhancing the extensibility of the road scene segmentation method.Third,in light of the poor predictive performance,or even complete failure,of road scene segmentation methods under adverse conditions caused by domain distance,this paper investigates a road scene domain adaptation segmentation method based on data augmentation.An enhanced algorithm of hidden layer space features via bidirectional classlevel adversarial network replaces the original domain-level feature distribution alignment algorithm.This forces the segmentation method to consider class-level consistency,thus avoiding negative optimization during the domain adaptation process.By mixing source and target domains,as well as blending daytime images with their corresponding nighttime images,the domain distance between the source and target domains is effectively reduced.This achieves the dual effect of preventing overfitting in the segmentation model during training and making full use of time-based weak matching information to train style conversion models.Experimental results reveal that the proposed method is able to effectively utilize class-level consistency information and time-based matching information to reduce domain distance,thereby enhancing the versatility of the road scene segmentation method in autonomous driving scenarios.Finially,conclusive research is carried out via field data collection and verification experiments based on real-world autonomous driving tasks.Field data collected through an onboard autonomous driving experimental platform in real application scenarios are employed to validate the structured attention network and the uncertainty-based contrastive distillation loss function within the constructed vision-based autonomous driving road scene segmentation model.Experimental results from bird’s-eye view vehicle segmentation and trajectory prediction tasks indicate that the proposed methods consistently enhance the prediction accuracy of the road scene segmentation model,thereby confirming the practical application value of the proposed algorithms.,conclusive research is carried out via field data collection and verification experiments based on real-world autonomous driving tasks.Field data collected through an onboard autonomous driving experimental platform in real application scenarios are employed to validate the structured attention network and the uncertainty-based contrastive distillation loss function within the constructed vision-based autonomous driving road scene segmentation model.Experimental results from bird’s-eye view vehicle segmentation and trajectory prediction tasks indicate that the proposed methods consistently enhance the prediction accuracy of the road scene segmentation model,thereby confirming the practical application value of the proposed algorithms.