Research On Visual Semantic Segmentation And Instance Segmentation In Road Environments For Intelligent Vehicles

As an important breakthrough in the restructuring and upgrading of the automobile industry,intelligent vehicles have great advantages in terms of improving traffic safety,traffic efficiency,ride comfort,and reducing energy consumption.Environmental perception,decision planning,and motion control technology are three core components of an intelligent driving system.Later intelligent vehicles’ autonomous navigation and decision-making control are greatly influenced by the perception capabilities of the visual-based image segmentation technology in road environments,which is a critical part of the environment perception system.And semantic segmentation and instance segmentation are two essential subtasks of image segmentation,so it has good practical value to study them in road scenes.The two tasks are different and interrelated.Semantic segmentation is the basis of instance segmentation,and instance segmentation is a high-level extension of semantic segmentation.In recent years,considerable progress has been made in environmental perception with favorable environments.However,the semantic segmentation of images in autonomous driving under adverse weather conditions is still very challenging.In particular,the low visibility at night greatly affects driving safety.This paper firstly aims to explore image segmentation in low-light scenarios,thereby expanding the application range of intelligent vehicles.With the continuous improvement of the precision of scene parsing in the field of single-car intelligence,the concept of semantic segmentation has been further extended,that is,from the pixel level to the instance level of instance segmentation.Thus,in order to enrich and improve the image segmentation,this paper also expands the multi-objective instance segmentation for complex traffic scenes on the basis of semantic segmentation.The main research contents can be summarized as follows:(1)A low-light semantic segmentation system based on supervised learning is designed to address the issue that the semantic segmentation algorithm is not adaptive in low-light scenarios and thus affects the safe driving of intelligent vehicles.Considering the scarcity of labeled large-scale nighttime data,synthetic data collection and data style transfer that uses images acquired in the daytime,are performed based on the autonomous driving simulation platform and generative adversarial network,respectively.In addition,a novel nighttime segmentation framework is proposed to effectively recognize objects in dark environments,aiming at the boundary blurring caused by low semantic contrast in low-illumination images.Specifically,the framework comprises a light enhancement network which introduces semantic information for the first time and a segmentation network with strong feature extraction capability.Extensive experiments on the test datasets of Dark Zurich and Nighttime Driving show the effectiveness of the proposed method compared with existing state-of-the art approaches,with 56.9% and 57.4% m Io U respectively.(2)A novel multi-head instance segmentation framework is proposed based on the onestage detection method for complicated traffic scenes to effectively balance the accuracy and inference speed of the algorithm.Specifically,the proposed framework comprises of a backbone,a feature fusion module and a multi-head mask construction module.Firstly,complete high-dimensional feature maps are obtained by adding residual structures to the backbone.Secondly,in order to generate discriminative feature representations,the feature pyramid module is reconstructed by introducing self-calibrate convolutions and the information propagation path is improved by global attention mechanisms,so as to further optimize the feature fusion module of the proposed framework.Finally,a multi-head mask construction mechanism is proposed to significantly both improve the performance of large and small targets by refining the size distributions of instances in the traffic scenes.The proposed work on the open-source dataset Bdd100 k achieved 23.3% and 19.4% m AP@0.5:0.95 on bounding boxes and segmentation masks,respectively.Compared with the baseline,the approach increased by 5.2 % and 2.2 % on average.(3)Real-vehicle verification,ablation validation using available datasets,and experimental analysis of the proposed semantic segmentation and instance segmentation algorithms in road scenes are finished.Firstly,a comparative ablation experiment on the core components of the proposed method was performed on a publicly available dataset to verify the effectiveness of submodules.Secondly,the real-world road experiments of the semantic segmentation and instance segmentation on the self-built real-vehicle platform show that the proposed algorithms can adapt to the actual driving environments with good accuracy and reliability.In order to address the cognitive challenge of refined scenes of intelligent connected vehicles,this paper explores the visual image segmentation of road environments,gradually expanding from semantic segmentation research in low-light scenes to higher-order instance segmentation.The proposed semantic segmentation algorithm under low illumination has good accuracy on two public datasets and can be adapted to practical scenarios.The proposed efficient instance segmentation outperforms the original approach significantly in the public dataset.It also performs well in real-world scenes,satisfies real-time requirements for automated driving,and has good engineering application value.