Vision-Assisted Multi-Source Fusion Localization In Complex Environments

With applications such as autonomous driving in smart cities to high speed development,the demand for accurate,stable and reliable location services has also exploded.Global Navigation Satellite System(GNSS)is currently the most widely used navigation and positioning system,which can provide high accuracy location services worldwide.However,due to the inherent disadvantages of satellite signals,such as weak signal,poor penetration ability and susceptibility to interference,the application of GNSS system is greatly limited.Based on multiple positioning means and fusion of multiple sensor data,it is the trend of navigation and positioning technology development to provide the same spatial PNT(Positioning,Navigation and Timing)service with space-time reference.Therefore,this paper conducts research on multiple positioning means,including 5G-based positioning technology,vision sensor-based positioning technology,and multi-sensor fusion positioning technology for vision-assisted GNSS systems,for complex scenarios where GNSS performance is limited or fails.The main research contents and contributions of this paper are as follows:1.An indoor localization technique based on the AOA and TOF fingerprints of the new channel state detection reference signal of the fifth generation mobile communication standard is proposed,and the fingerprint matching algorithm is improved to achieve real-time reliable location estimation in indoor environment.Traditional fingerprinting algorithms usually use signal strength values(RSSI)as location fingerprints to achieve localization,but RSSI values do not provide a complete description of the channel,and the lack of direction information limits localization accuracy.In this paper,we innovatively use the angle of arrival and time of flight as location fingerprints to achieve localization,and the development of the new 5G air interface standard also lays the foundation for us to estimate AOA and TOF using channel state detection(CSI)signals.Traditional AOA observations are subject to large errors due to interference from multipath signals,especially NLOS signals,while the millimeter wave high attenuation characteristics of 5G and the strong directional characteristics of large-scale antennas make multipath effects in indoor environments smaller.Therefore,this paper achieves accurate estimation of AOA and TOF parameters by using spatialtemporal 2D spectral algorithm,and further establishes offline fingerprint library and improves online fingerprint matching algorithm to innovatively apply 5G communication technology to the field of positioning and navigation to achieve high-precision positioning in indoor environment.2.A detection algorithm for dynamic objects based on multi-view geometry and image processing techniques is proposed and successfully applied to the field of visual localization,improving the visual SLAM algorithm based on the feature point method,overcoming the problems of traditional visual SLAM techniques based on static assumptions,and achieving high accuracy localization in dynamic environments.The presence of dynamic objects in the environment can make the pixel differences between adjacent image frames contain not only camera motion information but also motion information of dynamic objects,thus rendering the visual localization algorithm ineffective.In this paper,we investigate the dynamic object detection algorithm based on the pair-pole constraint,and use it to optimize the feature point-based visual SLAM algorithm,which detects and filters out dynamic feature points in the feature tracking part of the visual SLAM algorithm to reduce its localization error in dynamic environments.The performance of the proposed algorithm is significantly better than the ORB-SALM2 algorithm and has similar localization accuracy compared with other dynamic SLAM algorithms,DynaSLAM,but since DynaSLAM uses convolutional neural networks to filter out dynamic feature points,we can achieve higher real-time performance using image processing techniques.Overall,our visual localization algorithm based on the ORB-SLAM2 algorithm optimization can achieve high accuracy localization in dynamic environments.3.We propose the Dynamic-SLAM visual localization algorithm based on semantic segmentation and S-AKF adaptive Kalman filtering algorithm based on semantic information,and further build a vision-assisted GNSS multi-sensor fusion localization system.For multi-sensor fusion localization technology,maintaining stable and reliable system operation under partial sensor anomaly or even failure scenarios is an important requirement put forward by the new generation of PNT services.In this paper,a multisensor fusion localization system using vision-assisted GNSS is proposed to address this issue.In terms of GNSS and vision multi-source data fusion,this paper investigates a semantic segmentation algorithm based on deep learning neural network to realize the extraction of environmental semantic information,and studies the base-S-AKFz adaptive Kalman filtering algorithm based on the extracted semantic information to obtain highly accurate position estimation by adaptively adjusting the filtering parameters in case of GNSS data anomalies.On the other hand,the visual localization algorithm is optimized based on the extracted semantic information,and the high-precision GNSS absolute position information is used to help the visual localization algorithm complete the initialization as well as eliminate the accumulated errors to improve the localization accuracy of the visual localization algorithm in the dynamic environment.Therefore,the vision-assisted GNSS multi-sensor fusion positioning system studied in this paper can achieve reliable positioning capability even in an occluded environment,and enhance the positioning accuracy and positioning range of GNSS system in urban environment.