Research And Application Of Visual-inertial SLAM Using Point And Line Features In Complex Working Environments

In complex unknown operating environments,visual SLAM technology often has the problems of inaccurate localization and tracking lost when applied to wearable systems.The visual-inertial SLAM technology based on point and line features is studied for the complex and diverse working environment under the interference of light changes,moving objects,etc.On this basis,with the localization of inspection operators as a typical application scenario,the wearable localization system is designed,the wearable hardware platform is built and the application software for inspection localization and monitoring is developed.Firstly,according to the characteristics of complex operating environments,a new visual SLAM based on the fusion of point and line features is proposed.The management of feature tracking and matching in ORB-SLAM2 is applied to both point and line features.The re-projection error model of line features is analyzed.A new optimization objective using point and line is constructed and the Jacobian matrix is derived in detail.Poses of camera and features are solved through non-linear optimization.Furthermore,a method based on multi-state constrained Kalman filter to fuse visual features with IMU is proposed as the front-end of point-line fusion SLAM,which combines the advantages of IMU data and visual information,solving the problem of inaccurate localization under the interference of violent motion,abnormal camera exposure,light changes and moving objects.For the loop detection module in visual SLAM,two improved methods are proposed.One is based on the point and line segment word pair method.The co-occurrence information and spatial proximity of point and line features are considered in loop closure detection,which overcomes the perceptual aliasing caused by similar objects in different scenes.The other is based on deep learning.The jamming area in the image is removed through the semantic segmentation network.To complete loop closure detection,Superpoints are extracted from the static image region.Besides,a triangulation verification and an epipolar geometry verification are designed,which significantly improve the accuracy and recall rate of loop closure detection in scenarios with high similarity and interference.Based on the research above,a wearable localization system for inspection operations is designed.Application software for patrol localization function and monitoring function is developed.The localization function software mainly includes modules such as environment map construction and real-time localization.The monitoring function software is based on the Reactor mode server and provides visualization functions and interactive functions.Finally,the real/simulated operating environment datasets and benchmark datasets are used for experiments on the modules and the whole system.The results verify the effectiveness of the proposed method.