Research On Key Techniques Of Optical See-through Heads-up Display System

The augmented reality technology intends to superimpose a virtual image generated by a computer onto a real environment in real time to enhance perceived information about the real environment.An optical see-through heads-up display system projects virtual image directly in front of the eye's line of sight.It can provide clear front and background views.Currently,it is one of the main realizations of augmented reality,and the development of the related key technology has attracted considerable attention.Further,a few problems related to the key technology of optical see-through heads-up display systems are observed to persist,for instance,the system calibration accuracy is not adequately high,the registration accuracy of three-dimensional tracking is low or unstable,and the extraction of scene information is influenced by the camera attitude.By focusing on the aforementioned technical difficulties,the authors studied the key technologies associated with the optical see-through heads-up display systems for resolving the existing problems.This study mainly intended to solve the calibration problem associated with the optical see-through heads-up display systems.A calibration method is proposed by combining the optical display image distortion and camera imaging distortion correction.Different coordinate systems are defined by analyzing the relation among the system components.Because the camera and the optical display system will introduce a certain degree of virtual image distortion during calibration,the distortion of the camera and the optical display part must be considered and corrected while establishing the model.The nonlinear regression estimation method is used to solve the model and calculate the system calibration parameters.This calibration method combines the optical display image distortion and camera imaging distortion correction,improving the calibration accuracy of the optical see-through heads-up display systems.In this study,a three-dimensional tracking registration method is proposed based on feature point cloud matching.The algorithm converts the pose estimation problem of cameras into the matching problem of the point cloud.By transforming the acquired two-dimensional image information into a three-dimensional point cloud,the pose estimation model of the camera is directly established between the point clouds,and the pose transformation matrix can be obtained.This not only improves the computational speed of the algorithm but also ensures the accuracy of the three-dimensional tracking registration algorithm.When the sensor acquires an image,the results of three-dimensional tracking registration are adversely affected when the feature points cannot be effectively retained,resulting in a small number of features.To solve this problem,semantic segmentation is employed for obtaining the content of the scene.Further,semantic segmentation can help to achieve stable registration of three-dimensional tracking and to improve the registration accuracy.Therefore,a method of semantic segmentation based on improved single shot multi-box detector deep convolution network is proposed,which does not need to extract features in advance,but uses learning method to train the network.The front end of the network uses VGG-16 to sample the image feature map,and then restores the size of the feature map layer by layer.At the same time,it obtains the classification results of different objects,i.e.the semantic content information of the scene.In order to make the three-dimensional tracking algorithm adapt to the scene with certain structural complexity,a registration algorithm for three-dimensional tracking based on multiobjective constraints is proposed,which combines the semantics segmentation results of the scene.The algorithm uses the pixel classification results of different objects to transform the two-dimensional image of the object into three-dimensional semantic object point cloud.According to the multi-objective constraints between the semantic point clouds,the pose estimation model is established and the registration matrix is obtained.However,only considering the texture or structure information of the scene makes the algorithm have limitations.In order to make the algorithm more stable,a three-dimensional tracking registration method based on gray-geometric constraints is proposed.The gray-scale constraints and geometric constraints are modeled simultaneously to improve the stability of the algorithm while ensuring the accuracy of the algorithm.Finally,an optical see-through heads-up display prototype system for assistant driving is constructed.By analyzing the driving environment,the optical display system is designed,and the road information is enhanced by combining the algorithm mentioned above.Experiments verify the real-time performance,stability and system security of the algorithm,thus improving driving safety.