Design And Implementation Of A Mobile Augmented Reality System Based On Deep Learning Illumination Estimation

Augmented reality is an interactive technology that integrates virtual scenes with real-world scenes.It has been widely used in military,medical,industrial manufacturing,education and other fields.With the prevalence of smart phones and other mobile devices,various augmented reality applications for mobile devices are also emerging.However,the existing mobile augmented reality applications generally underperform on illumination consistency.Illumination consistency means that lighting effects on rendered virtual object should conform to the lighting environment in real world.In order to achieve the illumination consistency between virtual objects and real scenes,it is necessary to make good illumination estimation for the illumination environment of real scenes,but most of the state-of-the-art illumination estimation technologies are not suitable for mobile devices.Aiming at the lack of suitable illumination estimation technology for mobile devices,this thesis proposes an illumination estimation solution for mobile devices based on deep learning.The contributions are as follows:(1)For the indirect light diffuse reflection estimation,a deep learning illumination estimation algorithm is proposed in this thesis.The depth-wise separable convolutional neural network is applied to construct the neural network,which greatly reduces the amount of network parameters and computation.Meanwhile,dual cameras on smart mobile devices are used to increase data input.Experiments show that the overall accuracy of the algorithm is not lower than state-of-the-art illumination estimation algorithms,and the time of illumination estimation on mobile devices is significantly reduced.(2)For indirect light specular reflection estimation and direct light estimation,an environmental cubemap texture construction method for mobile devices is proposed.a cube environment mapping texture construction method for mobile devices is proposed in this thesis.On the one hand,an accurate environment mapping texture generation scheme is proposed;on the other hand,a fast generation scheme of approximate environment mapping texture to meet the needs of real-time rendering in the process of accurately constructing texture is provided.The environment mapping texture generated by this method can significantly improve the visual effect of virtual objects.(3)Combined with the above illumination estimation methods,this thesis designs and implements an augmented reality system for mobile devices.The system can render the combination of virtual and real scenes with good illumination consistency and has good scalability.