Research And Implementation Of Virtual Character Assistant In Augmented Reality Environment

Augmented reality technology can accurately locate in the real environment,superimpose the virtual information generated by the computer on the real world scene seen by the user,and realize the expansion of the real world information.Augmented reality technology has three characteristics: virtual and real combination,real-time interaction,and three-dimensional registration.It has been widely used in many fields.The thesis builds a virtual character assistant in the augmented reality scene to help people's daily indoor life.Its immersive three-dimensional display makes virtual characters more realistic and vivid.With the help of speech recognition technology,people can interact with virtual character assistants through speech.Through HoloLens' s perception of the surrounding environment,indoor navigation can provide a path for virtual character's motion.The thesis mainly completed the following work:(1)The thesis studies the human-machine interaction technology in augmented reality scene.The illumination model and rendering process in 3D display technology are analyzed,and various special display effects are realized in the shader.Based on the HoloLens platform,two interaction methods,sight gaze and gesture recognition,are developed.The Chinese speech recognition and Chinese semantic analysis module based on Azure cloud service is developed,which expands the HoloLens Chinese voice interaction function.(2)Indoor navigation functions in an augmented reality environment is developed.The global coordinate system of environment is established by image identification technology,and its relationship with HoloLens coordinate system is established.A real-time modeling module for 3D environment based on HoloLens is developed.Then voxelize the model of the environment,determine and divide the walkable area to generate a convex polygon navigation grid.A path is found by the A* search algorithm applied to the navigation grid.After smoothing,the available route is finally generated.(3)The thesis studies the modeling and driving of virtual character,proposes a virtual character assistant framework and key technologies.A human appearance model is built by using Kinect to scan the human body.After repairing and re-texturing,the model is more accurate.Establish the matching relationship between human body appearance model and bones,and bind the bones to the appearance.(4)The thesis proposes the framework of the augmented reality system for the indoor life.Combined with Chinese speech recognition,indoor navigation and virtual character assistant,the thesis builds software and hardware platform to develop the indoor augmented reality assistant system.