Research On Key Technology Of Interactive Visualization Of Flowers And Plants Based On Somatosensory Interaction

Flowers and plants visualization is an indispensable part of modern digital forestry research.Form the early construction of flowers and plants models to the simulation of flowers and plants morphology,a relatively complete research system has been formed.With the full popularity of human-computer interaction,people are no longer satisfied with the simple flowers and plants display functions,but turn the research focus to the realization of visual interaction functions,and propose a variety of interaction-based flowers and plants visualization system.However,this kind of system mainly realizes the change of some model parameters through simple two-dimensional or three-dimensional HCI,thus realizing the simulation of virtual plants and lacking artificial active intervention.How to extend the depth and dimension of visual interaction of flowers and plants based on the continuous development of HCI technology and beyond the existing simple interaction methods has become a new research hotspot.With the rapid development of somatosensory interaction related technology and the research of flowers and plants visual simulation technology in recent years,we analyzes the existing problems of the interactive flowers and plants-based visualization system,and clarifies the feasibility of the application of somatosensory interaction technology in the forestry engineering field,then the interactive visualization method and research framework of flowers and plants based on somatosensory interaction are proposed,and the overall design of interactive visualization system is presented.On this basis,the paper studies on the virtual hand precise grasp and dynamic hand gesture recognition in detail,which are the two key issues for the development of interactive visualization system of flowers and plants based on somatosensory interaction.The main research contents are as follows:(1)An interactive visualization of flowers and plants based on somatosensory interaction is proposed,that is,a method to directly manipulate flowers and plants in a three-dimensional virtual scene provided by Unity3D by using virtual hands and gestures through the perception of human hand.Compared with the current interaction-based flowers and plants visualization system,the somatosensory interaction technology is closely combined with the visual simulation of flowers and plants,it has realized immersion in the realization platform,human-computer interaction and visual display of flowers and plants,which provides a new idea and dimension experience for flowers and plants visualization interaction.(2)Aiming at the problem of precise grasping interaction between virtual hand and flower plant,this paper constructs a Leap Motion virtual hand model suitable for visual interaction of flower plants,and proposes a precise grasping interaction method based on virtual hand.Firstly,real-time interactive simulation of virtual hand is realized by posture mapping and collision detection optimization between real hand and virtual hand;Secondly,a grasping algorithm based on physical and visual information is proposed,which can grasp flower and plant model accurately and stably;Finally,the interpenetration between virtual hand and object is optimized according to the kinematic constraints of human hand,and the precise grasping interaction of virtual hand is realized.(3)Aiming at the problem of dynamic hand gesture recognition in the interactive process of flower and plant visulization,this paper proposes a dynamic hand gesture recognition method based on improved HMM-PSO algorithm using Leap Motion as input device.Firstly,two Leap Motion are used to acquire human hand motion data from different angles and process them by data fusion.Secondly,a new feature vector suitable for visual interaction of flowers and plants is constructed.Finally,an improved HMM-PSO classifier is used for dynamic hand gesture recognition.The improved method in this paper can recognize gestures efficiently and accurately,and simulate flowers and plants in real time.It satisfies the human-computer interaction function of the interactive visualization system of flowers and plants.To verify the feasibility and effectiveness of the interactive visualization method based on somatosensory interaction,a virtual scene and display platform was built by selecting Unity3D as a mature development engine,combining the somatosensory devices Leap Motion and Oculus Rift,and a prototype system was designed and implemented.The user test shows that the combination of virtual hand grasping interaction and dynamic gesture interaction can realize direct interaction and immersion stereoscopic visualization of flowers and plants in 3D virtual environment.This paper solves the two key issues of virtual hand precise grasp and dynamic hand gesture recognition in the establishment of interactive visualization system of flowers and plants.It provides a new idea and method for promoting the development of flowers and plants simulation towards visual interaction and improving the visualization of flowers and plants.It also lays a solid foundation for the combination of digital forestry and virtual reality.