Research On Touch-Based Interaction And Stylized Rendering Techniques For Mobile Devices

With the rapid development of mobile computing technology,mobile devices have become an indispensable tool in people’s daily work and life.In recent years,the research of mobile device oriented human-computer interaction technology has attracted the attention of academia and industry.How to more harmoniously,naturally and efficiently adapt to business needs is one of the hot issues in the field of human-computer interaction research.Research,development and application of mobile device oriented natural human-computer interaction technology not only have practical significance at present,but also play an important role in building an ecosystem of human-computer symbiosis in the future.Creative design software and application system are essential tools for modern enterprise product design and service and are the key to supporting enterprise development and innovation.With the advent of the software cloud era,cloud-based creative design software has become a development trend.Mobile devices provide an important front-end support environment for cloud-based services,greatly improving the user’s interactive experience and convenience.Creative design software needs to support the modeling and representation of large and complex scenes to achieve effective processing of digital multimedia content.Efficient and natural interactive modeling methods for mobile devices and rendering technologies rich in artistic expression are challenges that need to be solved urgently:1)Mobile devices have limited screen space and few interactive means.How do design new interactive paradigms to achieve 3D interactive modeling and text information input?2)How can mobile devices realize real-time rendering of large-scale complex scenes because of their small memory space,narrow network bandwidth,and weak computing power?Aiming at the above challenges,this thesis makes full use of the touch characteristics of mobile devices to research 3D interactive modeling methods and text input methods for mobile devices,and realizes 3D interactive operation based on touch pressure and text input based on touch gestures;According to the computing characteristics of mobile devices,visualization and rendering technology are studied for music,2D and 3D model data in complex scenes,and stylized rendering of complex scenes for mobile devices is realized.This thesis focuses on mobile device-oriented touch interaction,stylized rendering,and cross-device collaboration.The main academic contributions and innovations are as follows:1.An interactive method based on touch technology is proposed.Through the analysis of the characteristics of the touch pressure distribution curve,a pressure sensing level(light pressure,medium pressure,heavy pressure.)division strategy is formulated,and the mapping relationship between the pressure sensing level and the touch command is established,realizing the regulation of the interaction speed.On this basis,a 3D interactive modeling method based on touch pressure is proposed.Users can select,translate,rotate,scale,and Boolean operation objects through touch control,which provides a new method for interactive modeling of 3D scenes on mobile devices.Under the guidance of Fitz’s law,the virtual keyboard is optimized in a ring shape to conform to the characteristics of thumb movement.The sliding input method based on contact tracking is used for text input,which reduces the fatigue of thumb movement.2.A music visualization method based on the emotion model is proposed.Based on the emotion model,a music emotion classification method based on KNN is proposed,which realizes the recommendation of background music in a virtual scene.The fast Fourier transform is used to obtain the sound spectrum data of music,map the audio spectrum data to the HSV color model space,and generate random particle effects based on the emotion model,which realizes the 3D visual display of music on mobile devices.While providing users with an immersive interactive scene,it can effectively achieve emotion management.3.A stylized rendering method for complex scenes is proposed.Based on the classic cyclic warfare generation network(CycleGAN),by introducing the global confrontation loss of binocular lifting,the local confrontation loss of label clipping,the cyclic loss of regional awareness and the high-level semantic loss,cartoon stylized images with more local details can be generated,and the consistency of the local characteristics of the characters can be better maintained.Aiming at the problem of high memory resource occupation and large computation amounts of highly realistic rendering algorithms,this thesis proposes a 3D mesh model simplification strategy,which realizes real-time stylized rendering of 3D models on mobile devices by computing 3D model feature lines and constructing 3D carving brushes.Compared with traditional non-photorealistic rendering methods,this method has the characteristics of less computation and low memory consumption,and the rendering results are more artistic.4.A cross-device interaction framework supporting multi-task processing is proposed.Because of the problems that a single mobile device cannot meet the practical needs in terms of display screen space,computing power,etc.,this thesis designs a communication mechanism between mobile devices,proposes a method of cross-device task delegation and management and realizes the continuous processing and real-time monitoring of tasks.The application research based on the framework shows that the framework effectively improves the interaction ability of mobile devices and expands the application scenarios of mobile devices.