Research And Application Of Vibration Tactile Based On Music Element Driving

The combination of vibration tactile and music is one of the directions in the research upsurge of the combination of human-computer interaction and art.The research core of the combination of vibration tactile and music is to use vibration tactile to represent music information.The difficulty is that the vibrotactile mapping of music elements is affected by many factors such as the performance of vibration drivers,music signal processing algorithms,and multi-scene differences.There is a lack of research on haptic interaction that conforms to the characteristics of human vibrotactile perception,combines highperformance algorithms,and integrates multi-element mapping,as well as research on the application of vibrotactile music for musical instrument and dance learning assistance.In order to solve the above problems,based on the national key R＆D project,this thesis aims to study the vibration tactile expression of music elements and solve the application of vibrotactile assistance in musical instrument and dance teaching.This thesis proposes a vibrotactile expression scheme for music elements and implements a vibrotactile expression system driven by music elements.The main contents are as follows:（1）Research on the perceptual characteristics of human tactile sensation to vibration.Based on the psychophysical method,this thesis designs and carries out the experiments of vibration tactile perceptual characteristics,including vibration tactile perception threshold experiment and emotion response experiment of vibration tactile perception.The perception threshold of four vibration perception parameters: vibration intensity,duration,interval time,space two-point distance and conclusion of individual emotional response differences under different combinations of vibration perception parameters are used as the selection principle of vibration tactile perception parameters,which provides an important theoretical basis for the subsequent vibration tactile interaction design of music elements.（2）Design of vibration tactile expression of music elements based on human vibration tactile perception characteristics.Based on the characteristics of human vibration tactile perception,the music elements at the auditory level are extracted by using the music information retrieval algorithm,and its vibrotactile mapping is designed.For the musical instrument and dance learning scene,based on the proposed vibrotactile mapping of music elements,the stimulation response compatibility method is used to design the vibration tactile expression scheme of music elements in line with the specific scene.Finally,the best tactile expression scheme of beat and rhythm vibration of musical instrument and dance learning scene is higher than 88% in user response accuracy,and the response time is less than 0.5s.（3）Construction and evaluation of multimodal music and dance teaching assistant system driven by music elements.In this thesis,the proposed vibrotactile expression method driven by music elements is transplanted to vibrotactile wearable devices,and a multimodal teaching assistant system integrating vibrotactile feedback of music elements is built.In order to verify the reliability and feasibility of the system,this thesis makes a practical test and application scenario evaluation.Compared with the traditional listening and visual channel learning mode,users use the system to improve the accuracy and response speed of instrument playing and dance movements.