Automatic Generation Of Labanotation Based On Deep Learning

In recent years,the use of digital methods to record and protect traditional cultural assets has become an important research topic.Similar to the staff in music can be used to record the composition,dance notations are generally used to record human movements.As a scientific and symbolic system,Labanotation is one of the most widely used notation systems in the world.However,manual notation takes a lot of time and effort,so it is necessary to use computer technology to generate Labanotation automatically.The main work of this paper is to study efficient algorithms from motion capture data to automatically generate Labanotation.Firstly,according to the characteristics of human movements,skeleton features conforming to the topological structure of human body are designed.Then,by analyzing the difference in upper and lower limb movements of human body in writing Labanotation,efficient deep learning algorithm is first proposed to identify the upper and lower limb movements.Finally,the proposed features and motion recognition algorithms are integrated into the platform of automatic generation of Labanotation,which realizes an end-to-end system of automatic generate Labanotation from motion capture data.The main innovation work is summarized as follows:(1)By analyzing the movement characteristics of human body,skeleton features that conform to the topological structure of human body are designed.Firstly,the BVH motion capture data stored in Euler angle form is converted into three-dimensional space data in the world coordinate system.And then the joint nodes adjacent to each part of human body are connected to form a skeleton vector feature,which is more robust and discriminative than ever and can better represent the spatial position and direction of human movements.(2)An upper limb posture recognition algorithm based on extreme learning is proposed by analyzing the notation of human upper limb movements in Labanotation.The algorithm can be used to generate fast and efficient upper limb movements of Labanotation.Upper limb movements focus on the final posture in Labanotation and do not pay attention to the relationship between the front and the back,the proposed recognition algorithm based on extreme learning can classify and recognize the upper limb movement very efficiently.(3)This paper proposes a multi-layer joint network model based on Long Short-Term Memory network(LSTM)and Convolutional Neural Network(ConvNet)algorithm for lower limb movement recognition.Lower limb movements focus on the dynamic process of movement in drawing Labanotation symbols.LSTM's self-joining network structure makes it a natural advantage to process dynamic time series data and ConvNet is very good at learning the spatial information of movement.LSTM model is integrated with the ConvNet model so that the two can complement each other and achieve a more accurate recognition effect than single model.(4)Based on the new features and new methods proposed above,an end-to-end automatic generation system from motion capture data to Labanotation is designed and implemented.The input of this system is the motion capture data obtained by motion capture device of the performer,and then through the extraction of the above new features and the recognition of human movements by the new methods,Labanotation score can be automatically drawn and generated in a few seconds.The generated Labanotation file can be saved locally,which realizes the perfect combination of computer technology and Labanotation.