Research On Human Motion Transfer Based On Deep Learning

Human motion transfer has broad application prospects in virtual try-on,gaming,film and video production,and other fields.The goal of human motion transfer is to copy the human motions from the source video onto the character in the target video,generating a synthetic target video.Existing human motion transfer methods consider human pose or human mesh as the foundation for motion transfer,building a generative network model based on human pose or human mesh to achieve image generation of human characters.Through investigation of existing motion transfer algorithms,we found that they still face two key issues: First,human motion transfer techniques relying on generative adversarial networks typically necessitate a substantial quantity of training data to attain satisfactory outcomes.Second,the current motion transfer models ignore poorly performing training samples,resulting in poor generation quality for certain movements.In this paper,we further researched and proposed two deep learning-based human motion transfer methods.(1)Human motion transfer method based on optimal transfer theory and local enhancement.This method uses human pose as the base for motion transfer,designing a generative adversarial network based on optimal transport theory and local enhancement to achieve image generation.The method consists of four modules: data preprocessing,pose-guided appearance generation,local enhancement of body parts,and human foreground-background fusion.Specifically,the data preprocessing module performs two key tasks,namely extracting the human pose of the source character from the source video,and segmenting the target video into human foreground and background.The human foreground appearance generation module uses a generative adversarial network to generate human foreground images based on human pose.The local enhancement module includes self-supervised facial enhancement and multi-local generative adversarial network-based body parts enhancement.The human foreground-background fusion module is used to fuse the body foreground and background to generate the motion transfer image.(2)Episodic memory network-based human motion transfer method.Existing human motion transfer algorithms ignore poorly performing training samples and do not refine those movements.Therefore,this method proposes a human motion transfer network based on episodic memory module.Specifically,the proposed approach involves storing both underperforming training samples and randomly selected samples in an episodic memory unit,followed by retraining the network model using this memory.Furthermore,to improve the generation quality of the synthetic images,a perceptual loss function is introduced to supervise image generation.The experimental results demonstrate that compared with the existing algorithms in the i PER and Complex Motion datasets,human motion transfer method based on optimal transfer theory and local enhancement has improved the peak signal-to-noise ratio by 1.38 and 1.47 respectively.Compared with human motion transfer method based on optimal transfer theory and local enhancement,episodic memory network-based human motion transfer method has improved the peak signal-to-noise ratio by 0.86 and 1.29,respectively.In this paper,ablation experiments are carried out on the proposed algorithm to prove the effectiveness of the methods.