Face Pose Detection And 3D Reconstruction Based On RGB-D Image

Transcranial magnetic stimulation technology is a physiotherapy method that uses pulsed magnetic field to act on the central nervous system of the brain to treat neuropsychological diseases such as epilepsy and cerebral palsy.In the process of transcranial magnetic stimulation,it is important to accurately and quickly detect the position of the acupoints and nerve regions in the skull.Based on this purpose,this thesis proposes a face pose detection and 3D reconstruction method combining color image and depth image.By detecting the pose of the current face and combining the transformation relationship between the three-dimensional face coordinate system and the medical skull coordinate system,the robot arm equipped with the magnetic stimulation coil is automatically positioned following the movement of the user's head to save medical resources and improve the user's treatment experience.The research contents of this thesis mainly include RGB-D image preprocessing,face region and feature points detection,3D face point cloud model reconstruction and face pose detection.Firstly,for the problem that the face pose detection based on two-dimensional color image is sensitive to the environment and pose,this thesis uses the depth camera to acquire the RGB-D image of the face and detect the three-dimensional feature points to define the three-dimensional face coordinate system;And then,this thesis prepares a 3D face point cloud model with complete data and high precision for real-time detection of face pose using point cloud registration method to improve the accuracy of attitude detection;In the pose detection stage,this thesis proposes an initial detection method which obtains face poses by solving the rigid body transformation relationship of the three-dimensional face coordinate system between the face cloud to be detected and the zero-pose three-dimensional face point cloud model.Because the transcranial magnetic therapy instrument is also for epilepsy patients and children,this thesis uses the ICP algorithm to perform point cloud fine registration based on the initial face detection algorithm to improve the attitude detection accuracy when the user's head is large.Using the initial face detection method as the point cloud coarse registration algorithm can prevent the ICP point cloud registration from falling into local optimum.The absolute error of the face pose detection in the single axis is within the(7)0,2~o(8)interval,which can meet the clinical application standard of the transcranial magnetictherapy instrument.This thesis simulates the medical environment of transcranial magnetic therapy instrument,and performs face detection experiments in a laboratory where the light source is uniform and the light is sufficient.The experimental data shows that the total average absolute error of the face pose initial detection algorithm in the range of face pitch(-60~o,30~o),left and right deflection and left and right rotation(-60~o,60~o)is 1.141~o,the error maximum is1.932~o,which is close to the upper limit of the allowable error of transcranial magnetic.However,in the range of pitch(-40~o,20~o),left and right deflection and rotation(-30 ~o,30~o),the absolute error range is(0.8~o,1.25~o),which can meet the clinical needs of facial pose detection for general users.Experiments show that the combination of point cloud coarse registration and fine matching can reduce the total average absolute error to 0.663~o,and the maximum error is only 1.010~o.It can also meet the clinical needs of face pose detection when the user's head movement is large.