| Orthognathic surgery aims to solve the problems of dental function and facial aesthetics,and achieve the symmetry and coordination of facial structure through surgery.With the continuous improvement of patients’ aesthetic requirements,the esthetics oriented treatment concept has gradually become the consensus of orthognathic surgery treatment[1].Patients with Dento-facial Deformity(DFD)often have social difficulties and even low self-esteem and depression due to facial Deformity.Therefore,in clinical treatment,orthognathic surgeons should determine the treatment targets of orthodontic and surgical operations based on aesthetic evaluation and combined with the degree of dental deformities[1].The evaluation of facial aesthetics mainly depends on the measurement and analysis of soft tissue.The appearance of soft tissue cephalometry has transformed the treatment concept of orthognathic surgery into "harmonious facial features and good function coexist".This change of concept suggests that the clinician should pay attention to the morphology of soft tissue when making the surgical plan.The analysis of soft tissue morphology and structure plays an important role in facial aesthetic evaluation and postoperative effect evaluation.Currently,the diagnosis of DFD is usually based on cephalometric analysis of lateral X-ray or CT data.However,orthognathic surgeons can only analyze the ratio between facial width and facial height based on soft tissue images.When evaluating facial aesthetics,doctors usually measure facial data by using a ruler and facial arch according to their work experience,which is time-consuming,subjective and highly dependent on experience[1].Therefore,there is still a lack of an objective and rapid assessment method for facial soft tissue in clinical work.In recent years,Artificial Intelligence(AI)has developed rapidly due to its technology of simulating human thinking[2].AI has the ability to extract information from large amounts of data,and can use its automaticity to perform calculations in order to be able to diagnose specific diseases in a relatively short time,so that clinicians can benefit greatly.The margin of error in the clinical application of AI will have a significant impact on the judgment and prediction of surgical procedures.Convolutional Neural Network(CNN)in Deep Learning(DL),as a widely used model in medical image analysis,has good image processing capabilities[1].Heat map regression is a mainstream method for facial key point recognition,which has the advantages of intuitive visualization,suitable model selection and interpretability of results.Although soft tissue cephalometry plays a very important role in the diagnosis of facial deformities,it has not been widely used in clinical work due to its cumbersome measurement methods and subjective measurement results[1].Face detection technology based on deep learning has been widely studied.However,due to the privacy of medical data,there are no deformities face datasets specifically used for orthognathic surgery research,and no network models have been reported that can simultaneously detect a variety of posture images such as front,side,smile,and open mouth,and can be used for facial deformity diagnosis.In this paper,convolutional neural network technology is used to detect and recognize facial soft tissue images,and heat map regression algorithm is used to recognize facial key points.The experimental results show that this method can accurately detect facial soft tissue images,and can effectively identify facial key points.In this study,we successfully collected professional facial deformity image data sets that can be used in orthognathus surgery research,and developed a Back High-Resolution Network model(BHR-net)based on thermal map regression algorithm to achieve accurate recognition of multiple pose image landmarks in maxillofacial region.According to these automatically recognized anatomical landmarks,clinicians can objectively obtain the data of facial morphometric indicators,and provide a reliable method for facial soft tissue topography analysis.Objective:A topography intelligent analysis platform based on multi-pose facial images was constructed,and based on the neural network of the platform,accurate recognition of facial malformed soft tissue markers and accurate calculation of measurement indicators were realized,so as to realize reliable diagnosis and measurement analysis of facial shapes.Methods:1.Construct BHR-net neural network:By developing the software PyCharm Community Edition2021.3.1,a plotting tool that can be used to label the training set and the test set of the neural network model is built in Python3.9.7,so as to realize the labeling of data set and the unification of data format.Meanwhile,based on the heat map regression algorithm,the core network model of this study was constructed in the Pytorch 1.12.1 framework,and then the model was trained in the Linux system ubuntu20.04,and compared with other published network models in the open source data set to confirm the effectiveness of the model.2.BHR-net network model parameter research:By further studying the parameter setting of BHR-net network model,the balance between prediction accuracy and computing power is achieved.This study mainly focuses on the pixel value of the output image and the training data set,and uses the normalization error to evaluate,so as to provide a reliable neural network model for the follow-up experiment to ensure the normal conduct of the experiment.3.The accuracy of BHR-net identification marks:The accuracy of markers identified by BHR-net is the basis of clinical application of the network model.This part of the experiment was based on the constructed BHR-net neural network,verified by training on a custom data set(volunteers’ facial multi-pose images and 1080 patients with facial development deformities),and then tested in a test set composed of 50 volunteers.Then,the mean value of manually marked marks was used as the gold standard,and the coordinate value of the marks identified by the model was compared with the mean value of the manual group to judge the accuracy of BHR-net in the recognition of facial soft tissue marks.At the same time,in order to ensure the safety of patient data used in AI,this experiment tried to invert the custom data set,and conducted preliminary verification on the invert test set.4.Accuracy of measurement indicators obtained by BHR-net:Combined with the practical significance of facial soft tissue measurement indicators in clinical application,the results of relevant facial measurement indicators were calculated according to the soft tissue markers identified by BHR-net,and compared with the manual group(gold standard).Through the calculation of facial related measurement indicators,the practical application value of BHR-net in clinical work was further evaluated.5.Clinical application research based on BHR-net:In this part of the experiment,facial images of 30 patients with facial deformity and 40 patients with snoring symptoms were collected.Then,based on the BHR-net network model,the above patients were identified and diagnosed,and the preoperative diagnosis accuracy rate and postoperative facial changes of orthognathic patients were analyzed.Meanwhile,facial features of patients diagnosed with OSA were analyzed.6.Social group evaluation of postural smile survey:The experiment carried out a questionnaire survey including 34 provinces,autonomous regions and special administrative regions(1469 people).Through the analysis of questionnaire survey results,this paper explores the aesthetic evaluation criteria of postural smile among different occupational groups in society.7.BHR-net identifies the accuracy of postural smiles:Based on the conclusion of the questionnaire survey and the BHR-net network model,this part of the experiment focused on the four measurement indicators of smile aesthetics involved in the questionnaire,and studied the accuracy of the network recognition of smile landmarks and the accuracy of the measurement indicators,so as to explore the accuracy of BHR-net in the application of local images,laying a solid foundation for the subsequent experimental research.Results:1.Successful construction of BHR-net neural network:This part of the experiment successfully developed a set of plotting software that can be used for the labeling of data sets,and realized the measurement of the distance between any soft tissue marks and the calculation of the Angle between any three soft tissue marks in the software,and realized the unification of the label data format.At the same time,a neural network model based on BHR-net is successfully constructed.Compared with other published network models,Mean Normalized Error(MNE)and 10%Failure Rate(FR10%)of the BHR-net network model constructed in this experiment are minimal,and the results are relatively optimal.2.The BHR-net network model parameters:In this study,the resolution of the output image was finally set to 128×128.Through the study,it was found that the training effect based on the open source data set alone could not meet the medical application.Therefore,this study needed to collect a custom data set conforming to the purpose of this study,so as to realize the application of BHR-net network model in the recognition of facial developmental deformity images.3.The BHR-net to identify the accuracy of the landmark:The mark point errors identified by the BHR-net network model constructed in this study are basically consistent with the results of the manual group,and the mark point errors of the inverse-phase test set are also basically consistent with the results of the normal data set.1.In RLV,there were 3 errors in Prn and 1 error in Sn,which were eliminated during statistical analysis.2.The accuracy of submental points in RFV,SMO,LMO and PS is low,and the accuracy of nasal root points in SMO and LMO is low.The accuracy of all other soft tissue markers was high.3.When the error=2mm,Mes,N point in each attitude accounted for the lowest proportion.When the error is 4mm,the accuracy of the nasal root point in LMO is 86%,and the number of other marker points with error less than 4mm is higher than 94%.4.The results of the invert data set show that when the error is 2mm,the accuracy of submental points in RFV,SMO,LMO and PS is low.The accuracy of nasal root points was low in RFV,SMO,LMO and PS.The mental point was 78%in RFV.The upper central incisor points were 74%in PS and 66%in LMO.The number of other marker points with error<2mm accounted for more than 80%.When the error=4mm,only the nasal root points accounted for 88%of the LMO,and the number of other marker points accounted for more than 94%.4.Accuracy of measurement indicators obtained by BHR-net:1.The results of normal data set showed that the width of nose base,the distance of wide mouth opening,the Angle of chin lip,the distance from chin point to midplane line,the distance from subchin point to midplane line,the height of midplane,the width of upper lip red and the distance between lips had significant statistical differences(p<0.05),and the p values of the other 20 measurement indicators were all≥0.05,with no statistical differences.2.The results of the reverse phase test set showed that the distance from the anterior chin point to the midface line,the distance from the chin point to the midface line,the height of the midface,the width of the upper lip red,and the Angle of the chin lip had significant statistical differences(p<0.05).The p values of the remaining 23 measurement indicators were≥0.05,and there was no statistical difference.3.In the lateral occlusion movement,the error value of the left side movement is 0.87,and the error value of the right side movement is 0.89.5.Clinical application research based on BHR-net:1.According to the markers detected by BHR-Net,the diagnostic accuracy rate of doctors for Class II and III Deformities,oral plane is 100%,and Maxillofacial Asymmetric Deformities is 70%.2.Sig detection between BMI and facial Angle was all<0.05,showing a high correlation.There was no significant correlation between AHI index and facial Angle.3.BMI was divided into obese group(>24)and normal group(≤24),and the correlation was only found in nasomental Angle.The facial width ratio was significantly associated with AHI(>10)in the diagnosed group.6.Survey on the evaluation of postural smile by social groups:1.A total of 1469 valid questionnaires were collected in this survey,and the analysis results showed that there was no difference between gender and answer results,while different majors had an impact.2.Spearman correlation coefficient(r)analysis showed that there were significant differences in lip beauty ratings among different majors.The aesthetic scores of the mouth thickness ratio of 1:1.5 and the mouth thickness ratio of 1:2 were higher in stomatology.The relationship between stomatology specialty and laughter line was weak but significant(p<0.05).Different majors have a certain influence on the aesthetic degree of upper lip concave;There was a positive correlation between the aesthetic degree score of dental radian concave upward and the major(r=0.08,p<0.05).3.The results of ANOVA showed that there were no significant differences in the aesthetic scores of lip thickness ratio 1:1.5 among different majors,but there were significant differences in the scores of middle smile line,upper lip lower margin shape and dental curvature among different majors.7.Accuracy of BHR-net identification of postural smile:The one-sample t test was conducted between the BHR-net group and the manual group,and the results showed that 98.77%of the markers with Euclide distance ratio values>0.9 and<1.1 were found.The results of the two groups of measurement indexes showed that the ratio of lip thickness was p=0.01,and the p of other measurement indexes were greater than 0.05,with no statistical difference.Conclusions:In this study,the BHR-net network model based on thermal map regression algorithm was developed by applying the facial development deformity image dataset,which realized the accurate recognition of multiple pose image landmarks in maxillofacial region and the measurement and analysis of soft tissue topography,and the diagnosis and analysis of dental deformity,OSA and smile aesthetics,as well as the evaluation of treatment effects.The rapid facial image measurement platform can effectively improve the speed of clinical diagnosis and reduce the work burden of doctors. |
PDF Full Text Request