Multispectral Image Recognition Of Placental Villi Based On Spectral Global Information

Placenta is an important organ for material exchange between the fetus and the mother.During human pregnancy,placenta is a binding organ between the mother and the fetus grown from the embryo membrane and the mother’s endometrium.Placenta structure is complex,and nap is the basic function of the placenta.Clinically,it is necessary to make histopathological analysis of the placental villous layer in order to judge whether the placental function is abnormal,However,the placental villus tissue is complex and changeable,which brings challenges to the diagnosis of placental function by pathologists.Moreover,the diagnosis process is time-consuming and tedious,and it is impossible to quantitatively analyze various indicators,so only experienced pathologists can give more accurate pathological diagnosis.Due to the use of microscope or mainstream pathological image scanner,only RGB pathological imaging can be observed,and spectral images that can reflect the characteristics of different tissues cannot be obtained.In recent years,with the development of image processing technology mainly based on deep learning and multispectral imaging technology,new ideas are provided for the study of identification and segmentation of microscopic pathology.In addition,compared with RGB color images,multispectral images can not only provide spatial information of villi pathology,but also obtain spectral information reflecting the chemical composition of pathological sections.Therefore,an accurately labeled multispectral image data set was established for the pathological tissues of placental villi,and the segmentation method of multispectral images was studied based on convolutional network image processing technology.In this paper,we established a microscopic multispectral data set of placental villi with three pixel-level labels of terminal villi,blood vessels and thrombus,and used this data set to study the segmentation method of placental villi tissue region.Meanwhile,Spec Tr(Spectral Transformer)model was designed for multispectral images of placental villi to identify the terminal villi,blood vessels and thrombotic regions of placental villi.On the basis of convolution neural network,combined with Transformer model in natural language processing technology,spatial features and spectral features are modeled simultaneously,and the correlation of information between different spectra in the Transformer model is analyzed visually.In order to solve the problem of different information distribution between spectra and insufficient sparsity of attentional matrix obtained by Transformer module,spectral normalization and attentional sparsity methods are proposed to improve the original Transformer attentional model and improve the calculation efficiency of the model while taking into account segmentation accuracy.In addition,in order to verify the effect of spectral information modeled by Transformer model on image segmentation,We also compare our proposed architecture with various state-of-the-art 2D and 3D convolutional neural networks to further verify the effectiveness on multispectral placental villi image segmentation.The experimental results show that the multispectral data set of placental villi established can provide an effective reference data set for the research on the segmentation of pathological regions of placental villi.In addition,in the study of Spec Tr segmentation method in this paper,compared with various mainstream 2D and3 D convolutional neural networks,Spec Tr model has the best segmentation effect on placental villi multispectral pathological images.The accuracy of DICE was 0.8534,0.6514 and 0.6111,respectively.The above results indicate that the 3D convolution module and Transformer module are helpful to improve the regional segmentation ability of the model for multispectral images of placental villi.Finally,according to the Attention Map visualization of Transformer module,the importance of different spectral bands can provide reference for the selection of spectral bands.Meanwhile,medical indicators such as the proportion of terminal villi in pathological sections of placental villi can be calculated according to the semantic segmentation results of villi,so as to assist doctors in the more efficient diagnosis of placental diseases.