Research On Deep Learning-based Sequential Slice Image Segmentation And 3D Reconstruction

In recent years,the field of computer vision is rapidly evolving,and the technology of restoring 3D model from 2D image has been warmly concerned by medical science,transportation,biology and other fields.For example,image-based 3D reconstruction is widely used in CT reconstruction,industrial defect detection,3D animation,game production,cytology,histology and other fields.Continuous biopsies of biological tissues are important technical means to study the morphology and structure of biological tissues or cells.By obtaining twodimensional images or constructing three-dimensional models,the internal structure of tissues and cells can be observed.Image segmentation technology is the basis of three-dimensional reconstruction,and the quality of the segmented image directly determines the quality of the reconstructed model.With the outstanding contribution of deep learning in the field of computer vision,especially in medical image segmentation,a large number of excellent models have appeared for medical image segmentation and 3D reconstruction.These models can also achieve good results when applied to biopsies,but there are some shortcomings.In order to solve the problems existing in image segmentation and 3D reconstruction of biological tissue sections,this paper proposes a more efficient image segmentation strategy and 3D reconstruction method,which are summarized as follows:(1).Firstly,aiming at the high requirement of target segmentation details in biological tissue slice segmentation and the complex structure or background factors of target segmentation,a multi-information fusion attentional mechanism image segmentation model was proposed.The model mainly consists of encoder,residual network,attentional mechanism and decoder.The shallow feature is fused with the deep feature by jump connection.The introduction of residual network can build a deeper network structure and solve the problem of network degradation caused by the deepening of network.Deep neural network can extract deeper feature information and enrich semantic expression.The introduction of attention mechanism can make the network pay more attention to the features of the target to be segmented and restrain the interference of irrelevant factors such as background.Finally,the feature map of the segmented object is output by decoder module.(2).In order to improve the discrimination ability of the network model to the target Boundary,a Boundary Enhancement Gaussian Loss Function(BEG Loss)for boundary information was proposed.The loss function has the weight of class balance,which can alleviate the problem of class proportion imbalance in image segmentation.By extracting and processing edge pixel information,this function will focus more attention on target boundary features.(3).In order to explore the internal structure of tissue or cell,a 3D reconstruction model of sequential slice image based on convolutional neural network is proposed.The model mainly consists of feature coding network and deformation network.By extracting the twodimensional features of the sequence slice image and combining the location information of different slices,the model transforms the sequence features into three-dimensional point cloud data through the feature coding network.The hierarchical propagation mechanism in the feature coding network can also train the feature correlation between point clouds.Since point cloud is a kind of unstructured data,it is difficult to represent the outline and shape of the surface of the object,so the representation of grid data can better fit the surface texture of the real object.The main function of the deformation network is to transform the point cloud into a grid,and finally generate the three-dimensional model of the target organization.Finally,in order to explore the internal structure of tissue or cell,a 3D reconstruction model of sequential slice image based on convolutional neural network is proposed.The model mainly consists of feature coding network and deformation network.By extracting the two-dimensional features of the sequence slice image and combining the location information of different slices,the model transforms the sequence features into three-dimensional point cloud data through the feature coding network.The hierarchical propagation mechanism in the feature coding network can also train the feature correlation between point clouds.Since point cloud is a kind of unstructured data,it is difficult to represent the outline and shape of the surface of the object,so the representation of grid data can better fit the surface texture of the real object.The main function of the deformation network is to transform the point cloud into a grid,and finally generate the threedimensional model of the target organization.In conclusion,this paper combined with relevant deep learning knowledge,through the study of target segmentation and three-dimensional reconstruction of biological tissue slice image,and put forward an effective solution to the shortcomings in this field.By introducing deep learning into biopsies,automatic segmentation and 3D reconstruction are carried out,and experiments are carried out on biopsies and medical image data sets to verify the validity and applicability of the model proposed in this paper.