Image Inpainting Based On GAN And Glomeruli Detection For Serial Sections

Objective: Kidney slices of mice are used for three-dimensional reconstruction in histological studies.Compared with traditional tomographic images,3D reconstructed microstructures can provide more spatial information and help us understand the physiological mechanism of the case.The histological imaging method of serial sections provides high resolution images with rich chromosomal information and nucleic acid/protein marker information,which cannot be provided by OPT or small animal MRI.Therefore,the processing of slice images has always been the mainstream procedure in histological research.Method: In this study,serial section images of mice were selected,including N7,N5,and17-day embryonic age E17 datasets.We propose a novel framework Consecutive Context perceive Generative Adversarial Networks(CCPGAN),which is used to repair partially damaged images before detecting glomerular targets.The damaged image and its neighboring images were input to network in parallel to learn semantic information from it which can restore the damaged parts.We showed qualitative and quantitative evaluation results,and we can steadily repair any damage in size and position.Then,based on Faster R-CNN,a multi-scale feature extraction mechanism is added,which can more accurately extract the feature region frame,thereby improving the target detection work of the glomerulus at different developmental stages.Results: This research conducted training and testing on multiple data sets,and compared the performance of multiple repair models.Qualitative and quantitative comparisons show that our method can reconstruct damaged areas,obtain ideal structural content and visual effects,and maintain global harmony.Local consistency and correct object generation.On the N7 data set,the MS-SSIM index reached 0.9923(1.50%),0.8897(21.45%),0.5574(52.54%)on the small,medium and large damage respectively,and the LPIPS index reached 0.0122(15.54%)on the small,medium and large damage respectively.0.1657(19.94%),0.3126(46.43%).Similar results were achieved with the N5 datasets.The improved Faster R-CNN model has an average recall rate of 0.932 for the four phases on the test dataset,an accuracy of 0.942,and m AP of 0.80.Conclusion: In this research,we propose a novel framework Consecutive Context perceive Generative Adversarial Networks(CCPGAN).Introduce a dual-path input deep learning network,which can detect histological structure and learn features from damaged images and adjacent images.For medical image analysis,the accuracy of the repaired content is more important than the consistency with the visual effect.In the glomerular target detection stage,a multi-scale target detection network is added to make more effective use of the characteristics of each level,and the detection results are improved.At the same time,automatic detection improves work efficiency and contributes to the study of kidney development.