Design And Implementation Of FPGA-based Blood Fog Removal Algorithm For Endoscope

With the development of surgical techniques,more and more operations are now using minimally invasive techniques.In laser lithotripsy minimally invasive surgery,the temperature of human tissue is higher than the temperature of endoscope tip,blood fog appears in CMOS camera,which leads to the problems of low contrast and loss of details of images collected by endoscope,which affecting the smooth operation.Therefore,it is urgent to study a video defogging algorithm applied to electronic endoscope,which can process the collected blood fog video in real time.This paper focuses on the video image defogging algorithm and its implementation on the endoscope blood defogging platform with FPGA as the main control unit.The main research contents are as follows:1.According to the difference of fog concentration in different scenes of blood fog images collected by endoscope,the traditional dark primary color prior defogging algorithm and histogram enhancement theory are deeply studied and analyzed,and a single image defogging algorithm applied to electronic endoscope is proposed.The algorithm uses dark primary color prior(DCP)principle to calculate the weight coefficients of the segmented images and estimate the depth of field information in different areas of foggy images.By combining the CLAHE algorithm with the linear stretch algorithm,which can enhance the foreground image well,an improved DCP-based single image dehazing algorithm is proposed.A simulation platform is built to verify the effectiveness of the algorithm.Experiments show that the single image defogging algorithm proposed in this paper has a good processing effect on blood fog in different depth of field.2.Based on the algorithm of removing blood fog from single image,an improved algorithm of removing blood fog from video is studied.In view of the problems of serious image color distortion,excessive local noise and poor real-time performance caused by single image defogging algorithm and the color distortion caused by histogram enhancement algorithm are suppressed by enhancing the luminance component and outputting the original value of chrominance component in the process of histogram processing of foggy images.In the calculation process of CLAHE histogram mapping table,the problem of excessive local noise in special scenes of foggy images is solved by adaptively adjusting the clipping threshold of block sub-images.The strategy of optimizing pixel depth of foggy image and improving inter-frame processing mode is adopted to solve the problem of poor real-time performance of video defogging algorithm.3.According to the hardware requirements of the video defogging algorithm and the characteristics of the core processor,a video acquisition and processing system with FPGA as the main control unit is built.In this system.firstly,a video deserialization circuit module is designed,which decodes the original MIPI signal of a single channel collected by CMOS camera to obtain the foggy video data in Bayer format and the line field control signal;secondly,a video format conversion circuit module is designed to transfer the foggy video data from Bayer domain to RGB domain;then,an endoscope blood fog removing circuit module is designed;finally,the hardware circuit design of the off-chip memory DDR3 read-write control module and MIPI-DSI video real-time display module are completed.4.The improved video defogging algorithm is implemented on the endoscope blood defogging system,and the reliability of the algorithm is verified by experiments.The hardware logic circuits of sub-modules such as block image weight coefficient calculation module,weighted histogram equalization mapping table calculation module and bilinear interpolation are designed.(Simulate the working scene of endoscope in human body,process the blood fog video collected by CMOS camera in real time.According to the experimental results,the reliability and effectiveness of the system and the consumption of resources on the board were analyzed.The results show that the improved video defogging algorithm has a good processing effect on the foggy video collected by electronic endoscope,and meets the real-time requirement of 60fps.