Medical Record Text Detection And Recognition Based On Deep Learning

In recent years,the medical record management of many hospitals is developing towards the direction of electronic medical record management system.However,a large number of hospitals still store a large number of old medical record files,which are mostly stored in the form of printed documents,handwritten forms and scanned pages for recording patient information and scientific research.Therefore,it is of great significance to study the automated medical record processing method in the medical field.Automated analysis and processing of medical records is challenging because of the complexity of the pages of medically scanned documents,including both handwritten and machineprinted texts,as well as the lack of contrast and illegibility.In this paper,a universal medical record text recognition model is proposed,which can recognize both handwritten and machine-printed texts in scanned files,eliminating the need to classify texts.The main work of this paper is as follows:1.Medical History Database(MHDB)was constructed with a total of 11,815 Medical History images.Medical record images included 2,541 surgical records,2,458 disease course records,3,462 pathological diagnosis reports,and 4,354 laboratory reports,totaling 11,815 medical record images obtained from 11 first-class hospitals,including Peking union medical college hospital,general hospital of the people’s liberation army of China,Qingdao municipal hospital,and Peking University people’s hospital.Ground truth was obtained by marking the positions of handwritten and printed characters respectively.Paper textures under 20 different lighting conditions were selected as the background from medical record images.We collect 260 kinds of handwritten characters of Chinese characters with 14 common types and different writing styles.By using image processing technology,printed characters and handwritten characters are pasted on the paper texture background respectively,and perspective,distortion,blur and other transformations are added to make a text recognition data set containing 3000000 images of the text line.2.A Text Line Detection and Classification model was proposed.TLDC(Text Line Detection and Classification)model in the text detection at the same time,can be handwritten and printed text classification,to avoid the cumbersome first detection of coordinates after the classification of types.At the same time,the height correction of text line bounding box is added to avoid the error of text line bounding box including other text lines.In view of the problems of insufficient contrast and blurred handwriting in the medical record images,the contrast enhancement of the medical record images was carried out in the pre-processing stage.Experimental comparison was made between TLDC and several other text lines in the three dimensions of accuracy,recall rate and fmeasure.The final experimental results confirmed that TLDC had a better effect on MHDB data set.3.The data is proposed based on a Mixup METR enhanced Text Recognition model(Mixup Enhances the Text Recognition).The model used Densenet as feature extractor and included BLSTM(Bidirectional Long short-term Memory)into the text recognition model.CTC(Connectionist Temporal Classification)was used to automatically align the images of labels and text rows,realizing the serialization recognition of text rows,and obtaining the final text sequence by mapping the predicted sequence output from the model to the final tag sequence.This design bypasses the character-level image segmentation for text line images,thus avoiding the overall accuracy decline caused by backward accumulation of errors in character segmentation.Moreover,by adding Manifold Mixup strategy at different locations,the problem of insufficient training data in handwritten text recognition was alleviated and the final identification accuracy was improved.