| Due to the rapid development of information technology,the medium for carrying text information has become more complex.Text is not only arranged regularly in document materials,but also appears in natural scenes.Therefore,scene text detection and recognition have always been of great concern in the field of computer vision.Starting from deep learning based scene text detection and recognition methods,this article conducts research on three aspects: text detection,text recognition,and end-to-end text detection and recognition.The specific work content is as follows:Firstly,a scene text detection model based on Swin transformer is proposed for the problem of arbitrary shape text detection.In the feature extraction phase,Swin transformer is used as the backbone network to model the relationship between remote text pixels.At the same time,a feature fusion module was designed to adaptively fuse features of different scales using spatial attention mechanism.Gradually correct the text boundaries through iteration,making the text box closer to the shape of the text,in order to improve the detection performance of arbitrarily shaped text.Secondly,a scene text recognition model based on pseudo labels was designed to address the issue of how to use real scene text data for text recognition.In the feature extraction phase,the global context attention is added to the backbone network to model the context information,and in the sequence modeling phase,a semantic model based on structured convolution is proposed to obtain the Semantic information of the text sequence.The network is trained using a pseudo label based approach,using only a small number of annotated and unlabeled real scene text images.Compared with the results of supervised training on a large number of synthesized datasets,this method saves a lot of training time while ensuring accuracy.Finally,to address the issue of separate training for detection and recognition networks,an end-to-end scene text detection and recognition model is designed.In the detection branch,a lightweight feature extraction network is adopted,deformation convolution is added to adapt to various shapes of text,and fusion modules are added to improve detection performance.In the recognition branch,the Masked Align operation is used to apply the detection results to the recognition part,allowing the entire model to be trained end-to-end.The comparison with existing end-to-end network models confirms the effectiveness of the proposed method. |
