| Text is the main carrier of information transmission.In natural scenes,text often carries very critical information which plays a crucial role in the understanding of scenes.Therefore,scene text recognition has been a very meaningful and concerned research topic for decades.Nonetheless,in real-world scenarios,scene text often suffers from various types of distortion,such as rotation,perspective distortion and distribution curvature,which increase the challenge of scene text recognition.In this thesis,we proposed several methods to solve the problem of irregular scene text recognition,which can effectively improve the accuracy of scene text recognition.The main work and innovation of this thesis include:1.After systematically analyzing the existing problems in the task of irregular scene text recognition and the advantages and disadvantages of relevant solutions,we determined the main framework of our system,which is based on weakly supervised text rectification.This method can improve the accuracy of scene text recognition without additional annotations.2.Aiming at the problem of complex and diverse text distortion in natural scenes without regularity,a Sequential Transformation Attention-based Network(STAN),which comprises a sequential transformation network and an attention-based recognition network,was proposed for general scene text recognition.We applied the idea of decomposition to the design of the sequential transformation network to greatly reduce the difficulty of text rectification,thus effectively improving the rectification effect of the irregular text.Furthermore,we designed a grid projection submodule to ensure that the entire process of decomposition,transformation and merging is smooth.Then,the text recognition network takes the rectified image as input and predicts a character sequence based on the attention mechanism.Finally,benefiting from the weakly supervised training manner,the entire network is able to be trained in an end-to-end manner,requiring only text line annotations.Moreover,after using the sequential transformation network,the recognition accuracy was improved by more than 5% on some irregular datasets.3.Considering that the text rectification network which is based on geometric transformation constraints will be limited by the processing capabilities of geometric transformation and cannot flexibly deal with the deformed and irregular scene text,this thesis proposed a Global and Local Cascaded Rectification Attention-based Network(GLCRAN)for scene text recognition,which comprises a global and local cascaded rectification network and an attention-based recognition network.In the design of the global and local cascaded rectification networks,we achieved a highly flexible and easily optimized text rectification network by cascading a global rectifier and a local rectifier.Additionally,in order to solve the problems of information loss caused by cascaded rectification and low image resolution caused by multiple interpolation sampling,a new grid generation and sampling strategy was designed in this paper to further improve the rectification system.The state-of-the-art results on several datasets demonstrate the effectiveness of this method. |
