| In our daily life,electronic devices and mobile terminals with handwriting input are widely used in various fields,where handwriting mathematical expression plays an important role in the field of education.In the field of traditional handwritten text recognition,the symbols tend to be on a horizontal line.Different from traditional handwritten text which is limited to one-dimensional space structure,handwritten mathematical expression has complex two-dimensional space structure.The difficulty of handwritten mathematical expression recognition is that we not only need to recognize the classes of symbols in the expression,but also need to analyze the spatial relationship between symbols.The traditional method of handwriting recognition cannot effectively adapt to the two-dimensional structure of mathematical expression.At present,the research of handwritten mathematical expression recognition is still in the exploratory stage.To solve the problem of on-line handwritten mathematical expression recognition,this paper designed a handwritten mathematical expression recognition system based on Recurrent Neural Network(RNN).This system of this paper used the one-dimensional stroke paths of different order to capture the twodimensional space information of handwritten mathematical expression,and then realized the integrated recognition solution of symbol segmentation,symbol recognition and structure analysis at the same time.The main contributions of this paper are summarized as follows:1.Proposed to build a strokes structure graph according to the spatial relationship and temporal relation between strokes pairs in the expression,in which the strokes structure graph is a directed graph with strokes as nodes and the connections between strokes as edges.The system can perform the integrated recognition solution by recognizing the stroke nodes and edges between strokes in the strokes structure graph.2.The system extracted multiple stroke paths from strokes structure graph.And then resampling and feature extraction were performed on each stroke path to build the input sequence.After the input sequence was processed by Recurrent Neural Network,the system would decode and integrate the output sequence of multiple stroke paths.3.In the network training stage of the classifier,the system used the loss function of local Connectionist Temporal Classification(CTC)to calculate the loss of short stroke sequences in the stroke.This system not only reduces the algorithm complexity of symbol pre-segmentation by integrated recognition solution,but also effectively covered the two-dimensional structure of mathematical expression by using multiple stroke paths.Experiments show that multistroke path has better recognition effect than single stroke path.Compared with other systems,he proposed system can achieve a significant result on symbol segmentation and classification as well as relation recognition. |
