Research On Flowchart Image Retrieval Method

Flowcharts have specific advantages in expressing abstract ideas,and some abstract processes and ideas can be clearly expressed using flowcharts.Flowchart image retrieval has an important significance in the fields of document management and information retrieval,which can help users to search target documents or information quickly and improve work efficiency.Although the application scenarios of flowcharts are extensive,there is currently a lack of research on flowchart retrieval.Traditional image retrieval only focuses on visual features,such as edges,contours,etc.,and cannot extract semantic information from a flowchart.Therefore,the traditional image retrieval method cannot realize the retrieval of flowchart images.In this thesis,the flowchart image retrieval is divided into two important parts: flowchart image recognition and flowchart similarity detection.The current research on the similarity detection of flowcharts is either inaccurate or takes too long to complete the retrieval task,which makes it difficult to apply the current flowchart similarity retrieval method in real-world scenarios.However,the current high-accuracy flowchart image recognition method is difficult to recognize the symbols and line segments in the flowchart at the same time,and it is necessary for two different models to cooperate to realize the recognition of symbols and line segments in the flowchart.Therefore,flowchart image recognition methods require a lot of computing resources.Aiming at the above questions,this thesis proposes a flowchart similarity detection algorithm and a flowchart recognition method.The main research contents of this thesis include the following two aspects:(1)Flowchart graph similarity detection method based on graph neural networkIn this thesis,a flowchart similarity detection method named SAGEN(Self-Attention Graph Embedding Network)is proposed to solve the problem that the existing flowchart similarity detection methods are difficult to keep a balance between detection accuracy and detection speed.SAGEN first initializes the nodes and edges of a flowchart graph into feature vectors.After that,SAGEN generates message vectors using two adjacent nodes and their connecting edges,and updates the states of the nodes through message propagation.In order to improve the detection accuracy without affecting the detection speed too much,SAGEN uses the multi-head self-attention mechanism to capture the dependencies between nodes.Finally,SAGEN aggregates all node feature vectors to obtain the final graph vector.After obtaining the graph vector,SAGEN calculates the similarity between different flowcharts by the distance between graph vectors.Experimental results show that SAGEN can reduce the computational amount of flowchart retrieval to a lower level when the accuracy is higher than that of other current similarity detection methods.(2)Flowchart image recognition method based on DETR(DEtection TRansformer)In this thesis,a flowchart image recognition method named MSFF-DETR(Multi-Scale Feature Fusion DEtection TRansformer)is proposed to solve the multi-scale problem incurred by the two different scale targets of symbols and line segments in the flowchart image recognition,and improve the recognition accuracy of the flowchart image recognition.MSFF-DETR first uses CNN to realize the encoding of flowchart images,and outputs two feature maps with different scales.Aiming at the multi-scale problem caused by the large size difference between line segments and symbolic scales,MSFF-DETR performs feature fusion on feature maps with larger scales so that they can contain higher-level semantic information.In addition,MSFF-DETR uses dilated convolution instead of traditional convolution to improve the extraction ability of small-scale features.After that,MSFF-DETR inputs the feature map into two connected transformers,then uses its powerful global information awareness and generalization ability to predict line segments and symbols in the flowchart images.Finally,the flowchart image can be converted into a flowchart graph.Experimental results show that MSFF-DETR is superior to other flowchart image recognition methods in terms of recognition accuracy in end-to-end flowchart recognition.