| Document Visual Question Answering(DocVQA)refers to answering questions based on visual documents,This task requires effective use of language information in images to answer questions,DocVQA has been widely used in real life.However,the existing research mainly focuses on the "in-line" question,which means the answer lies in the same row,column or sentence where the keywords are located.This kind of question has a simple linear relationship,however,there are a lot of more challenging situations to be solved in practical applications.This paper conducts research on the basis of systematic review and analysis of relevant work.The main work is as follows:A new type of DocVQA question,called "in-region" question,is proposed.Unlike the existing "in-line" question,the question and the answer exist in a certain region and do not have a simple linear position relationship in the "in-region" question.To address such a question,we need to locate and understand the text in the region.This paper constructs a new DocVQA dataset for "in-region" questions:RDVQA,which contains 8582 images and 8814 questions.Furthermore,we propose a new model,Reason-over-In-regionQuestion-answering model(ReIQ)which includes two key modules:Spatial-Token Pre-trained Module(STPM)and DocVQA State Tracking Module(DocST).Two new pre-training tasks,MTBR and STR,are designed in the STPM module.In the DocST module,we use the attention mechanism to conduct multiple rounds of reasoning for questions and visual documents.The experimental results of ReIQ model on RDVQA are superior to StructuralLM,indicating its effectiveness in solving RDVQA.Finally,based on the ReIQ model and RDVQA dataset,we develop a DocVQA system.Users can easily upload document images,submit questions,and obtain answers.The system provides a user-friendly graphical interface on the front end,and the ReIQ model processes the user’s questions and documents on the back end,ultimately outputting accurate answers. |
