Research On Question Answering System In Ship Domain Based On Reading Comprehension And Image Description Generation

With the rapid development of knowledge and continuous innovation of technology in the marine domain,there is an increasing need to obtain accurate information and solve practical problems.Traditional FAQ Q&A and knowledge graph(KG)-based Q&A systems perform well in specific scenarios,but they have limitations in dealing with certain problems in the marine domain.FAQ Q&A systems mainly deal with predefined problems and are weak in dealing with novel or long-tail problems;while KG-based Q&A systems are limited by the coverage and accuracy of the knowledge graph and may not be able to deal with problems involving domain The KG-based Q&A system is limited by the coverage and accuracy of the knowledge graph,and may not be able to cope with questions involving extra-domain knowledge or real-time updates.In addition,in the field of shipping,many questions not only involve textual information but also need to consider information related to images.Image description generation models can extract key information from images and translate them into natural language descriptions.By incorporating a reading comprehension question and answer system,we can make full use of image description information to provide more comprehensive,accurate and insightful answers to questions involving images.Therefore,in order to better handle questions involving image information in the marine domain,we incorporate image description generation into the Q&A system.A Q&A system based on reading comprehension and image description generation for the marine domain has better flexibility to handle open domain questions,fuse multiple sources of information,cope with complex problems,and provide interpretability.This thesis details the research of a ship domain Q&A system based on reading comprehension and image description generation to achieve accurate answers to ship-related questions by integrating image descriptions,domain knowledge and natural language processing techniques.The main research contents of this thesis are as follows:(1)In this thesis,an image description generation method based on target detection and knowledge enhancement is proposed.First,in the target detection stage,the thesis proposes a Fusion target classification detector(FTCD)that fuses multidimensional information,through which the labels of targets such as faces,goods and objects in the graph are obtained.Secondly,the knowledge graph is introduced,and the target labels obtained by the target classification detector are used to query the related knowledge in the knowledge graph.Finally,the set of labels of targets and related knowledge are fed into the model together for encoding;an attention mechanism is introduced at the decoding end of the model for guiding the model to generate image descriptions after selecting the appropriate information.The experimental results show that the proposed method has 1% accuracy improvement compared with the benchmark LBPF model.The experimental analysis shows that the performance improvement of this thesis’ s method compared with the benchmark model mainly comes from the introduction of knowledge graph and the target classification detector proposed in this thesis.(2)This thesis proposes a reading comprehension question-and-answer model based on image description and news text,based on the study of image description and reading comprehension question-and-answer model.The model is able to combine visual and textual information to generate more accurate and comprehensive answers.First,the images in the news are used to generate the corresponding description text through a knowledge-based augmented image description model,and then the description text is stitched with the context of the news in which the images are located,and the text is converted into a vector representation using a pre-training model and input to the reading comprehension quiz model.During the training process,a contextual understanding and inference module is introduced,which helps the model capture the semantic relationships between texts and discover potential knowledge,thus improving the accuracy and reliability of the Q&A system.Then the model is trained and tuned using data from the domain knowledge base so that it can be adapted to the problems in the ship domain.Finally,the model performance needs to be optimized by cross-validation and tuning hyperparameters during the model training process.With this reading comprehension Q&A model that fuses image and text information,we can better understand and answer questions involving visual information,thus improving the performance of the Q&A system.(3)In this thesis,we propose and build a reading comprehension quiz system for the ship domain.The system first constructs a ship domain dataset containing image descriptions,news articles,standard knowledge of ships and other domain texts through data collection and pre-processing.Next,answer prediction is performed using the reading comprehension quiz model based on image descriptions and news texts proposed in this paper.After the model is constructed,it can be integrated into the user interface of the quiz system so that users can enter questions and obtain answers.In addition,the system is equipped with the function of relevant chapter retrieval so that the chapter related to the question can be quickly retrieved from the domain knowledge base based on the question entered by the user when answering the question.The system provides a complete set of online training,model management and version iteration functions for algorithm models.Through this system,users can upload training datasets,select suitable algorithm models,set training parameters,and start online training tasks through the system interface.The system will automatically carry out the training process and provide real-time training progress and performance indicators to facilitate users to understand the training status of the model.