Research On Interpretable Grading Of Specialty Text Based On Knowledge Points

With the wide adoption of internet technology,many national specialty qualification examinations are changed to the manner of online test.Automatic grading of specialty subjective answers can help optimize the review process,improve the efficiency,and avoid the deviation of review quality caused by personal factors,which is of great significance for maintaining the authority and fairness of the national examinationsThere are many challenges on intelligent grading of specialty texts.From the perspective of text comprehension,most of existing methods adopt word embeddings and neural networks to solve the semantic ambiguity problem of text.However,in specialty text,there are different forms of content such as specialty concepts,numerical formulas and etc.It is difficult to have the pre-trained embeddings of numbers and operators in formula in the same latent space since their meanings are tightly combined with each context.This makes it difficult to infer the specialty text taking into account both semantic and logical aspects.From the perspective of model,it is necessary to provide evidences with the grading results.The interpretability requirement on the grading model is actually one of the common challenges of deep neural learning methods.From the perspective of data,there are only a small quantity of labeled samples available for model training since in the application scenario these labeled data are provided by experts.The lack of training data makes the deep learning process more difficulty.To solve the above challenges,in this thesis we study the interpretable grading problem of specialty text.The main contributions are as follows:(1)For discussion-based subjective texts,we propose an interpretable grading model with mutual attention mechanism.A bidirectional recurrent neural network and maximum pooling are used to model the semantic representations of sentence and paragraph of text.For the given knowledge points as references,each student answer is compared and the matching degree is inferred by the mutual attention mechanism,which are regarded as sentence level evidences for grading result.The matching degree is also taken into account with the overall score as the loss function for justification.This joint learning can benefit from the high-level interactions between these two closely related tasks.In the predication phase,the matching points with knowledge are used as the proof for scoring a specialty text.(2)For the calculation-based subjective texts,we propose an interpretable grading model based on the concept-value matching mechanism.A sequence labeling model is used to match the subjective text to the concepts in knowledge points so that the calculation and values can be compared with the exact concepts.We adopt convolutional neural networks to model the text and formulas and infer the matching degree of student answers with the reference knowledge text.Finally,the matching result and the score are combined for joint model learning so as to help verify the correctness.The grading and matching results are together provided for the interpretable grading of calculation-based texts.(3)We verify the proposed methods on real datasets against different metrics.The data are selected from a national specialty examination.To verify the robustness of model,we compare the performance against different sizes of training sample and the results show that the models work well even on the small quantity of labeled data.To justify the contribution of each model part,we select several variants of our model to compare the performances on different attention mechanisms,pooling strategies,and network structures.In order to verify how much latent knowledge can be applied to different questions,several transfer learning strategies are analyzed.With the experimental results,the adaptability of the model is discussed for different situations.