Research And Implementation Of Automatic Evaluation Methods For Machine Generated Press

In recent years,the integration of artificial intelligence technology and traditional industries has resulted in numerous new applications.Machine-generated press is a practical product of AI technology in the field of news creation,and its writing efficiency is significantly better than the traditional press releases production mode.However,the current quality of machine-generated press leaves much room for improvement.Efficiently evaluating the quality of press releases is an important part of improving the quality of machine-generated press.Manual evaluation methods are commonly used to assess the quality of press releases,but they are inefficient and costly,and cannot cope with the evaluation of a large volume of news.Currently,automatic evaluation methods used in machine-generated press rely on generic evaluation metrics for machine-generated texts,which are not targeted and systematic.In the thesis,we address the challenges of heavy manual dependence and difficulty of quantitative evaluation in the assessment of machine-generated press releases.We conduct research on two aspects:subjective and objective consistent evaluation,and interpretable evaluation of machine-generated press quality.The main work is summarized as follows.(1)Research on subjective and objective consistent evaluation of machine generated press quality.First,a hierarchical and multi-granularity evaluation system is designed,which starts from both text and image modalities of machine-generated press quality,covering both fine-grained local features and coarse-grained global features.Second,the metrics in the evaluation system are quantified in two dimensions:subjective and objective.The subjective evaluation method adopts expert scoring quantification,and the objective evaluation method adopts a rule-based quantitative modeling method of evaluation metrics.The validity of the objective evaluation metrics quantification method is verified through the correlation analysis of subjective and objective evaluation results.Finally,based on the results of the above two steps,a subjective-objective mapping model with tree structure constraints is constructed.The model treats the evaluation system as a priori tree structure,takes the objective ratings corresponding to press releases as input,and encodes and outputs the predicted ratings of the subjective-objective mapping of press releases at different levels along the tree structure by distinguishing the priori constraints at different levels in the tree structure.The model is trained by a two-stage optimization strategy combining supervised and weak supervision,which makes the prediction scores of the subject-object mappings reasonable.Experiments show that the binary scores of press release quality predicted by the proposed subject-objective scoring consistency model are highly consistent with the scores of professionals.(2)Research on machine generated press quality interpretable evaluation.The thesis proposes a machine generated press quality interpretable evaluation method based on hierarchical graph neural network.First,the method uses a pre-trained model to encode press release content information and press release evaluation information.Secondly,the hierarchical graph neural network is constructed with expert experience as a priori knowledge,and a multilevel attention mechanism including type-level and node-level is introduced to aggregate node information and realize the update of press release content coding and corresponding evaluation coding.Finally,the updated press release codes are used to predict the rewriting opinion tags from the predefined ten types of press release quality evaluation tags.The experiments show that the interpretable evaluation model can accurately predict press release evaluation tags.