Construction Of Evaluation Index System For Readability Of Online Health Information

Objective Using the textual and non-textual factors as the theoretical framework to construct a universal online health information readability evaluation index system,test its reliability and validity,and determine the weight of the evaluation index,so as to provide a scientific online health information readability evaluation tool for health education practitioners,and to provide basis for the design,selection and revision of highly readable health education web pages.Methods The research was divided into three phases:Phase One: Building an evaluation index system for the readability of online health information initially.We used the literature research to systematically retrieve the relevant literatures on the readability evaluation of online health information,extracted relevant indexes based on the basic theoretical framework of textual and non-textual factors,and preliminarily drew up the primary and secondary indexes.We adopted the Delphi method and invited 20 experts to score the importance of the indexes.According to the mean value of the index importance scores,the coefficient of variation(CV),and the suggestions proposed by the experts to further screen and modify the indexes,we initially constructed the online health information readability evaluation index system.Excel 2007 was used to input the data.IBM SPSS 24.0 was used to analyze the data,and calculated the effective questionnaire recovery rate,expert authority coefficient(Cr),Kendall’s coefficient of concordance(Kendall’s W)to determine the degree of enthusiasm of the experts,the degree of authority of the experts,and the degree of coordination of the opinions of the experts,so as to test the reliability of the expert consultation results.Phase two: Testing the reliability and validity of the index system.We conducted the preliminary reliability and validity test on the readability evaluation index system of online health information,according to the methods used by foreign scholars.As a result of that,the "Online Health Information Readability Evaluation Tool" was formed.Then the tool was used to evaluate the readability of 30 hypertension health education web pages.Cohen’ s kappa coefficient was calculated to test inter-rater reliability and Cronbach’ s alpha coefficient was calculated to test the internal consistency.Based on the contents of 30 hypertension health education web pages which were randomly selected from the reliability test,the "Knowledge Test Questionnaire" was compiled and used to do reader tests among the residents with high and low health literacy.The accuracy of the test was assessed.We also used the Chinese readability formula to calculate the text difficulty coefficient of the 30 hypertension health education web pages used in the second round reliability test.Correlation analysis was used to calculate the correlation coefficient between the readability level and the correct rate of readers’ answers,as well as the correlation coefficient between the readability level and the text difficulty coefficient to test the construct validity.Phase three: Determining the weight of the index system.The analytic hierarchy process was adopted.The "online health information readability evaluation index system" was the target layer,all the first-level indexes were the criterion layer,and the second-level indexes under each first-level index were the scheme layer to establish a hierarchical structure model.We combined the results of the third round of Delphi expert consultation,and used the difference between the mean value of the experts’ importance of each index to determine the Saaty scale and construct the judgment matrix.Yaahp12.3(standard edition)software was used to calculate the weights and combination weights of each index and conduct the consistency test.Finally online health information readability evaluation index system was established.Results Phase one: We extracted a total of 7 first-level indexes and 55 second-level indexes through literature research.Twenty experts were invited to participate in the Delphi study,and 16 experts completed three rounds of consultation.The mean score of the importance of the first-level index was 4.31～5.00.The coefficient of variation was0～0.11.The mean score of the importance of the second-level index was 3.94～5.00.The coefficient of variation was 0～0.20.The expert authority was 0.872,the expert coordination coefficients were 0.359 and 0.231(P < 0.05).Combined with the experts’ comments and revisions,we constructed an evaluation index system for readability of online health information,which included 7 primary indexes and 33 secondary indexes.Phase two: We referred to the reliability and validity test method of the readability evaluation tool that had been widely used abroad,and the hypertension health education web pages were selected for readability evaluation.The overall Cronbach’ s alpha coefficient was 0.704,and Cronbach’ s alpha coefficient of each dimension was 0.766,0.742,0.744,0.731,0.655,0.650,0.846,respectively.The overall Cohen’ s kappa coefficient was 0.63,and the Cohen’ s kappa coefficient range of each item was0.62～1.00.Through reader test and additional verification of the Chinese readability formula,the correlation coefficients between the readability evaluation results and the correct answer rates of readers with high and low health literacy were 0.511 and 0.424respectively(P < 0.05).The correlation coefficient between the readability evaluation results and text difficulty coefficients was-0.369(P < 0.05).The index system had sound reliability and validity.Phase three: The weight of each index had been determined by analytic hierarchy process.The first-level index weights were ranked from high to low in order of literal expression,content design,actionability,organization and layout,typographic design,visual aids,numerical application,and the weights were 0.2713,0.2220,0.1816,0.1331,0.0819,0.0643 and 0.0458.In the second-level index,the combination weights in the top 4 were clear and specific behavior guidance,use easy-to-understand words,use short sentences,and scientifically accurate information.The combination weights were0.1014,0.0974,0.0682 and 0.0651 respectively.The CR value of the consistency test was 0.0000～0.0304,all < 0.10.The consistency of the index judgment matrix was good,and the weight distribution was reasonable.Conclusion This research constructed the online health information readability evaluation index system,which includes 7 first-level indexes and 33 second-level indexes.The reliability and validity test and weight calculation of the index system has been completed.The construction process of the index system is scientific and reasonable.The reliability and validity is good,and the weight calculation process is accurate and reliable.The study provides references for health education practitioners to design,select and revise high-readable health education web pages,and lays a foundation for further readability improving of online health information.