Research And Practice Of Standardized Assessment Tool Of Information Literacy For Medical Students

PurposesAs medical science moves towards knowledge medicine and evidence-based medicine, medical students'information literacy and lifelong learning requirements have been put on the agenda and have become the gold standard for the global practitioners, because they are supposed to be the main force of medical research and clinical practice in the future. However, measurement research of information literacy for medical students is week both at home and abroad, which have greatly influenced the effectiveness and efficiency of information literacy education for medical students. This research is to use qualitative and quantitative methods to develop a standards-based information literacy assessment tool for medical students with high reliability, validity and responsiveness for objectively, accurately, comprehensively and reliably evaluating medical students'information literacy level, and thereby provides basis for strengthening scientific, standardized and normalized information literacy education for medical students.MethodsIn this study, literature information analysis, including scientific knowledge visualization, classification analysis, and co-occurrence analysis were used to develop the information literacy objectives system for medical students, and on the basis of that, we construted the initial test items. Personal interviews and Focus Group were used to furtherly develop test items, and comprehensive index method was used to calculate the weight and standardized scores of each test item. Internal consistency reliability coefficient, expert judgement, factor analysis, ANOVA analysis and effect size were adopted to test the internal consistency reliability, content validity, constrcuct validity, discrimination validity and responsiveness of the assessment system.Results(1) The information literacy assessment theoretical framework for medical students has four measurement dimensions, including Information Awareness, Information Knowled -ge, Information Competencies, and Information Ethics and Morality. The dimension of Information Awareness comprises domain awareness, advanced awareness, trail awareness and evidence-based awareness. The understanding towards declarative knowledge about evidence-based practice related terms, medical information related terms, medical informa -tion retrieval, and reference literature indexing norms are under the dimension of Informa -tion Knowledge. Information Competencies mainly include abilities of expressing informa -tion needs, information acquisition, information evaluation, information organization and management, information innovation and use, as well as evidence-based practice. Informa -tion Ethics and Morality is comprised by the awareness of rational use of information and the act of information fair use.(2) The content validity of each test items, including content accuracy, clarity of item as well as the extent of institutional adoption, was evaluated by experts consultation. Experts'positive coefficient was 0.875, and authority was of a high level. The results of experts'evaluation showed relevant convergence (Kendall coefficients of W = 0.126, p = 0.000 <0.05). After content validity analysis, the entire test system was cut in numbers, and some items were organized and expressed in a more scientific manner.(3) The weight of each item was determined by comprehensive index method. Item analysis was conducted using the measurement data, then five items with a lower discrimination and a highest or lowest difficulty were deleted.(4) Factor analysis was made to test the construct validity of the whole system. Results showed that totally four common factors were extracted, which are adequate compared with the conceptual and operational definition of medical students'information literacy measure -ment and consistent with four dimensions identified in the measurement theoretical frame -work. The construct validity of the measurement system is good.(5) The ANOVA analysis showed that the differences of information literacy level amoung five-year program medical students and eight-year program medical students and students majored in health management are statistical significant(p=.000<0.05, p=.018 <0.05), indicating that the differentiation of the measurement system is good.(6) The value of Cronbach'sαfor the whole measurement system is 0.706, Spearman -Brown half-spit coefficient 0.780, and Guttman half-spit coefficient 0.776. The internal consistency reliability is high.(7) Effect size statistic (ES) analysis showed that before and after the program of "Medical Literature Retrieval and Use" intervention, the level of each dimensional and overall information literacy level of medical students were significantly increased, overall ES value is 2.29, the responsiveness was high. The final form of the test items can be used for measuring the level of information literacy for medical students.Conclusions(1) "Standardized" is mainly reflected in two aspects: First, the development of test items is based on the authoritative information literacy assessment standards of higher education at home and abroad and information literacy-related target in the international standards of medical education, that is, the "basis" of the assessment tool is "standardized". Second, "standardized" is embodied in the scientific research and design as well as the systematic validation path, that is, the "process" is also "standardized".(2) The establishment of information literacy test items for medical students should focus on the principles of objectivity and applicability.(3) The relative importance of test items is determined more scientifically and reasonably as we combined the subjective weight with objective weight of each item.(4) Quantitative evaluation of qualitative research, such as experts consultation, making the research more scientific and rigorous.(5) This study established the first domestic standardized assessment tool of medical information literacy, and has taken a substantial step to enhance scientific, standardized and nomalized assessment research of information literacy. Meanwhile, the comprehensive use of various research methods ensure that the research process and findings are scientific and rational.(6)The shortcomings of this study and future research direction are 1) comprehensive and specific items need to be further strengthened, 2) the subject background of experts needs to be further expanded, 3) the coverage of actual measurement sample size needs to be further increased and so on.