Adaptive Cognitive Diagnosis Testing Of Vocabulary Knowledge

Vocabulary Knowledge(VK),as one of the essential knowledge required for the development of English language talents in the new era,is crucial for college students’ English learning.However,how to measure vocabulary knowledge has been one of the difficulties in language assessment research,and it is also an area of great interest.The current vocabulary knowledge tests confronted at least three problems:(1)the theories used for test analysis are mostly classical testing theories and item response theories,which cannot obtain multidimensional and personalized diagnostic information;(2)the existing tests are administered in a single way,mostly paper-and-pencil tests and computer-based tests,which still need to be improved in terms of test efficiency.To address the above issues,this study aims to develop a vocabulary knowledge test based on cognitive diagnostic theory and Computer Adaptive Testing(CAT)technology that adequately covers multiple attributes of vocabulary knowledge and provides multidimensional diagnostic information.Specifically,this study uses the Word Part Levels Test(WPLT)as the test base and further introduces the Cognitive Diagnostic Computer Adaptive Testing(CDCAT)to develop a more comprehensive,rapid,and valid vocabulary knowledge test.To achieve this goal,this study(1)designed a vocabulary knowledge test based on the WPLT to be administered to Chinese college students,obtained the Chinese sample parameters of the items and then initially developed a CAT program to conduct a simulation study to compare the efficiency and accuracy performance of the CAT under different termination rules to test the validity and applicability of the WPLT item.(2)developed a CDCAT program for a simulation study to compare the performance of different item selection algorithms(SHE,PWKL,PWCDI,PWACDI,GDI)and different fixed-length rules(6,9,12,15,30)or(10,15,20,25,30)on estimation accuracy and exposure rate for three attributes and five attributes of vocabulary knowledge to test the cognitive diagnostic classification effectiveness and test efficiency.The final results of the study showed that(1)the difficulty parameters,discrimination index and test reliability of the WPLT item bank based on the Chinese sample met the item bank construction criteria of the CAT.The CAT based on this item bank was able to achieve the same test accuracy as this same type of paper-and-pencil test with a smaller number of items,proving the validity and applicability of the WPLT item bank among Chinese college students.(2)The CDCAT developed based on the WPLT improved cognitive diagnostic effects and test efficiency.Specifically,only 9 and 20 items were needed to meet the diagnostic effectiveness and test efficiency criteria of the CDCAT for three and five attributes.In addition,based on the relatively better performance of estimation accuracy and exposure rate,selecting the 15-item and PWACDI algorithm or the 25-item and PWKL algorithm as the selection algorithm and termination rule in CDCAT applications can be a more reliable strategy.Compared with previous studies,this study has the following advantages:(1)Cognitive Diagnostic Assessment(CDA)can provide detailed and specific attribute mastery information,and vocabulary knowledge can be further divided into five attributes based on three attributes by prefixes and suffixes,thus providing more dimensional and fine-grained mastery information.(2)CAT has been widely used in the field of language assessment and has the advantage of improving test efficiency and accuracy,which is important for vocabulary knowledge tests that can cover a wide range of attributes.In summary,the approach of combining WPLT,CDA,and CAT provides valuable exploration for teachers to provide guided practice to improve students’ vocabulary knowledge,thus providing more accurate and efficient instructional support for students.