Design And Implementation Of An Online Examination System Based On Distributed Crawlers

With the high pace of development of the Internet,the education industry has achieved tremendous progress in online development and the education on the internet has grown in full swing.In particular,the online development of basic education was an unprecedented success during the epidemic,and in the post-epidemic period,online education is still on the rise.Most closely linked to online education is the development of online examinations.As an essential part of online distance education and competency-based training,the development of online examinations has enabled the efficient use of educational resources and contributed to the achievement of educational equity.The quality of online examinations is determined by the efficiency of the construction of online question banks and the assembly of online papers.Currently,in order to promote educational equity,there are numerous open question bank resources for users to choose from.However,for educators,the quick composition of a paper that meets the desired requirements is the most basic prerequisite for the realisation of an online examination.For the composition of the target question paper relies on a good question bank construction.Therefore this system is based on distributed crawler resource acquisition from different online question bank sources for the construction of question bank data.And based on the question bank construction,an automatic paper composition method based on genetic algorithm is designed.In terms of the specific examination system design on the internet and implementation,on the client side of the system,a Vue.js-based design architecture is applied for the system design and implementation of the online examination and backend information management.On the server side of the system,a server-side control layer and data layer based on Spring boot were designed and implemented.In the design and implementation of the distributed crawler,the flaws in the use of the popular online resource acquisition framework scrapy on the distributed architecture were analysed.In the overall architecture,the data monitoring and unified management of each sub-crawler module by introducing a unified state management middleware,redis,meets the implementation of scrapy on the distributed architecture.In the modification design and implementation of specific sub-modules,improvements to the download middleware are achieved through the design and implementation of a network communication agent pool based on timed task scheduling;improvements to the download queue filtering method are achieved through the design and implementation of a middleware for de-duplication of target request sources based on murmurhash.In the design and implementation of automatic paper grouping,a specific application of genetic algorithms based on genetic algorithms in paper grouping is realised through the modification of the unified paper grouping method by means of decentralised question grouping,allowing the effectiveness of genetic algorithms in specific paper grouping to be concretely realised.The system has been subjected to rigorous functional and performance testing,as well as interface testing of the various service sub-modules.The test results meet the expected objectives of the system.It has also been used concretely in specific practical teaching tests,and is significant for improving the online teaching and the teaching management.