Research On Key Technologies For E-Learning With High Concurrent And Bigdata

Since the 21st century, with the rapid development of information technology, the popularity of education informationization and the popularization of higher education, the Web-based distance education is booming, and the pattern of interactive learning in modern distance and continuing education has formed. From 1999 to 2014, there are 17362 learning resources repositories builded by the 69 modern distance education experimental colleges which are used by more than 19320 thousand students on Online learning in their spare time. Online learning has forced an implementation of MOOC, Micro-Course Online Video and flipped classroom, Mobile learning emerges at the historic moment.However the present online learning system generally can’t keep up with the speed of all kinds of resources construction and education scale development. The main reason is that the concurrency and data processing ability of existing system can’t satisfy the needs of the requirements. To improve the quality of teaching, colleges roll out different management and supervision measures, such as recording learning time to inprove enthusiasm for learning and designing mobile learning solutions to meet the requirements of learning anytime and anywhere. In the modern distance education experimental colleges there are a large number of students from 30 thousand to 100 thousand, even 2000 thousand students in Open College. Most of the students have their own work, they have to learn in their holiday, evening or the time before exam. Then it will cause series of problems. So many students learn in the same time on the system, it will challenge the concurrent ability of the system. Every system has crashed again and again in the high concurrent access. System crashes discourage students’learning enthusiasm seriously.Many online learning systems were built essentially based on traditional relational database and the Session mechanism. There are congenital deficiencies in the system architecture:One is that each user should establish a Session in the memory. The concurrency of a single node can’t be improved with limited memory. The other is that the maximum number of connections in relational database is limited by the server performance and network speed obviously. After ten years of development of distance education, the formatted data records in all colleges is in million and ten millions and will keep rising to a hundred of million and thousands of million, or even to the magnitude of TB. Once the main table data records rise to hundreds of millions, then the data records related to the main table will rise to dozens of times, hundreds of times, even to PB level. In such environment of high concurrency, large-scale data requirements, how to construct the Web information system with stable performance in the multilayer model on the existing online learning system, is an unavoidable prominent problem in the system.There are six traditional methods to solve the high concurrency problems:(1) using high-performance server or cluster; (2) the efficient programming language (SQL optimization, the database buffer pool, etc); (3) as far as possible to use the static methods (such as information release system CMS generate the dynamically released contents into Static page; (4) separation of content (image server and other file server stored separately, implement simple triage strategy); (5) the WEB caching technology (client, server and proxy server caching mechanism); 6)database hash and table te-nology (an information table, separate data of the different function module into different databases or tables by program, analysing data characteristics to differentiate fine-grained database table). Even we use these methods, it is hard to satisfy the demand of the large data mode of high concurrent access. Because these traditional methods do not consider the memory problem of Session using in a single Web server node of the cluster. On the contrary, the traditional methods needs to use their own unique mechanism to maintain the Session consistency in their various clusters.This Dissertation explores how to improve the various perfonnance of online learning system by adjusting the implementation strategy of the elements of the internal system and interlayer element deployment scheduling strategy and adding individual new elements.The Dissertation describes the stable Web system construction technology under large amount of data in four aspects:the Web system architecture and database technology, the refactoring of modern access control UCON mode, the model design and application of 0-Session, the building of the pressure prototype system by using a large amount of data.The services of this stage of the HTTP protocol are stateless, but the application requirements of a lot of Web service is "stateful". So people construct a Session in the third layer of the WEB business application container combined with the cookie of the first layer or URL rewriting technology, Then we can realize the stateful service in Web information system. The typical application is the shopping basket service on shopping site. However, with the sharp increasing in the number of users, using Session-cookie or rewriting Session/url face to the limited memory problems, each user needs to maintain a Session and insure not release the memory in a short time. The more concurrent users, the more memory storage will be consumed. Over a period of time the number of concurrent reaches a certain amount, memory storage is occupied, then it will cause the system congestion and then users can not access the system. So in systems with high concurrency, it is particularly important to reduce memory consumption. In this Dissertation, based on a series of messages (Json) and pass strategy of messaging model 0-Session model is put forward, and the need to exchange or need to give messages to subsequent pages, on the basis of pass strategy, deposited the communication card information in a relational database or based on the keys to effective NoSQL database, when need directly with passage number from a relational database or no database access communication card message, memory consumption is greatly reduced. After the author use 0-Session technology in the online learning system, the concurrent ability of on-line examination system significantly increase. In an online test, the test time is unified, researchers focused, a test will last two hours. If all Sessions in the 2 hours is in maintain state, the memory consumption is too huge. In addition, the network environment is complicated, the mode of the Session with the server must ensure that the examinee exam local machine network in two hours won’t have any "mistake", otherwise, it will cause the loss of the Session then the students could not submit the test paper although they have finished their exams. The implementation of O-Session scheme, the examination time, Session and other factors of memory consumption naturally disappear, so as to improve the concurrency server performance; in exam it is only demanded to keep online when the student download the paper and submit the paper. Then it will not have a influence on the exam when the Web restart and the exam is offline in the middle of the exam.O-Session plan is essentially slowing down the response speed to improving concurrency performance, trading time for space. The Session mechanism is based on the server’s memory, the read-write speed is better than the O-Session message passing scheme. O-Session message passing mode will store the message with pass in a relational database or the NoSQL library through the network, it has a certain response time and will cause network overhead. But it’s worth costing the time to maintain the Session program in the high concurrency systems.After all the response ability of a single Web server is limited, it is very important to select the Web server cluster to alleviate the high concurrent access system in the second layer. The multi computer system are connected as a loosely coupled system by cluster, the machine between processes can communicate with each other. Load balancing will decompose the key task into several sub tasks, and then distribute them to the cooperative nodes according to certain strategy, completes the work task. Through the load balancing to extend the server bandwidth, increase the throughput of the whole system, is a cheap and effective method. Network data processing ability is multiplied, the flexibility and availability is improved significantly.There are three main categories in load balancing technology:Based on DNS load balancing, Reverse Proxy (Reverse Proxy) of load balancing (represented by JK2, NGINX) and technology based on Network Address Translation (NAT:Network Address Translation) of load balancing. Load balancing based on DNS is to use the same server domain name corresponding to multiple servers, each server using a different IP address, no distinction between primary and secondary servers, it is exactly equal to the individual, any server can handle the client request. Load balancing based on reverse proxy concentrates on receiving all the corresponding HTTP from the network’s request by a special computer (commonly known as request dispatcher), and then assigns to each server internal network according to certain rules. Based on the network load balancing address conversion technology is implemented by modifying the network layer of the TCP/IP protocol stack, it receives from the Internet legitimate address of the client request.Through the network address translation technology, put the request on the basis of specific load balancing strategy forward into the internal network address.The request allocation algorithm of the load balancing at present includes the wheel method, the least connection method and the fastest connection method, etc. In the implementation of all these load balancing, the Session management is a big problem. The O-Session technology can apply to each kind of solution. This Dissertation introduced NGINX to implement the load balancing based on the reverse proxy in the second layer.Data-Storage method has been developed from punched cards to the tape drive and to the hard media storage technology of CD-ROM and hard disk since 1950. The database changes dramatically every 10 years, including network,hierarchical, relational and object database. Developments of database technology promote IS (Information System) development rapidly, especially the Web IS. When the data accumulated to some degree in Web IS, the concept such as massive data and VLDB (very large data) have been appeared in succession and the data accumulation leap from quantitative change to qualitative change process. Then the advanced database technology is expected. When people’s attention from transaction processing technology of ACID to CAP with Professor Eric Brewer, Big Data time has finally coming out.It’s the traditional solution of pure relational database storage that can’t resolve restrictions of connection number of Database Connection Pool in the high concurrent systems. According to different Big Data application, people will choose NoSQL database alone or NoSQL database joint with relation database to storage in order to enhance the capability of data-access., First the location in the different kinds of data storage is sorted out based on Combined storage. According to the test results of high concurrency, the four basic principles that the data storage of big data high concurrency system combined storage strategy needs to follow are given at this dissertation:1.Using table partition patterns of system core business data from large database table store in online operation of relations database (referred to as production base).2. All business data with big table of the core business store in the NoSQL database.3.The based data of the system’s code base, data dictionary, user permissions system in the production data, also should save a copy in the NoSQL repository.4.large data has high demands on statistical analysis. Then statistical data should be created, batch program that is designed exports the statistical data that is interested in from NoSQL database to statistical library for the use of statistical analysis.In the Web multilayer system structure, the model of user role permission belongs to the fifth layer of security authentication level. The traditional system of user role permission has to assign roles to each user.The method become unfit for the current system management with millions of users. In addition, the access control model also would verify authorization management model based on the condition of authorization and controllable process with new technology. To solve the first problem, with the way of the user classification, this Dissertation extracts a certain categories of users in system. Their action is a part of system function. The system unifiedly gives corresponding role, and will not manage these users as the system user. For example, in online learning system that this Dissertation researcher, teachers and students users are not in the user management system and are assigned roles and permissions to. Because the action of this kind of user is defined as a subsystem, the user naturally inherits all the roles of the system; at the same time, according to some specific property values of user class, a subsystem is created based on an attribute value of user class. The subsystem based on attribute value is given role, then Corresponding to the attribute value of the user, naturally inherits the attribute values of all the roles and functions of subsystem. For the second question, based on the original RBAC access control, condition model that increased the permissions, realized evolutive and monitored permissions scheme monitoring in the modern access control model UCONabc. With the combination of these two elements, this Dissertation presents system-oriented access control model(S-UCONabc) in the fifth security authentication layer of Web multi-layer architecture.On the basis of the idea of combining the current needs and long-term development in Chinese Modern Distance & continuing education, the O-Session model is into the first and third layer structure of the online learning system, in the second layer NGINX is used as load balancing, system-oriented modern access control model of S-UCONabc is into the fifth layer; combined with the advantage of relational database and NoSQL database in the fourth database layer, a prototype system of online learning user access pressure test that 1700000000 students (user TB data) are using was designed and created. The prototype system of online learning using combined storage solution with relational database and NoSQL database.In accordance with the four principles:1.The basic information of 1,700,000,000 students are into partition table of the relational database Oracle.2.Students’course selection, students’ login information, students’ authentication informa-tion, log information stored in the NoSQL database. Because these data directly associate with the core large table, produce the mass data that are several times or more larg than this.3.Code base, data dictionary, user permissions system in the production data, save a copy in the NoSQL repository.4.Created statistical library for the use of statistical analysis database.The innovation of this Dissertation lies in:1.Put forward the access control model of S-UCONabc, which can effectively reduce the difficulty of user management, and improve the flexibility of system design.2.Propose and establish the 0-Session model, which can increase the concurrent ability of server node substantially.