Design And Implementation Of Big Data Container-based Training Platform

At present,big data industry is developing rapidly and there is a large talent gap.Many universities offer big data majors.In the training of big data talents,computer experiments are an indispensable link.However,traditional experiments have the limitations of low resource utilization and difficult maintenance.Based on the above reasons,cloud computing solutions are introduced to build a big data laboratory.The training platform is built mainly Based on the Docker container and Kubernetes.The B/S architecture is adopted as a whole,allowing users to One-Click deploy a complete set of big data experiment environment in the browser.When the user is running the experimental project,the backend uses the Kubernetes API to automatically deploy the experimental environment,and starts a cluster composed of multiple containers for the user.Provides SSH-based remote desktop function,you can complete experimental operations through the command line terminal in the browser,without installing any software.Designed and implemented a file management tool based on the SFTP(SSH File Transfer Protocol),providing users with the function of uploading files to or downloading files from the experimental container.Provides document writing function for course designers.Curriculum designers can configure the experimental environment,either by choosing the default image provided by the system,or using the image built by themselves.In addition,they can configure the container’s CPU and memory resource requirements and limits.Course learners complete the experimental tasks step by step according to the designed experimental documents.The big data training platform allocates temporary environments for users,which can be destroyed after use,and a large number of cluster environments can be simulated with a small number of machines,support for multi-user simultaneous online project training,and improve resource utilization.