Design And Implementation Of Online Data Science Experiment Cloud Service Platfor

As data science is widely applied in various fields,data science experiments become an important part of university teaching and research,but the traditional experimental environment has many problems such as inconvenience,inefficiency,and unscalability.Therefore,this paper aims to explore how to build a data science experiment cloud platform that provides online services to universities in a Saa S mode,and proposes an effective platform performance optimization method for the performance problems faced by the platform in high-concurrency and high-load scenarios.The main contributions and innovations of this paper are as follows:(1)Analyzing the requirements and characteristics of data science experiments in universities,designing and implementing an easy-to-use and scalable cloud platform that provides online data science experimental environment for universities in Saa S mode.The platform is based on Kubernetes technology,using GPU isolation technology to realize the multi-tenant mode of the cloud platform,ensuring the resource isolation and security of the cloud platform.And a micro-benchmark test set for the data science experiment platform is proposed and implemented,which provides a basis for subsequent performance evaluation.(2)To solve the performance problem of cloud platform caused by high concurrency and heavy load,a concurrent scheduling method for data science experiment tasks,called D2SAC-TS,is proposed.This method combines the differential evolution strategy(DE)with the SAC algorithm with discrete action space(DSAC),and uses the ideas of reinforcement learning and evolutionary optimization to adaptively optimize the scheduling strategy.This method first establishes a mathematical model to describe the scheduling problem according to the characteristics of the platform tasks and the load status of the computing cluster,and defines relevant indicators and objective functions.Then,according to the model of the scheduling problem,the elements such as state space,action space,reward function,state transition function,etc.are determined,and the D2SAC-TS model is used for learning and optimization.Finally,the trained model is combined with the platform scheduling module to realize the automatic scheduling of concurrent tasks on the platform and improve the service quality and performance of the platform.(3)Testing and evaluating the main work of this paper.Firstly,functional tests are performed on various functional modules of the data science experiment cloud platform to verify its usability and stability.Then based on the micro-benchmark of the platform,concurrent experiment tests are conducted to compare the performance differences of D2SAC-TS algorithm with other traditional,meta-heuristic,and reinforcement learning direction task concurrent scheduling algorithms from aspects such as average waiting time,average execution time,task throughput,etc.The results show that D2SAC-TS algorithm outperforms other algorithms in various indicators,and has strong adaptability and robustness.