Design And Implementation Of Elastic Deep Learning Model Deployment Platform Based On Kubernetes

Artificial intelligence and cloud native are the current mainstream frontier technology fields driven by open source software.Therefore,AI platforms based on cloud native architecture are also constantly being explored and practiced.As one of the important branches in the field of artificial intelligence,deep learning has been deeply researched and achieved considerable results on how to train accurate network models.However,from data collection to the construction of network models,the ultimate goal of deep learning projects is to deploy models online to provide services.Therefore,the current hot technology of artificial intelligence extends from improving the accuracy of algorithm models to the engineering of models.There are the following problems in the process of model landing engineering:(1)Different models have complex requirements for environment and version dependencies when encapsulating,and conflicts are prone to occur between models;(2)Model services need to consume a lot of computing resources and require Fully consider the impact of elastic resource allocation on cluster stability;(3)As a delay-sensitive task,model service has higher requirements on request response time and success rate.Therefore,the main research goal of this paper is to explore and design a flexible model serving method and design and implement an efficient model deployment solution.Aiming at the problem of deep learning model landing engineering,this paper builds a deep learning model deployment platform based on Docker container and Kubernetes container management technology,which encapsulates and automates the deployment of the trained model,and realizes elastic scaling and real-time monitoring of resources.Function.The main work of the thesis includes:(1)Distributed deployment of model services based on Kubernetes technology,and optimized for the lag problem existing in the native responsive expansion and contraction mechanism,a load prediction model based on EMD-LSTM was proposed,which was predicted according to historical load conditions.The future trend is to perform expansion/reduction operations in advance before high/low loads come.In the study of the load forecasting model,through the experimental comparison with the traditional statistical method ARIMA and the neural network-based algorithm LSTM,two current mainstream time series forecasting algorithms,it is verified that the EMD-LSTM model proposed in this paper is suitable for the load in the cloud environment.Predictions have higher accuracy.(2)For the current mainstream deep learning frameworks Py Torch and Tensorflow,this paper studies how to encapsulate the model trained based on these two frameworks into a service,and how to evenly forward the traffic to the back-end nodes when the service is deployed on multiple nodes.The load balancing of requests,combined with the load prediction model proposed in this paper,improves the native elastic scaling mechanism of Kubernetes,and designs and implements an elastic deep learning model deployment platform.Finally,the platform is tested from the two aspects of function and performance.In the performance test,it is verified that the improved predictive elastic expansion and shrinkage mechanism in this paper can successfully predict the future load trend,and perform expansion/reduction operations in advance,thereby avoiding the need to achieve The sudden increase in response time and the rejection of a large number of requests caused by the expansion operation at the load threshold effectively reduce the average response time of requests,improve the success rate of requests,and enable the model to provide services more stably.(3)In order to be able to know the running status and resource usage of the cluster at all times and ensure that all nodes are in normal working state,the model deployment platform is designed to monitor the Kubernetes cluster.