Design And Implementation Of Algorithm Execution Engine In Data Publishing Platform For Privacy Protection

With the development of mobile Internet technology and personalized services,the issue of personal privacy leakage is gradually becoming the focus of public attention.By integrating numerous privacy protection algorithms,the privacy data publishing platform can improve the security of data while ensuring data availability as much as possible,so that users can publish data with confidence.The existing platform is still inadequate in the management of algorithms and their dependencies,algorithm execution and task scheduling,which affects the use of core functions to a certain extent,so it is necessary to redesign and implement the relevant parts of the platform with new technical solutions.The solution of using containers to encapsulate and isolate the execution of algorithms is proposed to solve the problems of environmental requirements for the execution of privacy-preserving algorithms and environmental dependency conflicts among different algorithms,and based on this,an algorithm execution engine based on container technology is implemented,and a set of priority-based memory-based scheduling policies is designed to improve the performance of the engine under large task volume.The engine is divided into an algorithm management module,a task execution module,a task monitoring module and a task scheduling module,and provides interfaces for third-party platforms to use the engine directly.Among them,the algorithm management module is responsible for managing algorithms and their dependencies,the task execution module is responsible for encapsulating the specific operations of containers during task execution,the task monitoring module is responsible for monitoring the operation status of containers corresponding to tasks,and the task scheduling module is responsible for controlling and adjusting the running containers in conjunction with the scheduling policy.A functional and scheduling test plan was designed for the algorithm execution engine and tested in practice.The test results show that the functional modules of the algorithm execution engine work properly and are able to schedule tasks according to the scheduling policy,which improves the overall performance of the algorithm engine for a large number of tasks.