Research On The Checkpoint-Based Task Scheduling Methods In Shard Clusters

With the development of modern data center,cloud computing is getting increasingly efficient and convenient.Cloud computing,supported by the shared clusters,has become an irreplaceable part of internet technology.More and more commercial companies and research institutes start adopting this technique in the area of health and education by constructing either public or private clouds.The successful deployment of cloud computing in the commercial and civilian area draws attention of many military researchers and decision makers.They believe in that applying the cloud computing technology to the C2 information systems can bring significant benefits.However,the traditional preemptive scheduling scheme in commercial cloud platform has some obvious drawbacks,such as the long response time and the low efficiency of tasks with low-priority.In the C2 information systems,it is unacceptable to sacrifice those low-priority tasks in the same way.Therefore,improving the quality of service for low-priority tasks while consistently ensuring the performance of tasks with high-priority becomes an urgent issue.The checkpoint technology,a method to save the progress while being processing,is famous for its appliance in the high performance computing area.By making checkpoints for certain tasks,they can be restored from their checkpoints if they are killed,instead of being restarted from the beginning,which saves time and resource.However,traditional checkpoint technology is normally being abandoned by commercial companies because of its low utilization but relatively heavy overhead.Based on the checkpoint technology,this work focuses on the checkpoint-based task scheduling methods in shard clusters.The main contributions of the work are as follows.(1)The concept of hybrid scheduling list is proposedThis paper proposes the concept of hybrid scheduling list which can cover all unfinished low-priority tasks,including tasks to be scheduled,checkpointed tasks and uncheckpointed tasks.The hybrid scheduling list keeps the execute and resume of tasks in order.(2)The checkpoint-based scheduling method is proposedWe propose a novel approach named Chord(Checkpoint with hybrid scheduling method).It not only guarantees the performance of high-priority tasks,but also leverages checkpoint technology to improve the performance of low-priority tasks,thereby benefits the efficiency of the whole cluster.(3)The effectiveness of Chord is validatedWe conduct extensive experiments by obtaining tasks from the Google cloud trace logs.The experiments comparing our propose with some baselines in the respects of average response time,whole efficiency and the occupation of resource,which shows the superiority of the Chord.Experiments also present the empirical optimum for two key parameters in the Chord,providing valuable reference for its real application.