| Nowadays, as the increasing of kinds of computation demand, many organizations have purchased various Cluster systems to provide service and meet their demands. These Cluster systems are all dependent entity, have their own operation systems, users, and provide specific computing services. So how to effectively manage running status, users and applications of compute resource is an important and meaningful problem.Cluster job management system is the key of a cluster, and the Job scheduling algorithm is the core of Cluster job management system. A good job scheduling algorithm not only reduces the waiting time of a job, but also makes full use of resources in Cluster system and improves CPU utilization.This paper focus on the research of the Cluster's job scheduling, includes the job scheduler, job scheduling algorithm and the job scheduling simulator of cluster system. We design and implement a scalable job management system for the heterogeneous Cluster, which supports plugged-in job scheduling algorithms and meta scheduling. And against the weakness of Backfilling algorithm and scheduling algorithm of network information crawling system, we propose the improvement, and do the experiment in the GridSim job scheduling simulator.Here are some of the major works of this thesis:(1) Design and implement an extensible job management system on a extensible and heterogeneous management platform base on JMX, in order to manage efficiently the computer resources and support job scheduling;(2) Research on the classical backfilling algorithm and analysis the weakness of the algorithm. This paper proposes and realizes the improved algorithm, then does the experiment in the GridSim job scheduling simulator;(3) According to the requirement of specific application, and based on the research on the job scheduling algorithm of the open source search engine Nutch's information crawling system(Crawler), this paper proposes the improvement and does the simulations in GridSim. |
PDF Full Text Request