Building A Grid Computing Platform Base On CSF4

The Grid is a new technology base on the Internet. Grid implementsdistributed computing by assembling computational resources all over theworld such as database systems, sensors and remote instruments, to supportscientific computing and provide other services.In a Grid environment, a huge mount of resources are distributed inmultiple virtual organizations. In order to utilize such a complex system, wehave to tackle some challenges never met before: 1. Heterogeneity ofresources. In the case of hardware, for example, the architecture andcapability of processors can be different. In the case of software, for example,OS and local resource managers/schedulers can also be different. 2. Dynamicresource. Usually, Grid computing is not deployed in a static environment,e.g.: new resources can join a Grid at any time, and existing resources alsocan be invalid at any time. 3. Secure resource access. Current securitypolicies in LAN can't address issues in the environment of Grid computing,such as defying vicious attacks from any sites, authentication andauthorization of inter-organization requests, and access control of resources.The meta-scheduler is able to provide a virtualized resource access interfaceto end users, and enforce global policies for both resource providers andconsumers as well, so it can solve these problems well, and it also plays moreand more important roles in computational Grids.This paper gives an implementation of CSF4 meta-scheduler andproposes a Grid computing system prototype GDIA (A Grid Infrastructure forData-Sensitive Applications) based on CSF4 meta-scheduler. It's targeting athigh performance computing and data-sensitive grid applications.CSF4 (Community Scheduler Framework 4.0) is the first WSRFcompliant meta-scheduler, and released as an execution managementcomponent of Globus Toolkit 4. It provides a bunch of services, such as jobservice, queue service, reservation service and resource management service,by which CSF can queue, submit, monitor and control jobs and reservations.By using WS-MDS, CSF4 implements resource information sharing andsynchronization among virtual organizations. CSF4 also supports ascheduling policy plug-in mechanism so that users write their owncustomized scheduling policies without re-compiling CSF4 source codes.In the reality, most of Grid organizations still adopt GT2 as their Gridinfrastructure. GT2 (i.e. Pre-WS) components adopt a traditional server/clientcommunication model, which is more efficient than the Web Servicetechnology that GT4 WS components adopt;however, interfaces betweenWS-GRAM and Pre-WS-GRAM aren't provided in GT4. This paperimplements a Grid service named ResourceManagerGramService to supportinterfacing with Pre-WS-GRAM.The resources in Grid are located in different sites and owned bydifferent VOs. Each site may have its own policies enforced by local jobschedulers, and the protocols used by these local schedulers, such as LSF,PBS, SGE, Condor etc, may also be different. On GDIA, grid users caninteract with different local job schedulers via standard GRAM protocols.Moreover, the users are able to reserve resources in advance and have morecontrol on jobs (such as pause and resume) in a LSF cluster, which has notbeen supported by GRAM yet. GDIA also extends RSL to support thesenon-GRAM features, and implements the translation betweenPre-WS-GRAM style RSL and xml style RSL.GDIA integrated Gfarm datagrid system to provide a global interface fordata-sensitive applications in a grid environment. Gfarm adopts GSI securitystandard in the Grid environment. It only accepts full proxy and L1 limitedproxy. For example, a job has been submitted to a host;after that, if this jobasks to use data from other hosts;its next data access will be rejected. In thecase of CSF4, we can relax this restriction. The underlying Security issuescan be mitigated with credentials in the community. After all, the possibilityof vicious attacks in a Grid community isn't that much. Different methodsaiming at WS-GRAM and Pre-WS-GRAM (Gatekeeper) are accepted. Thesetwo methods make use of Delegation Service of GT4 and Java CoG librespectively.Currently, GDIA have already been deployed and available for the usersof the PRAGMA (Pacific Rim Applications and Grid Middleware Assembly)testbed, which is consisted of multiple heterogeneous clusters, and enabletransparent data access by Gfarm and effective meta-scheduling of CSF4.The stability and scalability of the system is achieved by using bioinformaticsapplications, such as iGAP and wublast, as driver applications during theprocedure of development, validation and optimization.In the future, CSF4 will contain more advanced scheduling policies, suchas data aware scheduling, and co-scheduling etc. An information provider forCSF4 also will be developed. It will be designed to be more suitable forclusters, and provide statistical information such as capacity of clusters,length of job queues etc.