Study On The Parallel Computing In GRAPES High Resolution Numerical Weather Prediction Model

The development of high resolution precise numerical weather prediction (NWP) model is one of the mainstreams in atmospheric science and weather prediction model research. Nowadays the scale of high performance computer is steadily extended in company with the improvement of computing capability. However, whether the capability of super computer system with peak performance more than one PetaFlops can be fully utilized by numerical weather prediction model to solve its numerous scientific computing and massive data processing problems, depends heavily upon the parallel computing scheme and parallel implementation method of given numerical model, especially the computational efficiency of its core algorithm. The background of this dissertation is the Global/Regional Assimilation andPrEdiction System (GRAPES), a new generation research and operation numerical weather forecasting system developed independently by China. The scientific computing principle of GRAPES model is analyzed. After a comprehensive study on the main factors which will affect model's parallel computing design and implementation, the parallel computing scheme of GRAPES model is proposed and corresponding parallel computing system is developed. Moreover, several improved solutions are implemented for problems which are crucial to parallel computational efficiency, which included the methods for parallelization of semi-lagrangian scheme and the algorithm for solving Helmholtz equation. Followings are the main works and results included in this dissertation.1. Through a comprehensive analysis for the computing principle of NWP model, it is revealed that data parallelism is the suitable parallel computing strategy for numerical model. Since NWP model's data flow is computed in turn, there are data dependences between different integration time steps. Therefore, parallel computing of numerical model is only possible in the same integration step. By the same token, data parallelism strategy is usually adopted in the parallel computing of NWP model, i.e., the prediction region is divided into certain blocks according to the number of computing cores and computed separately.2. Since the complexity of numerical model system grows increasingly, software engineering approach should be introduced into the organizing and management of NWP software system. There are some software specifications which should be kept in software developing process. In this dissertation, a layered software framework for GRAPES is designed according to the architecture of high performance computer used in this model. A parallel programming interface (PPI) function library which complies with software engineering specification is developed, and consequently, the basic software framework of GRAPES is established. 3. The variable aggregation problem in the parallel computing of Lagrangian interpolation for the pole region of GRAPES global model is discussed. The"put-scheme"for parallel computing is proposed which is based on"supply as center", then the"get-scheme"is also implemented which is based on"demand as center", and the task dispatch algorithm is improved. It is shown by test results that both these two schemes can reduce the pole's grid aggregation effect on the Lagrangian interpolation parallel computing of upstream grids. However, the"get-scheme"is superior to"put-scheme"in computing efficiency: 1) the"get-scheme"reduces the memory requirement for pole area; 2) the"get-scheme"gets rid of the blindness of data communication in pole region; 3) the"get-scheme"extends the admissible displacement range of upstream points in low latitude region. Therefore, the"get-scheme"has better performance in load balance and parallel scalability.4. Solving Helmholtz equation is a crucial calculation and time expansive step in GRAPES model. In this dissertation, a generalized minimal residual (GMRES) algorithm based on PETSc scientific computing toolkit and Hypre parallel preconditioning function library is implemented. Compared with generalized conjugate residual (GCR) method currently used in GRAPES model, GMRES needs less iteration while has higher precision and better parallel scalability. In high resolution precise model, if the convergence precision of Helmholtz equation is improved, GMRES method will greatly promote model's calculation rate and operation efficiency in massively parallel computer.5. Through the tests with different convergence precision for solutions of Helmholtz equation, which were for the ideal running, practical material adiabatic model running and fully physical process running, the results show that the computing precision of framework is often shaded by that of physical process. However, the precision of whole model is the combination of precisions in both dynamic framework computing and physical process computing. Errors in any step of integration will have certain extent influence on prediction results. Therefore, in order to improve the computing precision of NWP model, each computing process of the model should be examined carefully.6. Lagrangian interpolation and solving Helmholtz equation are core algorithms in GRAPES. In this dissertation, parallel communication analysis models for two parallel Lagrangian interpolation methods and parallel computing time models for two solving methods of Helmholtz equation are designed on IBM-cluster1600. Through the tests which using same grid size for each processor, the parallel scalability of GRAPES model is also taken on IBM-cluster1600. Some conclusions can be drawn from the tests of GRAPES model running on Galaxy-1A super computer: 1) The parallel efficiency and scalability of integration calculations in GRAPES model is quite high. The efficiency of 10-days prediction performed in 2048 cores can reach 90%. 2) One bottleneck for the parallel scalability of GRAPES model is I/O. How to apply GRAPES model into layered computer architecture properly is another challenge.This dissertation develops a parallel system with high scalability for GRAPES model. Such parallel computing model system had been put into operation in National Weather Center of China (regional model is operational and global model is quasi-operational). It operates precisely and stably and can meet the requirements for real time operation. In the developing process of parallel computing system for GRAPES model, we also gained some experience for the parallel implementation of GRAPES data assimilation system. These works are important foundations for the development of GRAPES numerical prediction system.