Multilevel Iterative Methods And Solvers For Reservoir Simulation On CPU-GOU Heterogenous Computers

The black oil model and its generalization are the fundamental mathematical models for describing the reservoir simulation problems and the most widely used computational models in modern oil industry. While numerical study of fast al-gorithm and corresponding solvers for the large scale discrete systems of these oil reservoir models has recently made great progress, there are still many issues requir-ing further study. In this work, we study efficient parallel preconditioned GMRES methods based on multi-grid method to solve the black oil model and a polymer flooding model on CPU-GPU heterogeneous computers. The main research findings and innovations are as follows.Based on multi-color ordering strategies and geometry multi-grid method, we develop two parallel Poisson solvers called GMG-V-CUDA and FMG-CUDA for the Poisson equation’s finite difference discrete linear systems on CPU-GPU heteroge-neous computers. Our numerical results suggest that GMG-V-CUDA have better al-gorithmic and parallel scalability than GMG-V-OpenMP which is an efficient Open-MP implementation of multigrid methods on CPUs. Furthermore, FMG-CUDA is more efficient than the Fast Fourier Transform in the state-of-the-art cuFFT library for solving the Poisson equation. Of at least equal importance is that, in our setting, GPU is more cost-effective (in terms of initial cost and daily energy consumption) than modern multicore CPUs for geometric multigrid methods.We present a colorization algorithm by using connection strength matrix, the impact of factors, path length and other factors. At the same time, we give an upper bound estimate for the number of colors needed. Based on the proposed coloriza-tion and ordering algorithm, we present a parallel Gauss-Seidel iterative method, by using the principle of the smallest sparsity factor priority. Furthermore, based on the parallel Gauss-Seidel iterative method, we propose two corresponding par-allel algebraic multi-grid methods called RS-AMG-MC and SA-AMG-MC. We develop their corresponding serial and parallel solvers on CPU-GPU heterogeneous computers. By comparative experiments on the Poisson problems and the pressure equations arising in reservoir simulation, we show that SA-AMG-MC improves con-vergence rate of SA-AMG, and RS-AMG-MC performs better algorithm and parallel scalability than SA-AMG-MC does.Considering the standard black oil model and polymer flooding model, we ob-tain their Jacobian systems by using the Backward Euler scheme time-discretization, the Operator Newton linearization, and spatial Upstream Weighted Finite Differ-ence scheme discretization. A multi-stage preconditioned GMRES method is pro-posed for these Jacobian linear algebra systems by taking advantages of physical and analytical properties of the reservoir models. Furthermore, two multi-stage precon-ditioners BMSP1and BMSP2are designed for reservoir equations without and with implicit wells, by using Auxiliary Space Preconditioning method and Alternating Block Factorization method, respectively. Our numerical experiments demonstrate that BMSP1and BMSP2are more robust than ILU(0) preconditioner and CPR pre-conditioner. Furthermore, exploiting the grouping technology of the DoFs from the connection strength matrix and data-structure, we develop the parallel solvers MSP-GMRES-OMP and MSP-GMRES-CUDA under OpenMP and CUDA parallel programming environments. Numerical experiments show that both MSP-GMRES-OMP and MSP-GMRES-CUDA have favorable algorithmic and parallel scalability.We investigate the efficiency and robustness of the proposed methods (or solver-s) by applying it to three industry benchmark problems (SPE1,SPE9and SPE10) from the SPE (Society of Petroleum Engineers), two real-world matured reservoir fields with high heterogeneity, high water-cut, geological faults, and complex well scheduling, and a reservoir fields mechanism model. Numerical results indicate that the multi-stage pre-condition GMRES solvers based on algebric multi-grid method are suitable to solve the standard black-oil model and the model of polymer flood-ing. It also indicates that the proposed methods are robust with respect to the heterogeneity, anisotropy, and number of wells. Furthermore, the proposed solvers obtained a great success in solving scale and computational efficiency. For example, the OpenMP parallel speedup is up to3.2times, and the grid size of simulated prob-lem is up to10million cells on a HP desktop workstation, and the CUDA parallel speedup is up to3.6times based on a single GPU card (Geforce480).