Research On Parallelization Method For Self-calibration Of Radio Interferometric Visibility Data

Modern radio interferometric arrays are mainly composed of many telescopes,and their unique design provides the array with a wide field of view and the ability of rapid detection.These interferometric arrays present new scientific and technical challenges as well as more detailed sky brightness distributions.The large increase of observational data scale makes the storage and numerical calculation of massive astronomical data become the key factors in the field of radio astronomy research.In the traditional stand-alone environment,Open MP,GPU+CUDA and other methods have great coupling with the existing radio astronomy algorithms,which is not conducive to the rapid transplantation of the algorithms.Under the framework of DASK,the decoupling of task scheduling strategy and radio astronomy algorithm is conducive to the elastic scaling of the algorithm in computer hardware and software resources,and the migration from a single computer system to a distributed cluster can be realized.In this paper,the gain self-calibration in radio interferometric visibility data is taken as an example to conduct data processing and analysis on the Measurement Set(MS),and the parallel optimization and implementation of gain self-calibration algorithm are carried out in combination with the open source distributed parallel framework of DASK.The specific research contents are as follows:1.The principle of gain self-calibration algorithm is analyzed on radio visibilities,and the parallel loading of measurement sets,and the parallel numerical calculation of self-calibration are realized by combining with various parallel strategies provided by DASK.2.The DASK framework is used to implement the gain self-calibration algorithm in multi-thread parallel,and the algorithm is transplanted from a stand-alone system to a distributed system.3.Test the performance of gain self-calibration algorithm based on Dask.Array in the distributed system and provide visual dynamic monitoring.The experimental results show that compared with Num Py.ndarray,the gain selfcalibration based on Dask.Array is an effective way to solve the numerical calculation of massive astronomical data,improve the efficiency of matrix calculation,and give full play to the utilization of multi-core CPU in distributed system.In this paper,several self-calibration algorithms are implemented based on the parallel framework of DASK.The running environment is not limited to single machine and distributed system,and the execution mode of multi-thread or multi-process is satisfied,which improves the adaptability of hardware and software resources.The main innovations of this study are as fellow:(1)Dask.Array is used to store and calculate astronomical data instead of Num Py.ndarray;(2)To improve the utilization of multi-core CPU by using the parallelism of fine-grained programs with lazy computing and matrix blocked computing strategies;(3)Using the DASK framework,shift to highly parallel processing to improve the performance of the gain selfcalibration algorithm for high performance computing.It is foreseeable that the use of the DASK framework will become necessary for parallel processing of large amounts of data in the field of radio astronomy.