Research Based On Spark Radio Interference Array Imaging Algorithm

The Square Kilometer Array Telescope(SKA)is the world's largest synthetic aperture array of radio telescopes to be built.Its basic principle is to correlate the sampling data of a large number of small antennas in the future,and obtain a giant telescope with a diameter of 1 km through subsequent data processing.The observation results meet the needs of human beings for basic research such as studying the origin of the universe.After the completion of SKA,it will generate a large amount of observational data at an unprecedented speed and become one of the world's largest data sources,bringing unprecedented challenges to the network transmission,IO and computing capabilities of distributed systems.SKA's data processing is mainly divided into imaging pipelines and non-imaging pipelines.The products of imaging pipelines are radio interference images,and the nonimaging pipelines are mainly used to search for pulsars.SKA's imaging pipeline is a typical application that is both data-intensive and computationally intensive.Developing a distributed execution framework for SKA data processing has become a research hotspot.In order to meet the follow-up construction work of SKA,this article focuses on how to correctly and reliably transplant the relevant data processing algorithms in SKA imaging and calibration into the distributed computing framework,analyze its implementation methods and performance changes,and gain an in-depth understanding of the imaging algorithms in the distributed computing framework.The implementation method and performance evaluation under the following are important for the subsequent development of SKA scientific data processing,which is also the focus of the SKA bridging stage.This article focuses on the following three aspects of work,namely:1.After understanding the relevant knowledge of the wide-field imaging algorithm,it is verified through experiments that the faceting algorithm is suitable for subsequent experiments to solve the imaging caused by the w term.2.After understanding the operating principle of Spark and the principle of radio interference array imaging in detail,the serial imaging pipeline related algorithms are implemented on Spark,and the algorithm execution efficiency is compared and analyzed.3.Analyze the efficiency of the algorithm based on the Spark radio interference array by studying the relevant parameters of the Spark algorithm when it is running.4.Through the staged analysis of the Spark pipeline imaging program,and by comparing the corresponding imaging pipeline program implemented by Dask,the analysis obtains the execution efficiency of related steps on Spark,and then points out the direction that can be further optimized.In general,the implementation of the related algorithms of the imaging pipeline in the SKA carried out in this paper can effectively run on the SPARK,laying a good foundation for the future SKA radio telescope observation and processing work.