Research On Performance And Optimization Of Distributed File System Under Kunpeng ARM Architecture

From artificial intelligence and astronomy to atmospheric science,life science,and exploration,high-performance computing is being applied to various sectors of national economy,driving technological innovation and development in countries worldwide.The computing speed at the top of the TOP 500 has increased by nearly 20 million times from 59.70 GFlop/s in 1993 to 1102.00 PFlop/s in 2022.However,high computing speed brings high energy consumption.In the green and low-carbon era,supercomputers not only need to meet the requirements of high performance,but also have low energy consumption.The ARM architecture using the Reduced Instruction Set occupies the low-power processor market.Starting with ARMv8,the AArch64 architecture has been added to improve its performance,and is gradually developing in the field of high-performance computing.In numerous high-performance computing applications,such as atmospheric science,life science,and artificial intelligence,large amounts of data need to be processed and generated,placing high demands on file system storage management and I/O performance.As many high-performance computing applications begin to adapt to the ARM platform,it is crucial to study the migration of distributed file systems based on the ARM architecture and explore its performance advantages to improve the computing performance of high-performance applications.For the above problem,this article uses the Huawei Kunpeng 920 processor based on ARMv8.2 architecture as the computing platform to study the performance and optimization of distributed file systems on the Kunpeng ARM architecture.First of all,this article verifies the enabling of four open source distributed file systems Lustre,BeeGFS,FastDFS and MinIO on the ARM architecture;According to the enabling conditions of different distributed file systems,the distributed file systems are migrated,and their functions and performance are evaluated using different verification schemes to verify the operation performance of the four file systems under the ARM architecture.Secondly,this paper explores and studies the performance and optimization of Lustre distributed file system under Kunpeng ARM architecture,uses different parameter settings and hardware configurations to find the best optimization scheme of Lustre distributed file system under Kunpeng ARM architecture,and provides recommendations for optimizing performance;Finally,Based on research results in performance optimization,high-performance computing meteorology application WRF was deployed on the Lustre file system for performance optimization validation.The aim was to explore the impact of distributed file system performance optimization on high-performance computing applications.The research results of this paper show that the four distributed file systems Lustre,BeeGFS,FastDFS and MinIO work well under the Kunpeng ARM architecture,and all functions and performance are well performed.At the same time,based on the performance research of Lustre under the Kunpeng ARM architecture,this paper’s optimization scheme effectively improves Lustre file system performance.In the deployed high-performance computing application WRF,the use of the optimization scheme has improved the overall performance of WRF by 13.85%,including 12.8 times the performance of WRF file I/O.This article analyzes the enabling situation and performance optimization research of distributed file systems under the Kunpeng ARM architecture,which can not only supplement the Kunpeng ARM architecture ecology,but also provide a reference for performance optimization of distributed file systems under the Kunpeng ARM architecture.