Design And Implementation Of Financial Historical Data Archive And Query System

In the context of enterprise digital transformation,the digital capabilities of the financial industry are constantly improving.As an industry that relies heavily on data,the continuous expansion of online financial services of financial institutions and the continuous increase of enterprise information platforms will generate more comprehensive,multi-dimensional and large-scale business data in a wider field.Today,in the era of "data is the factor of production",the ever-increasing amount of historical data has not only brought broader space and opportunities for the development of financial institutions,but also brought new challenges and challenges to the construction of financial institutions’ information systems.Due to the particularity of financial institutions which is limited by the needs of financial supervision,auditing,judicial and other scenarios,the management of their historical data has become particularly important.Business data is often required to be preserved for a long time or even permanently.Therefore,developing historical data archiving capability is particularly needed to financial institutions.In addition,due to the particularity of financial data,financial supervision and other departments often need to query historical data from a long time ago,so developing a complete financial historical data archiving and query system is a future trend.According to the current situation of historical data management of a financial institution,this paper conducts an in-depth study on the archiving and querying of its financial historical data.The financial historical data archiving and query system has been established through designing functions such as data archiving,data storage,data management,data service,task scheduling and monitoring.Technically,the storage technology realizes the reliable storage of massive historical data through S3 Object Storage,and uses its distributed capability to ensure the reliable archiving of historical data.The query technology mainly uses Spark technology combined with Hadoop technology to achieve efficient query of archived data,while minimizing the consumption of system resources.This paper mainly introduces the construction process of the system from the aspects of system requirements analysis,system architecture design,system function design,etc.,and provides the construction plan of the financial historical data archiving and query system.The system has been tested and it shows that the design function of the financial historical data archiving and query system given in this paper can meet the needs of financial institutions for historical data archiving and management,and can also meet the query needs in supervision and other scenarios,and the system performance meets the design goals.It also provides a useful reference for other data filing system designs.