Design And Implementation Of Data Acquisition System For The Formation Process Of Blockchain In Bitcoin

As a critical infrastructure for constructing distributed ledgers and enabling value interoperability,the Blockchain system has become a huge data source as the amount of data on the blockchain grows.For example,the data produced during the blockchain formation process displays many micro-data about blockchain operation.The micro-data play an important role in many research work,such as blockchain platform construction,finance,and security.However,due to limitations in data size,operating efficiency,and performance,blockchain systems do not record and retain these micro-data.Therefore,the data produced during the blockchain formation process could be more traceable and easier to obtain.This thesis designs and implements a data acquisition solution for the data produced during blockchain formation to solve the above problems.Firstly,this thesis formulates data standards to describe and represent the blockchain formation process.The data standards describe the blockchain formation process at four scales: block synchronization process,block synchronization stage,block synchronization event,and block synchronization time,and design appropriate data structures to represent the blockchain formation process at each scale.Based on the data standards,this thesis implements the data acquisition solution using the code instrumentation technique.The instrumentation code and instrumentation output methods have been well-designed to ensure the independence of the data acquisition process and provide extension capability of data.After completing the design and implementation of the data acquisition solution,and considering the challenges of the data acquisition solution,such as data acquisition process control,data acquisition scale improvement,sharing and use of data acquisition result,this thesis designs and implements a data acquisition system for the data produced during the blockchain formation.The specific work includes:(1)Implementing full control over the data acquisition process: This thesis divides the data acquisition process into two stages: data output and data exporting.The data acquisition node is only responsible to acquire and output data.And the work of reading and exporting data is completed by other system modules.Through the above design,the data acquisition progress can be transformed from data output progress to data exporting progress.Since the data exporting progress is entirely controllable for the system,full control over the data acquisition process is achieved.(2)Supporting parallel data acquisition under multiple data sources: The data acquisition work under a single data source has limitations and cannot support more research work.Therefore,the system needs to support parallel data acquisition under multiple data sources.To effectively improve the data acquisition scale,this thesis fully considers the efficiency issues in data acquisition,data processing,and data access.From the perspectives of architecture design,functional module design,and code implementation,this thesis finally supports parallel data acquisition under any number of data sources.(3)Supporting data opening: There are many problems in data sharing and permission allocation when using data,which has become a critical obstacle to better utilizing the data.Therefore,this thesis designs and implements a data opening service to provide data opening interfaces for external parties.The interface supports various types of data querying and data mixing under multiple data sources.The interface has good compatibility and scalability and can adapt to different data usage scenarios.(4)Optimizing the application capabilities of the data acquisition system: This thesis improves and optimizes the application capabilities of the data acquisition system from various aspects enabling the data acquisition system to support a wider range of data applications,including data real-time,data reliability,data accuracy,efficiency of storing and retrieving massive amounts of data,as well as system availability,scalability,and extensibility,After completing the design and implementation of the data acquisition system,this thesis deploys three data acquisition nodes to acquire data and complete functional testing.This thesis also conducted a non-functional evaluation of the data acquisition system from two aspects: data real-time and system failure rate.The evaluation results show that the system has sufficient application capabilities and can fully adapt to the data demands of most research work.Finally,this thesis summarizes the shortcomings of the current system in terms of the richness of data acquisition,automation control of the data acquisition process,scale enhancement of data acquisition,real-time and reliability of data,automated operation and maintenance of the system,and prospects for future work.