| In the field of storage measurement,in order to obtain higher data measurement accuracy,the data acquisition time of data acquisition and storage system is longer and longer,the data sampling rate is growing,thus the data volume of acquisition and storage system is also growing,some data volume even reach hundreds of gigabytes,the number of channels acquired the data stored in the file is also more and more,some number of channels even reach thousands of channels,the number of data files acquired and stored is also more and more,some number of data files even reach thousands of files,the traditional data processing technology has been unable to satisfy the requirements of large volume data,so it is very important to study the large volume data processing technology.It is a prerequisite for research and development of data processing software which can deal with large volume data.Firstly,this paper analyzes the multiple storage measurement system,obtaining the conventional functions of the data processing software in the field of storage measurement,and then carrying on the technical research on the large volume data processing difficult problems related to these conventional functions,is also the focus of this study.Memory mapping file technology improves the efficiency of large volume data read,instead of the block reading data technology to read large volume data;The improvement of the equidistant point algorithm of data visualization puts forward the data analysis algorithm within the block to accurately capture the transient data and optimize the data display;File index positioning method instead of the traversal file name method used to find the target data file to improve the efficiency of searching target data file;The multi-thread technology based on Windows API function instead of the MSComm control interrupt service mode used for large volume data sample readings,both to solve the problems of data loss in the process of large volume data sample reading,but also improve the system efficiency;Rough punishment algorithm instead of the 5.3 smoothing algorithm used for smoothing of large volume data,so that the smoothing effect of large volume data is better.Finally,from the perspective of software engineering,using the programming language LabVIEW as the platform of software development,using the technology of large volume data processing to design and develop the vehicle test software.And using the software testing methods "white box test" and "black box test" to design software test cases,testing the software’s internal logic structure and implementation of the function.The test results show that the internal logic structure of the software is reasonable,the implementation of function is reliable,and can meet the requirements of large volume data and small volume data simultaneously,and solves the problems of traditional data processing software deal with large volume data undesirability or even can not deal with large volume data.It has a high practical value in the field of storage measurement. |
