Distributed Storing And Parallel Querying Of PDM Document Based On NoSQL

Document is one of the main information resources of Manufacturing enterprise productdata management. Almost all of the current Product Data Management softwares are based onthe product structure tree to achieve the management of the document, and by using therelational database (RDB) and file integration system to achieve the storage of thedocument.The advent of the era of "big data" makes the problems of the traditional PDMsystem based on RDB existing in the aspects of high scalability, high concurrent access andhigh availability more and more prominent, such as the server and user terminal are more andmore high-grade, the number and capacity of the storage equipment are increasing day by day,the speed of user access is slower and slower, and the time to backup is longer and longer.Meanwhile, RDB is not suitable to represent hierarchical models, and standardSQL language is difficult to directly achieve complex hierarchical query requirements,Besides, the efficiency of traditional level traversal algorithm is very low in the massive datascale.Therefore, it is very meaningful to seek higher efficiency of document storage schemeand hierarchical query scheme.Firstly, aiming to meet the different requests of the large document files which requesthigh-throughput storage and read and small document files which request rapid response, thispaper proposed to add enterprise private cloud storage platform which based on HadoopDistributed File System to NoSQL database, jointly providing the file storage service. Atthe same time, this paper proposed a storage systems comprehensive evaluation model,combined with multidimensional attribute decision theory to determine the threshold ofdistribution stored of the file.Secondly, aiming at the hierarchy traversal of large-scale product node information, thisthesis proposes a hierarchical query processing algorithm based on MapReduce.By exploitingMapReduce parallel computing mechanism, the method of product node informationprocessing is analyzed and the algorithm is designed detailedly.Finally, experimental tests were done to verify the model and algorithm proposed in this paper. The document storing experiment results show that MongoDB haveadvantage of small file storing and the threshold should be set up under17MB under thecondition of large numbers of files. The hierarchical queries experiments which based onMapReduce verified the validity of the algorithm, the experiment results show that thisalgorithm is more efficient than traditional methods in large-scale data node level inquiry.