Design And Implementation Of Large Data Service Platform Based On Hadoop Cable Production

With the rapid development of Internet of Things technology,the level of automation has been continuously improved,making the amount of data generated in industrial manufacturing into a geometric progression,and the resulting quantity type presents structured,semi-structured and unstructured characteristics.Traditional data storage systems and data processing methods in the face of today’s industrial networking under the large data heterogeneous multi-source mass characteristics,often does not apply.In this paper,the cable manufacturing for the research background,for the cable manufacturing data generated by the same diversification,high complexity,large amount of data and other characteristics of the cable production of large data platform design,and the use of Internet of things data Acquisition and transmission technology,mainstream Hadoop large data processing technology and improved association rules Apriori algorithm to achieve a large data service platform based on Hadoop cable production.The main work of this paper has the following aspects:(1)Large data processing technology research and Hadoop environment structures.Studied the existing mainstream Hadoop large data processing technology,mainly Hadoop Map Reduce and HBase database,and completed the Ali cloud Hadoop cluster and HBase cluster structures.(2)Research,improvement and implementation of Apriori association rulealgorithm.According to the needs of this large data service platform,Apriori correlation algorithm is selected to analyze the correlation of mass production data.Firstly,the principle of Apriori algorithm is analyzed.Combining with the characteristics of Mapreduce calculation model of this platform,the Apriori algorithm is improved.After the simulation and test,the experimental results are analyzed,and the performance of the improved algorithm is better than that of the traditional algorithm.More suitable for large data processing conclusions.Finally,the data mining module in the platform to improve the Apriori algorithm,and for the cable production data for correlation data mining,and data display and analysis.(3)Cable design and implementation of large data service platform.The demand analysis of the large data service platform for cable production is carried out,and the function modules such as the acquisition module of production data,the distributed data storage architecture module,the real-time monitoring of the production data and the data of the order warning data are designed respectively.For the database design of the platform,based on the specific cable production data relationship,the production data were respectively relational database and HBase data storage model design.For the platform of the software framework design,mainly with the combination of Hadoop Spring MVC framework.Which in the production data storage part of the production data for the number of requests to frequently control the number of real-time data problems,the use of Active MQ message queue technology to deal with upload data.Distributed data storage framework part,through the design of Map Redcue calculation model for distributed data storage.In the implementation part,with the help of the built Hadoop platform,combined with Spring MVC framework to complete the platform to achieve.The large data service platform designed for cable production satisfies the relevant requirements of cable production and the relevant rules of mining production parameters,and has the significance of guiding production.