Design And Implementation Of Data Warehouse Based On Intelligent Manufacturing

Made in China 2025 puts forward the goal of building a strong manufacturing country,and China's manufacturing industry has gradually entered the era of intelligent manufacturing.With the development of intelligent manufacturing and the popularization of information construction,the intelligent terminals,sensors and industrial software of manufacturing enterprises will generate massive business data in production.These massive business data generally have the basic characteristics of big data,but the storage and calculation of these industrial big data are relatively difficult for enterprises.At the same time,in the information construction of enterprises,due to the reasons of information isolation,data complexity and inconsistent standards,data isolation among various industrial software has produced the phenomenon of"information island" in enterprises.The phenomenon of "information island" makes the production data not fully integrated and applied,which seriously restricts the development of data-driven intelligent manufacturing mode.How to solve the storage and calculation problems of industrial big data,help enterprises eliminate the phenomenon of "information island",and find valuable information from massive data,which is of great significance in the development of enterprise intelligent manufacturing.In view of the above problems,this paper takes the warehousing business in the manufacturing industry as an example to study,and adopts the way of big data+data warehouse to process and analyze the warehouse historical order data,mainly to solve the problems of data storage,calculation difficulties and data non interoperability.This paper designs and implements a distributed data warehouse system based on spark+impala,which processes the data related to the warehousing business of manufacturing industry in a unified way.Finally,data mining and visualization technology are used to apply the data in industrial software,solve the problem of warehouse location placement,realize the interconnection of data in industrial software,and provide decision-making optimization support of data support for enterprises。The main work of this paper is as follows:(1)Aiming at the problems of irregular,missing and heterogeneous production data,a distributed ETL System Based on kettle+sqoop is designed and implemented,which can extract,transform and load data,improve the speed of ETL,solve the problem of data update,and realize the integrity and unity of data.(2)To solve the problem of inconsistent storage in production data,a data warehouse system based on spark + impala is designed and implemented,which determines a variety of fact tables,dimension tables,dimension models and overall architecture.Spark and hive are used to transform the data,impala is used to query the data interactively,so as to improve the calculation speed in different scenarios and realize the unified storage and fast query of data.(3)Aiming at the problem that industrial data is difficult to share and use,the data warehouse system is applied.Firstly,the association rules of historical order data are mined to optimize the location of goods,and the data of goods and sales volume are visualized to realize data sharing and favorable data mining.