Research And Application Of The Multi-source Heterogeneous Data Acquisition For The Data Center In Iron And Steel Enterprises

The steel industry is a pillar industry in China.At present,most steel companies have realized production automation and information transformation.In recent years,the realization of digital and intelligent steel companies and the intelligent manufacturing of steel companies have become important propositions.Industrial big data analysis is a key method for realizing the intelligence of iron and steel enterprises.In this paper,from the point of the application value-oriented functions and goals,the data requirements to be analyzed and used for various application systems were analyzed,a new multi-source heterogeneous data collection model was proposed,and a data collection system to realize these functions was developed.(1)In the intelligent manufacturing,steel companies should realize intelligent management and control functions such as production management,quality management,energy management,environmental management,and performance management.In order to realize these functions,the collection requirements of multi-source heterogeneous data such as real-time data,business data,unstructured data,and data exchange scenarios of steel companies were analyzed,and a new multi-source heterogeneous data collection model was proposed.(2)For real-time data exchange,the design of engine and protocol analysis separated from network communication was proposed.One engine can analyze multiple data protocols.For business data exchange,the engine realized the reading and collection of relational table data by means of SQL analysis.For the exchange of historical data,the clock advance algorithm was adopted to realize the collection of the latest data and the patch collection of historical data.(3)The multi-source heterogeneous data collection system provides a unified data collection platform for multi-source heterogeneous data,with strong scalability and good reusability,avoiding the development of different interfaces to adapt to different systems.(4)The multi-source heterogeneous data collection system was applied in iron and steel enterprises.More than 50 data collection links were connected on-site,and more than 10,000 data collection signals were collected with a collection period of 1s,which realized real-time data such as automated production equipment and sensors,business data such as ERP and MES,and unstructured data such as text and surveillance video collection.