Research And Design Of Big Data Processing Platform For Marine Diesel Engine

With the development of marine diesel engines towards automation and informatization,various types of sensors are widely used in marine diesel engines,which collect various parameters of marine diesel engines at all times.As a result,a large number of marine diesel engine monitoring data have been accumulated,which promotes the entry of marine diesel engines into the era of big data.Through the analysis and excavation of big data of marine diesel engines,valuable information contained in them can be found out in the “big data” era,which provides new ideas for solving many problems such as fault diagnosis,performance monitoring and emission management of marine diesel engines.In view of the urgent application demand for marine diesel engine big data,this paper analyzes the functional requirements of marine diesel engine data processing platform,presents the overall architecture of the platform,and completes the design of data acquisition,storage,and query modules for data transmitted remotely by shipborne terminals,and finally tested the performance of the platform.the specific content is as follows:(1)Combining the data sources,data features and data application scenarios of marine diesel engine big data,the functional requirements of the marine diesel engine big data processing platform are analyzed,and the overall design of the platform is given.On this basis,the key technologies of big data involved in the platform are introduced in detail.(2)For the shipboard diesel engine status parameter data and ship geolocation data of marine diesel engines transmitted by a large number of shipborne terminals,the data acquisition module of the platform has been completed,which mainly includes: development of upper software and design of Flume acquisition system.(3)According to the preservation format of ship's diesel engine state parameter data and ship's geographical location data,select the appropriate Hbase data loading method,and perform Rowkey and pre-zoning design on the Hbase table.(4)For the weaknesses of HBase that can only be queried by the row keys,the advantages and disadvantages of Hive-HBase and Phoenix-HBase architectures are compared.Phoenix is selected as the Hbase query engine,and the Hbase secondary index is implemented based on Phoenix and the Hbase Coprocessor mechanism.(5)Build Hadoop+Zookeeper+Hbase cluster,complete the performance testi ng of the Hadoop platform according to the big data platform benchmark stan dards,and achieve statistical analysis of the daily fuel consumption of marine diesel engines based on the platform.