Design And Implementation Of Artificial Intelligence And Big Data Module Of Smart Pipe Gallery Cloud Platform

The Artificial Intelligence and Big Data module is an importance part of the cloud platform of smart pipe gallery.The module collects,preprocesses,stores,analyzes and applies the data generated in the operation and maintenance process of the pipe gallery,which has the function of enabling the whole platform.Based on the existing structure of the smart pipe corridors,this thesis designs and implements an Artificial Intelligence and Big Data module specifically facing the actual application scenarios and business requirements.Its main components include:Big Data platform,automatic monitoring of abnormal events in the pipe gallery and automatic inspection of camera failures in the pipe gallery.The main work of this thesis can be summarized as follows:1.Based on the existing structure of the smart pipe corridors system,establish a Big Data platform specifically for the smart pipe gallery using Hadoop technology stack(Hadoop,Spark,Hive,etc.).Technologies like Sqoop can help importing the data generated by other modules of smart pipe gallery from original database(such as MySQL)into the Big Data platform in full or incrementally.As for the data that has been uploaded to the Big Data platform,we use Spark to do some customized data preprocessing,such as data cleaning,data normalization,etc.In addition,we use Hive to do query and analysis on existing data to provide data support for the platform-related decisions.Finally,the Big Data platform provide support from model training to online deployment.The above functions greatly improve the service efficiency of the smart pipe gallery operation platform.2.This thesis uses object detection technology of computer vision to monitor the abnormal events in the pipe gallery in real time such as abnormal people,abnormal fire,abnormal smoke,etc.which reduces labor costs and improves inspection efficiency greatly.According to the actual application scenario of the underground pipe gallery(faster prediction speed,less hardware resources,etc.),this thesis makes corresponding improvements on the basis of Yolo v3 model,mainly using EfficientNet-Lite to replace its original backbone,so that the model accuracy is not significantly reduced,the model volume is greatly reduced,and the detection speed is significantly improved.3.This thesis uses video quality assessment technology to do automatic inspection and generate inspection report for future references,checking whether the camera brightness is abnormal,whether it is blocked,whether there is snowflake noise in screen,whether it is jittering,etc.This technology helps building an automated,intellectual and informational pipe gallery and reducing operation and maintenance costs as well.