Research On Task Scheduling Optimization Methods Of Mine Safety Supervision Cloud Platform Based On Hadoop

With the application of the Internet of Things in coal mine production activities,a large number of heterogeneous complex transducers have been set in the coal mine.These transducers are used for real-time supervision of production process,safety environment,personnel,equipment,etc.,resulting in an exponential growth of supervision data collected by the transducers.To guide safe production,massive coal mine safety supervision data need further mining and utilization.Traditional data processing and storage technology can no longer meet the practical needs of mine big data.The massive data storage capacity and distributed parallel processing capacity of cloud computing technology provides strong technical support for the integration of various data resources of mine safety production supervision system and the storage and data mining of massive data.In this thesis,Hadoop cloud platform is applied to the safety supervision system of coal mine.In order to improve the efficiency of mining massive data,the task scheduling optimization method of safety supervision cloud platform is studied from two aspects of supervision data itself and cloud platform scheduling algorithm.This thesis mainly carries out the following work:(1)The mine Internet of things,Hadoop cloud platform and resource scheduling algorithm are summarized.(2)The architecture of the coal mine safety supervision system is described,and the collection and storage of the coal mine safety supervision time series are illustrated in combination with the safety supervision system of Huaibei Mining Group.In order to improve the quality of the data mining,it is necessary to preprocess the original data.Combined with the actual situation of Huaibei Mining Group,this thesis introduces the data collection and preprocessing technology of coal mine safety supervision system.Aiming at the compressed coal mine safety supervision data,a data decompression method based on piecewise interpolation is proposed.The method segments the time difference between every two data records and inserts the random value between two records in each segment,which effectively fills the missing data.Aiming at the outliers in the coal mine safety supervision time series,a method of cleaning up the outliers in the safety supervision time series is proposed.This method cleans up different kinds of outliers in data and improves data quality.(3)The mine safety supervision cloud platform is a distributed heterogeneous cloud platform,while the homogenization assumption of the default scheduling strategy of the Hadoop makes it impossible to dynamically adjust its task share according to the comprehensive computing capacity of cluster nodes,which may lead to resource waste of advantage nodes and resource strain of disadvantage nodes.Moreover,the resources allocated for all tasks are of uniform specifications,which may lead to a large number of fragmented resources,resulting in decreased scheduling performance of Hadoop tasks in heterogeneous clusters.To solve this problem,an adaptive task scheduling strategy based on similarity analysis is designed in combination with the heterogeneous mine safety supervision cloud platform.This strategy can adaptively adjust the resource share of the task according to the real-time operation of the job and the computing capacity of the node.Experiments show that in heterogeneous clusters,the optimized scheduling strategy can improve the execution efficiency of jobs,shorten the execution time of jobs and improve the utilization of cluster resources.