Research On The Agricultural Environment Big Data Processing System For The Internet Of Things

With the continuous development of information technology,the amount of data in modern society has shown explosive growth,and the amount of data generated by human beings is more than ten thousand times that of ancient human beings,gradually moving gradually from the Internet era to the era of big data.In China,agriculture is the primary industry.With the continuous maturation of agricultural mechanization and the development of Internet of Things technology,Internet technology and wireless sensing technology,China’s agriculture is gradually moving into the information era and the era of big data.How to effectively tap these massive agricultural data and obtain high-value information from them to better guide agricultural development is an important prerequisite for the realization of agricultural intelligence and informationization.As a large agricultural country with wide geographical distribution and complex agricultural production environment,China has a large volume of traditional agricultural data,long time span and slow update,while the new agricultural data generated by agricultural Internet of Things is unstructured and time-sensitive,so this has put forward higher requirements on big data technology for processing agricultural-related data.In this paper,we analyze and study the needs of environmental data sensing applications in agricultural Io T and design an agricultural environmental data processing system based on Spark cluster.The system is divided into three major functional modules to realize the pre-processing,storage and computational analysis of agricultural environmental data.Firstly,the system is built on a fully distributed Spark cluster with distributed storage of data through HDFS;secondly,the machine learning library of Spark MLlib combined with clustering algorithm is used to process agricultural environmental data;finally,Spark SQL is used to realize data query and other operations.The experiments selected greenhouse watermelon growth environment data for clustering analysis,and clustered the most suitable growth conditions of watermelon with two algorithms,comparing the accuracy of K-means and GMM algorithms,reflecting the effective improvement of the performance of the clustering algorithm by distributed storage and parallelized computation,and illustrating the importance of the clustering algorithm for agricultural environmental data mining.The experiments prove that the agricultural environmental data processing system designed and developed in this paper is of great practical importance for the application of environmental data processing of small and medium-sized greenhouse crops and the development of agricultural production.