Data Mining And Application Technology Of Crop Disaster Spectra And Images Based On Cloud Computing

With increasingly wider application of the information technology in the field of agriculture,the sources of agricultural data have become broader,the data dimension keeps getting higher,the updatse speed has become faster,and the date types have also become more diverse.By monitoring and measuring various aspects of the physical environment,the information and Internet technology has generated mass data in an unprecendented speed,which requires large-scale colelction,storage,preprocessing,modeling and analysis of mass data from various heterogeneous sources.The mass agricultural data need to be intergrated into the computer system through various technologies such as artificial intelligence,machine vision,data mining and cloud computing.By building the information base and exploring valuable information from it,it can provide decision-making support for agricultural production,improve yield,reduce the cost and reduce the negative impact of agricultural production activities on the environment.Big data processing is a main issue that needs to be addressed during processing and analysis of agricultural data.Based on parallel computing and distributed memory,the cloud computing resources and operation can be distributed to various nodes of system,which can accelerate the speed of processing,analysis,storage and decision-making of big agricultural data.In particular,in order to achieve high-concurrency reading and writing,efficient memory and access of high dimensional and mass data,high extendibility and high usability,it requires reasonable big data analysis and processing technology to explore the value of agricultural data and improve the data analysis and application ability.In this paper,we study the data mining and parallel computing of the spectral images of crop disaster on the aspects of crop disaster,visible-near infrared spectrum,hyperspectral imaging and cloud computing,and the main research works and innovation points are as follows:(1)Based on the PCA,characteristic waveband selection and spectroscopy theory,we explored the characteristics of crop visible-near infrared spectrum under pest stress,conducted principal component extraction of visible-near infrared reflectance spectrum under different levels of pest stress,and extracted the optimal wavebands with the simulated annealing algorithm.Then,we used the machine learning methods such as hierarchical clustering,K-Means,Fuzz C-means and SOM to conduct clustering analysis of crop pest spectrum,and explored the application potential of different machine learning algorithms in the identification of the visible-near infrared spectrum of crop pest.(2)Based on the research in(1)above,we proposed a parallel classification and identification method for visible-near infrared spectrum based on cloud computing.We used the Hadoop and Spark frames to build the cloud computing platform,conducted classification research on the collected visible-near infrared reflectance spectrum of crop freezing injury,used the MLlib machine learning library provided by Spark to realize multiple machine learning methods such as decision tree,random forest,artificial neural network and SVM,analyzed the feature extraction and classification of collected spectral information,and conducted spectral classification and identification of crop pest and freezing injury.The experimental results show that compared to the traditional spectral analysis technology,the parallel spectral data mining algorithm based on cloud computing can provide better speed-up ratio and higher data mining efficiency.(3)We extracted the spectral and image information from the hyperspectral image by collecting the hyperspectral image information of crop waterlogging,conducted PCA and color space conversion of RGB image,and conducted visual distribution of image information.The SPA algorithm was adopted to select characteristic waveband of hyperspectral image,and extract the optimal waveband of hyperspectral image.The spectral and image information in hyperspectral image was integrated,and different classification models were used to conduct classification prediction and identification of hyperspectral image under different levels of waterlogging.(4)Based on the research in(3),we proposed a two-stage classification and identification method for the hyperspectral image of crop waterlogging based on cloud computing.By extracting the visible-near infrared spectrum information and digital image information in the hyperspectral image of oilseed rape,preprocessing and space conversion were conducted to remove the image and spectral noise,and build corresponding spectral matrix and image matrix respectively.The Spark platform and MLlib machine learning library were used to realize the parallel classification algorithm.The collected images and spectral matrices of crop waterlogging were classified and analyzed;then,the classification results of image and spectral matrices were integrated to obtain the final classification and identification results.In this way,the lossless fast detection method for hyperspectral image of crop disaster based on cloud computing can be realized and built.Furthermore,we also conducted experiments and proved that under the precondition to ensure the classification and prediction accuracy,the parallel classification algorithm has the advantages of high speed-up ratio,extendibility and size-up,which can improve the classification efficiency of crop hyperspectral images.