Research On Robustness Of Near Infrared Tobacco Quantitative Model

As a rapid, accurate and inexpensive analysis means, near infrared (NIR)spectroscopy is widely applied in tobacco industry. But tobacco is a nature complexproduct, its diffuse reflectance spectrum contains not only the physical and chemicalinformation of the sample, but also carries complex background information such asinstrument type, environmental parameters et al. The variety of the spectrum makesNIR quantitative model strong influenced by external variations, poor adaptability andcentral network modeling can not be successfully implemented.Aiming at this problem, this dissertation starts from the basic theory of the NIRspectroscopy, influencing factors on how to establish the NIR quantitative model withhigh prediction accuracy and stability were studied. It was mainly focused on the keytechniques of stability of the spectra measurement, establishment of the robustmathematical model, the calibration model transfer and central network modeling.Meanwhile some improvement suggestions were also proposed. The main researchcontents of this paper are as follows:(1) For instability of tobacco spectra measurement, the characteristics of themeasurement errors were analyzed, and the errors from the samples, operations,equipments and other aspects were also analyzed and discussed. Experiment resultsshowed that the impact of these factors will cause the instability of the NIR spectraand analysis errors. The following approaches can be used to control the fluctuation ofthe tobacco spectra measurement:Samples: The sample size should be40mesh or more, and the smaller the particlesize, the less fluctuation of the collected spectra. Sample temperature in the range of5-250C relatively had less effects on the fluctuation of the spectra. Water content oftobacco had a great influence on the NIR spectra, and in practical applications thewater should be controlled within the range of4-8%. In addition, the water content ofthe tested samples should be consistent with the calibration samples.Operations: The compression of the sample can be used to reduce the error of themeasured spectra and the use of sample cup rotation could also decrese the influence of the inhomogeneity of the samples. Meanwhile the sample powders should be fullymixed for futher error deduction. Experiment results showed that the averagesampling error accounted for about30%of the overall errors. It is recommendedthat in practice some operation trainings of spectra measurement should be carried outfor certain people in order to reduce the sampling error, while the suggested samplingerror of the operators should be less than1/3SEP.Instruments: The study found that the increasement of the resolution could notcompletely improve the performance of the model. For tobacco samples,8cm-1resolution could best meet the practical needs. Wavelength drift had a large impact onthe predicted results and standard materials should often be used for accuracycorrection of the wavelength. The starting time of the instrument also affected thestability of the measured spectra. Short starting time will lead to unstability of theinstrument, the spectra, and even the predicted results. In practice, it is recommendedthat the spectra should be measured at least1h after the starting of the instrument.(2) In order to bulid a robust, reliable and accurate mathematical model, theprocess of how to extract the useful information of the spectra and reduce theinterference of the background were studied from the aspects of both theoretical andpractical. And the factors including the calibration sample selection, wavelengthselection and model compensation that influenced the rubustness of the model werealso discussed.Calibration sample selection: Compared several calibration sample selectionmethods including the manual selection, principal component space, GH-NH and K-S,the study found that GH-NH method is optimal both from the coverage of sampleconcentration and modeling results. In practice, the combination of GH-NH andmanual selection will be the best way for the representative sample selection.Wavelength selection: A new feature selection method RFFIM-PCA based onrandom forest feature importance measure (RFFIM) and principal component analysis(PCA) was proposed in this paper. This method could effectively eliminate theinterference of niose and redundant information and choose the wavelength regionswhich are related to the tested constituents and less insensitive to the externalvariations. The model also has the higher robustness and accuracy compared withother models. Model compensation: The compensation and correction of the high water contenttobacco model were studied and it was found that adding three samples of differentwater contents could achieve the good compensation effect.(3) In order to improve the universality and adaptability of the model, somecommonly used model transfer algorithms were studied and the necessity criteria ofthe tobacco model transfer and transfer evaluation criteria were also presented in thispaper. For transferring between two different instruments, a new method(RFFIM-Shenk’s) which combined the RFFIM wavelength selection and Shenk’salgorithm was proposed and it also compared with other model transfer methodsincluding S/B, DS, PDS and Shenk’s. The result showed that RFFIM-Shenk’sachieved the better transfer effects. Due to the effect of model transfer betweentobacco shreds and powders was not ideal, a hybrid modeling strategy was adopted inthis paper. The study showed that around10%new samples added to the existingmodel could achieve desired results.(4) For the current NIR applications of the tobacco industry are mainly based onthe single instrument, central network modeling cannot be implemented successfully.In this dissertation, the construction of fast analysis network platform of tobaccoqulity was proposed. The basic network architecture, tobacco qulity database, spectralibraries, construction of the cloud computing services platform were designed andanalysied.