Studies On Methods And Its Application Of Calibration Model Optimization And Transfer Of Molecular Spectrometric Data

Abstract:Molecular spectroscopy, including ultraviolet-visible spectroscopy, near infrared spectroscopy (NIR), mid-infrared spectroscopy and Raman spectroscopy, has been widely used in routine chemical analysis. For example, NIR and Raman spectroscopy combined with chemometrics have gained wide acceptance in many fields by virtue of high speed, low cost and its ability to record spectra for solid and liquid samples without any pre-treatment. However, these spectra usually consist of weak signal, complex background and overlap peaks due to interference and other optical effect. The key to success in spectrometric application is extracting useful information from the complex spectra and making a good model. In this thesis, outlier detection, spectra preprocessing, wavelength selection, calibration, calibration transfer and multi-spectra fusion methods are studied.1. The elimination of singular sample has important role in improving of model robustness and accuracy. With the help of Monte Carlo sampling, an outlier detection method is applied to the NIR milk data and Raman gasoline data. Outlier detection method based on Mahalanobis distance and robust PLS method are used as reference methods. The results indicate that the model performance improved significantly by using the outlier detection method.2. A wavelength selection method based on competitive adaptive reweighted sampling is used to find the most relevant and important variables and make a compressed model for NIR spectra and Raman spectra. For classification of different vinegar,11wavelengths are selected and a PLS-DA model is calculated based on these selected wavelengths. For prediction of total acid of vinegar, only5wavelengths including4348cm-1,4694cm-1,5365cm-1,7104cm-1and7236cm-1are selected and a least squares regression (LS) model is build on the selected wavelengths. For classification of different vegetable oil,4Raman shift are selected and the result is shown in PCA score plot. For determination of alcohol content in white wine,27Raman shift are selected to build the model. All the results indicate that wavelength selection improves the prediction result of spectrometric data.3. In order to solve the calibration transformation problem, a new method based on canonical correlation analysis (CCA) for calibration model transfer is developed. CCA is a very powerful tool that is especially well suited for relating two sets of measurements (spectral response of two instruments). Compared to PCR or PLS, CCA exploits the correlation rather than the covariance. In this method, after mean centering of the data, canonical correlation analysis is carried out between the spectra. Then the canonical vectors are used to obtain the transform matrix. It is shown that the transfer results obtained with the proposed method based on CCA are better than those obtained by PDS when the transfer subset has sufficient samples.4. Two new calibration methods combined with wavelet preprocessing are developed and employed to make calibration model. The two methods are the random forest(RF) and support vector machine(SVM). Random forest coupled with near infrared spectroscopy are used to detect and identify honey adulteration. Support vector machine coupled with Raman spectroscopy are used to determine the aromatic hydrocarbon in gasoline. Both the results obtained by the new methods are better than ones obtain by PLS model.5. How to do properly model validation becomes very important in spectral analysis. However, the comparison conduced in most of spectral analysis is based on the use of a single dataset or cross validation with fixed sample partition, which obviously takes the risk of drawing a wrong conclusion. By changing test sets or sample partition, the distributions of prediction error of different samples were therefore derived and further compared statistically, thus allowing for reliable comparison.With the help of Model Population Analysis, the proposed method is employed to analyze a NIR dataset as well as a Raman dataset. The results showed that the method can avoid drawing a false positive conclusion.6. There are different types of NIR instruments and Raman instruments. Including portable spectrometer and large instrument used in laboratory, CCD detector spectrometer and Fourier transform instrument. Different performance is achived by these instrument with different resolution. With the help of suitable chemometrics methods, some weak performance instrument can improve the prediction performance, and get the results as better as the good instrument.7. Every kind of spectra has its advantages and disadvantages. A new multi-spectra fusion method based near infrared spectra and Raman spectra is studied in this thesis. After preprocessing and principal component analysis, NIR and Raman are integrated by used of different principal components. PLS model is established by the multi-spectra and the prediction accuracy is increased compared with the results obtain by single kind of spectra.