An Application Of Functional Data Outlier Detection In Spectral Data

In order to ensure the safety and effectiveness,traditional Chinese medicine has been in the process of standardization.Currently,spectroscopy technology has been widely used.To build a regression model between the concentration of the active ingredient and the absorbance of the solution,one most suitable wavelength has to be selected in advance,which is susceptible to the type of drug solution.In fact,spectral data can be regarded as functional data for its mapping relations,so we analyze it in the perspective of functional data analysis,which is an innovative application.In this paper,for the purpose of finding the outlier spectral data,we proposed two improved methods based on the current functional data outlier detection research:(i)Oja depth outlier detection:FPCA is applied to the spectral data,followed by calculating the Oja depth for the first principle component score and the second one.Then the outliers can be determined with the boxplot detection rule.(ii)Derivative outliergram:calculate functional depth on the derivative functions of the spectral data and detect the outliers based on the numerical relationship that MBD and MEI should be satisfied.With simulation studies conducted through three examples,we compare the performance of our proposed methods with that of four existing methods on detecting the outliers and have the following conclusions:firstly,for the cases of shape outlier,the derivative outliergram and Oja depth outlier detection have the highest positive prediction value and negative prediction value respectively;MS-Plot can detect all of the outliers with high false positive rate.Secondly,for the case of magnitude outlier,functional boxplot is the best method while the two improved methods are not applicable.To illustrate the method.we analyze a spectral data of 89-dose six-mixture liquid samples.The aim is to detect the unqualified samples and compare the performance of these six methods.The results show that the derivative outliergram has the highest negative prediction value with a lower positive prediction value than MS-Plot.