The Analysis And Processing Of Splicing Abnormality Spectra In LAMOST

In October 2008,China's large area multi-object fiber spectroscopic telescope(LAMOST)ceremony held at the national observatory Xinglong observatory.The telescope was conducted in the October 2011 pilot survey,in order to check the equipment performance and evaluate the feasibility of the survey plan.In September 2012,LAMOST started sky survey officially,by the end of the June 3 2013,a total of approximately 4149500 spectra,including a pilot survey of 1338750 spectra.In September 2013,LAMOST officially released DR1 data which included altogether 2204860 spectra.It produce a large amount of spectra in LAMOST observation,we noticed that the released spectra meet the quality requirements accounts for only 70%of all observed data.Even in the published data,there are also poor quality of the spectra.Among them,the abnormal splicing spectra is one kind of poor quality spectra and this article mainly research the mining and analysis of the abnormal splicing spectra in order to mine abnormal splicing spectra in the mass spectra data.The research contents of this paper include:(1)Introduction of related technologies.This paper is divided into three parts.The first part introduces the python language and the astronomical data processing technology based on python.It mainly introduces the package and the convenience of spectral processing when dealing with spectral data based on python.The second part introduces the architecture and basic principle of high performance computing platform.The third part introduces parallel computing technology on high performance computing platform of python.(2)Identification and classification of abnormal splicing spectra.This is the core part of this paper,which mainly introduces the principle of the method,the threshold value and the custom recognition function of abnormal splicing spectra.According to a large number of experiments,we extract mathematical statistic feature of the spectrum and according to these characteristics,the thresholds are defined.After experimental analysis,the evaluation function of a spectral anomaly classification is proposed.Through the function,the abnormal spectra are divided into three grades to provide different quality levels for spectrum spectroscopic study.(3)Parallel processing of abnormal spectrum recognition in high performance platform is introduced.In this part,we study the parallel implementation of the method of serial anomaly spectrum recognition in order to run on the platform,and we introduce the usage of high performance computing platform.Splicing abnormality is a phenomenon of poor continuity spectrum showed in the splicing wavelengths of the red and blue end.In the spectral processing,this problem can be caused by several factors,such as stability of instrument,observation condition,the response function,and so on.It has important effect to the spectra quality whether the splicing is normal or not.In the current work process,it is inefficient recognizing and pushing the abnormal spectra by eye checking one by one.In this paper,a method of automatic detection of splicing abnormality spectra for LAMOST is proposed by which we can improve work efficiency greatly.In this method,first of all,we get the red end and blue end of the test spectrum in the splicing wavelengths after flux normalized and the feature lines deleted.Then,we fit the continuum in the red and blue end separately.Thirdly,we calculate the differences of flux between the two fitted curves at a series of independent variables with regular intervals.We get the average and standard deviation of the differences and the area of the two curves formed.An evaluation function is presented in this paper on the basis of the average,standard deviation and the area got above which can be used to judge whether the test spectra are normal or not and determine their abnormal class.The method has been proved to have a good effect in the reorganization of splicing abnormality spectra through a mass of experiments.At the same time,we approach a method for abnormal splicing spectra reorganization and anomaly classification on high performance computing platform.The parallel processing method has greatly improved efficiency compared with the single machine.