Research On High Compression Ratio For The Data From The On-board Fourier Transform Spectrometer

The FY-4 launched on 11 th December of 2016 leads the meteorological detection of China,even of the world to a new era.The Geostationary Interferometric Infrared Sounder(GIIRS)on FY-4,as the first Fourier Transform Spectrometer works on the geostationary orbit,can offer the spectral information respectively range from 685cm-1 to 1130cm-1(LW,Long Wave)and range from 1685cm-1 to 2250cm-1(MW,Medium Wave),with the 0.625cm-1 spectral resolution.The data collected can be used to further predict the vertical distribution of the atmospheric temperature?humidity and the water vapor profiles for the more precisely weather forecasting.The data gathered by Geostationary Interferometric Infrared Sounder(GIIRS)is a datacube of three dimensions,including two dimensions for space and one dimension for interferogram.The data transfer rate now is about 66Mb/s at the current scale of the FPA(Focal Plane Array).The amount of the data will correspondingly multiply while the size of the FPA needs further expansion to meet the continuous developing demand,resulting in the heavier pressure on the bandwidth.Besides,the data directly generated is interference data,which has to be performed Fourier Transform on to achieve spectral information for further study.As no-direct data,a small distortion of it may finally cause big errors on the analysis,so the accuracy of the interference data is undoubtedly of significant importance.The dissertation concisely introduces some advanced Fourier Transform Spectrometers that are already operating in space or still under research,along with their parameters.Their schemes to reduce the amount of the data are attached attention as well.This dissertation mainly concentrates on compression algorithms to achieve the goal to decrease the data.Firstly,the data compression problem can be viewed from the perspective of probability in mathematics.After briefly introducing the math knowledge connecting with the compression,the substance of the lossless compression and lossy compression are explained.Additionally,the principal steps and their own purposes of a complete compression system are simply described.The scheme design finally decides to primarily aim at interferogram dimension based on the entropy and correlation calculation in the space dimension and the third dimension of both the interference datacube and the spectral datacube after performing FFT.Moreover,it mainly concentrates on lossless and near-lossless compression algorithms with almost ignorable distortion due to the requirement for the data's accuracy.In terms of lossless compression,the scheme that segment the interferogram into two parts with different features and independently operate Vector Quantization(VQ)and Linear Prediction(LP)was proposed.The parameters of the Vector Quantization part and the Linear Prediction part can be respectively determined by Matlab simulation that takes both the balance of the relatively high compression ratio and low resources consumption into consideration.The biggest difference is the prediction times,which divides the scheme into once prediction and every time prediction.The 4.2 GHz CPU realizes the compression scheme based on the once prediction within 0.405 s,while FPGA costs 0.7942 s.Under this situation,a MW datacube can achieve the highest compression ratios range from 3.5 to 4.When it comes to a LW datacube,its compression ratios are obviously better than MW but it can still get higher with more resources to realize the every time prediction,which will use about 0.753 s by CPU and 1.2504 s by FPGA,to make the compression ratios rise to 4.7~5.4.After analyzing the relationship between the noise and the compression ratio,there is a conclusion that apart from increasing the resources,reduce the noise of the data to be compressed can also result in higher compression ratios.When it comes to the near-lossless compression,based on the definition of near-lossless compression: the errors are required to be less than the noise of the original data.Since the final required data is not the direct interference data,but the spectral data after Fourier transform,the parameters of noise estimation and distortion should occur in the spectral data.Estimating the noise by std and standardizing the three parameters of mean-square spectral error(MSSE),spectral correlation(SC)and spectral correlation(SC),which can be used as a criterion for judging whether the near-lossless compression is eligible.The near-lossless compression scheme principally consists of simple decimation,quantization,and entropy coding.The qualified scheme can achieve a compression ratio of 7.80~23.72 for the long-wave data,and a compression ratio of 6.21~13.15 for mid-wave data.The experimental results prove that the proposed schemes are available and can reduce the data volume while ensuring the accuracy of the data.The research of this dissertation can be a good reference for future study.