Research Of Correlation Measurement Theory On High-dimensional Sparse Data

This paper is mainly focus on the theory and method of correlation measurement ofsparse data. The research starts from fundamental theory by expanding the conception ofStatistical Framework. It converts a normal sample space into a sparse sample space bydefining a Visual Variable. And, lead to a Sparse Statistical Framework through definingthe σ-algebra and probability measure on sparse sample space. Therefore, that gives out andetailed explanation of sparse data theoretically. Moreover, it summarizes and gives outseveral methods and thoughts of correlation measurements of sparse data in empiricalworks. Such as introduce entropy into the correlation measurement, and measurescorrelation by a standardized entropy value. Furthermore, this paper presents an idea ofsparse correlation path, that exploring an solution of indirect correlation. In the last, thepaper designed a collaborative filtering algorithm as an experiment platform and collectedan experiment dataset. Based on these, making recommendation on sparse data bymissing-value interpolation. Hoping to make an comparison of several correlationmeasurements by empirical study.