Research On Optimization Method Of Non-negative Latent Factor Analysis On High-dimensional And Sparse Matrix

With the rapid development of information science and technology,the number of entities involved in a big data-related Internet-application-system is quite huge,while most of the relationships among them are not yet observed.Commonly,such data are modeled into a high-dimensional sparse(Hi DS)network.The Hi DS matrix is an important data structure that can effectively describe entities and their relationships in such a Hi DS network.The data organized in an Hi DS matrix reflect people's really historic behavior.Hence,they contain plenty of knowledge,and are of high significance for data mining tasks.A Non-negative latent factor(NLF)model can efficiently acquire useful knowledge from such Hi DS matrices filled with non-negative data,which is one of the most promising non-negative latent factor analysis(LFA)approaches.A Single Latent Factor-dependent,Non-negative and Multiplicative Update(SLF-NMU)algorithm is frequently adopted to construct an NLF model on an Hi DS matrix.It is highly efficient in terms of its computation and storage,yet it suffers slow convergence.And this is because that an SLF-NMU algorithm is essentially derived from an additive gradient descent(AGD)method by manipulating learning rates to cancel negative components to satisfy the non-negative constraint.Focusing on above problems,this paper proposes serval novel methods to optimize a basic SLF-NMU algorithm's convergence rate as well as its prediction accuracy for missing data in an Hi DS matrix.The main contributions of this work include the following.(1)By adjusting the learning rate implicitly,we propose to control the learning depth of an SLF-NMU algorithm in linear and nonlinear ways,respectively.In this study,we first deduce the update rules of such schemes,and then prove their convergence properties by rigorously mathematical analysis.Subsequently,we empirically validate that changing the learning depth of an SLF-NMU algorithm results in significant impaction of an NLF model's performance.(2)A generalized momentum method is proposed to construct a fast NLF(FNLF)model.According to prior studies,a GD-based algorithm's convergence rate can be significantly improved by a classical momentum(CM)method whose update rules are depended on gradient term explicitly.However,an SLF-NMU algorithm trains non-negative LFs multiplicatively for keeping their non-negativity.It depends on gradients implicitly.To make it is compatible with a standard momentum method,we propose a generalized momentum method,and then apply it into the construction of an NLF model by replacing gradient term with the increment of involved decision parameters.Based on the proposed method,we further propose a novel momentumincorporated SLF-NMU algorithm(SLF-NM2U)to build an FNLF model.Empirical studies show that the convergence rate can be significantly improved by such a generalized momentum method.Meanwhile,the prediction accuracy of an NLF model for missing data in a target Hi DS matrix is also improved.(3)Based on the Nesterov's fast gradient method(FGM),we present another fast NLF model,called NNLF.Although an FGM is similar to the Momentum method,it possesses the ability of acceleration faster than the later in theory.During the training process,a new state of involved LFs is obtained by updating them according to the current update velocity,and then a gradient descent algorithm is performed relaying on such a new state to correct the update,which accumulates the gradient of the previous state to achieve faster acceleration.In this work,we also deduce the update rules for an NNLF model,and based on it we further design an adaptive adjustment scheme to set hyper-parameter of momentum term.Empirical studies on large-scale datasets show that the availability of acceleration of our proposed method is very significant.Furthermore,it can effectively improve the accuracy of LFA tasks.