Extensional Study On Factorization Machine And Its Application

Factorization Machine(FM)is a recently proposed algorithm that is mainly used to solve the problems of feature combination when dealing with large-scale sparse data.It is a second-order polynomial model essentially.Thanks to the decomposition of interactions,FM can learn the relationship between variables from the sparse data very well.This paper first extend the FM into Ordinal Regression areas and then propose the sparsified FM and sparsified HOFM,the concrete achievement is as follows:1)Ordinal regression(OR)is a special paradigm of machine learning.When the class labels are ordered,OR can make good use of ordered prior information for classification.At present,many methods have been proposed to solve the problem of ordinal regression.However,current methods seldom consider the inherent structure in the input data,thus losing interpretability to some extent.We propose a new OR method: Ordinal Factorization Machine with Hierarchical Sparsity(OFMHS).OFMHS combines the factorization machine with the hierarchical sparsity together to explore the hierarchical structure between variables in OR tasks.Experiments show that the proposed OFMHS not only achieves the comparable accuracy with the state-of-the-art methods,but also can recover the hierarchical structure among the features,which enhance the model's interpretability.2)Though FM has been applied in recommended systems,it fails to consider the sparsity of variables explicitly.For such a second order polynomial model,the process of feature selection should meet the following requirements: the linear terms and second-order terms that share the same feature should be included or ex-cluded at the same time;when the feature is noise,both should be excluded,otherwise,both should be included.Based on the sparse structure described above,this chapter proposes a Sparse Group Lasso-based Factorization Machine(SGL-FM).By adding Sparse Group Lasso to the loss function,SGL-FM not only achieves sparsity between groups but also within groups.From another point of view,sparsity within groups can be seen as a method of controlling the dimensions of the factorization;therefore,SGL-FM chooses the best k automatically when faced with datasets with different properties.Experimental results demonstrate the superior of our method.3)FM can only model the second order relationships between variables.HOFM is a higher order version of FM which can model higher order interactions.HOFM should have the same structural sparsity as FM: Linear and high-order terms for the same feature are simultaneously discarded or simultaneously selected.Based on this,this chapter proposes a method of sparsified higher-order factorization machine and verifies the effectiveness of the proposed method on the recommended system datasets.