The Group Norm Regularized Method Of Low-rank Analysis:theory And Financial Application

With the advent of the era of big data,big data has been widely used in many industries,such as Driverless Technology,Voice Recognition,Smart Healthcare,Intelligent Transporta-tion,Crime Prediction,Fintech and so on.At present,the major applications of big data in the financial industry include quantitative investment,corporate rating,and consumption recommendations.Data in the financial industry are characterized by structured data,huge data volume,high data dimension and noise,etc.Therefore,How to efficiently collect,store,process and analyze data in the financial industry is an important research topic.High-dimensional data processing often needs to face the "dimension disaster" problem,but most of the data is redundant and sparse.From the perspective of economics,most cus-tomers are similar;Commodities can substitute for or depend on each other;Many stocks are of the same type;Past price movements are likely to be repeated in the future;The ran-domness behind thousands of stocks may be driven by only a few dozen Brownian motions.These economic views are reflected in mathematics as important low-rank properties.Low rank analysis is a powerful tool to study and process big data.It can solve the dilemma of high dimension by studying low dimension.The traditional low rank analysis method has some problems,such as high computational complexity,need to design additional rank finding strategy,and the model is not robust.In this paper,a group norm regularized method for low rank analysis is presented.This paper first summarizes several types of low-rank problems in the financial background.In the third chapter,the framework for solving general low-rank analysis problems is given and relevant theoretical analysis is carried out.Then,the corresponding model and solving algorithm are designed for specific low-rank application problems in finance,and numerical experiments are conducted on synthetic data and real data.The specific results are as follows:First,we present the Group Norm Regularized Factorization Model(GNRFM)for ma-trix completion problem,and designs the corresponding algorithm:Accelerated Augmented Lagrange Multiplier Method(AALM).We carried out numerical experiments on synthetic data,financial recommendation system and image filling problem.Compared with tradi-tional methods,our model and algorithm have the advantages of wide adaptability,good anti-noise performance and prevention of overfitting.Second,the group norm regularized model GNRFM and corresponding AALM algorith-m for low rank representation problem is studied.Compared with the matrix completion problem,the low-rank representation problem has high computational complexity and the variables are difficult to solve.Therefore,we use the group norm regularized model with coupling variables to handle this problem.Compared with the traditional algorithm and model,our algorithm and model have faster computation speed,higher clustering accuracy and better anti-noise performance.Especially for the problem of stock clustering,we can di-vide stocks into its industry,find out the abnormal stocks,and give investors good investment advices.Thirdly,this thesis designs a factorization model and coordinate projection descentmethod for the low rank approximation correlation coefficient matrix in high-dimensional option pricing.Compared with traditional algorithms,our algorithm performs well in high-dimensional problems with fast calculation speed and high accuracy,and is suitable for high-dimensional Monte Carlo simulation.