Study On Optimization Models And Methods For Collaborative Learning

Collaborative learning is a class of statistical optimization methods to explore relationships between multi-view(modal)data and perform predictive analysis.It is widely used in education and cognitive science,medical research,image recognition,bioinformatics,financial risk management,meteorological environmental science,and it has become one of the hot research topics in the field of statistical optimization in the past ten years.The current research on multi-view(modal)learning is mainly divided into two parts: predictive analysis and exploration of relationships between data,i.e.,collaborative learning using multiple data sets for prediction and some statistical methods such as canonical correlation analysis(CCA)for exploring relationships between data sets,respectively.In this paper,we focus on the theory and algorithms of collaborative learning models to explore the relationships between data sets while using multiple data sets for prediction.We first investigate robust collaborative regression models with regular terms based on existing collaborative regression models,and then consider a general collaborative learning model framework with double sparsity constraints that can handle both discrete and continuous response variables,and extend it to multi-block sparse collaborative learning models.A matrix collaborative regression model is proposed for the multi-responses matrix data.We theoretically analyze the above models and design efficient algorithms.Numerical experiments show the effectiveness of the models and algorithms.First,a robust collaborative regression model based on the least absolute deviation is considered.The least-squares regression-based collaborative regression and robust collaborative regression are theoretically analyzed and their dual forms and statistical interpretations from Bayesian perspective are given.Considering the high-dimensional case,an efficient symmetric Gauss-Seidel based alternating direction method of multipliers(s GS-ADMM)algorithm with global convergence and Q-linear convergence rate is designed for their dual problem.Numerical experiments show that the robust collaborative regression is much better when the errors the error term is a heavy-tailed distribution or asymmetric distribution.Second,a sparse collaborative learning framework(SCL)with double sparse constraints is proposed to deal with problems with two sets of data and a shared response variable.It can handle classification problems with discrete response variables or continuous regression problems,as well as explore the relationship between the two data sets simultaneously.First,we analyze the properties of SCL,and propose the sufficient and necessary optimality conditions for sparse constrained collaborative learning.Based on the above optimality conditions,we design a gradient projected Newton algorithm,which globally converges to a unique local optimal solution with quadratic convergence rate or finite step termination.We generalize the two-block sparse cooperative learning model to the case of multiple blocks,and analyze three types of stationary points conditions for multi-block sparse constrained collaborative learning,and explore the relationship between the stationary points and the minimal value points.Finally,a large number of numerical experiments illustrate the excellent performance of the proposed method.Finally,a sparse matrix collaborative regression model is proposed for data in matrix form with multiple responses.We analyze the sparse matrix collaborative regression model in terms of canonical correlation analysis(CCA)and design a globally convergent symmetric Gauss-Seidel based alternating direction method of multipliers(s GS-ADMM)algorithm,in which we give a closed-form solution for each subproblem generated by the s GS-ADMM algorithm.In particular,we apply sparse matrix collaborative regression to the fault detection problem for the first time.Experimental results on simulated and real fault detection datasets show that the sparse matrix collaborative regression has higher detection accuracy and lower false alarm rate than traditional fault detection methods.