Research On Some Problems Of Incremental Twin Support Vector Regression

In recent years,Twin Support Vector Regression(TSVR)has become a hot topic in machine learning due to its advantages such as short training time and small fitting error.However,at present,most of existing TSVR and its variants only have off-line training algorithms.There is a lack of in-depth studies on how to inherit or improve generalization performance,accelerate solution speed,and improve model sparsity under incremental scenarios.This thesis mainly focuses on how to improve the training efficiency of TSVR and its variants under incremental scenarios,and design and analyze their corresponding incremental learning algorithms.The main contributions are summarized as follows.In order to address the problem that the existing Lagrange asymmetrical ν-twin support vector regression cannot efficiently update the model under incremental scenarios,an incremental reduced Lagrangian asymmetric ν-twin support vector regression was proposed.First,the constrained optimization problems are transformed into unconstrained ones by introducing the plus functions,and the semi-smooth Newton method is utilized to directly solve them in the primal space to accelerate the convergence speed.Then,the matrix inverse lemma is adopted to realize efficient incremental update of the Hessian matrix inversion in the semi-smooth Newton method and save time.Next,to reduce the memory cost caused by the sample accumulation,the column and row vectors of the augmented kernel matrix are filtered by the reduced technology to approximate the original augmented kernel matrix,and this ensures the sparsity of the solution.Finally,the feasibility and efficacy of the proposed algorithm are validated in the benchmark datasets.The results show that compared with some state-of-the-art algorithms,the proposed algorithm inherits the generalization performance of the offline algorithm and can obtain sparse solution,which is more suitable for online learning of large-scale datasets.In order to solve the problem that the existing pairing support vector regression cannot efficiently deal with learning under incremental scenarios,an accurate incremental pairing support vector regression was designed.First,by preprocessing the label set,all the labels in the training set are taken as positive numbers.Then,to reduce the number of support vectors as much as possible,the label set of the input sample is divided into two independent new label sets to tune the distribution of the samples.Secondly,the first sample point is set as a support vector to obtain an effective initial state for iteration.Finally,to ensure that the original samples still meet the Karush-Kuhn-Tucker condition,based on the method of accurate online support vector regression proposed by Ma et al.,the Lagrangian multiplier and boundary distance function of the abnormal samples are calculated and corrected.The results demonstrate that the designed algorithm can obtain the exact solution and has great advantages in shortening the training time of large-scale datasets.In order to solve the problems of slow training speed and large memory consumption of existing accurate incremental support vector regression and incremental support vector regression combined with reduction technique on large-scale data sets,a twin support vector regression based on enhanced self-organizing incremental neural networks was exploited.First,the stream data is fed into the enhanced self-organizing incremental neural network,and the original input stream is simplified into small sample prototype data which can reflect the distribution characteristics of the original data.Then,the prototype data is fed into twin support vector regression for training,and the obtained support vector set and the next batch of sub stream data are fed into the enhanced self-organizing incremental neural network for training until no more data arrives.The results reveal that compared with representative algorithms,the exploited algorithm can obtain sparse solutions and better generalization performance,and can greatly shorten the training time of large-scale datasets under incremental scenarios.