Incremental Random Weight Neural Networks Learning Method With LUPI Paradigm

The traditional machine learning paradigm generally considers an example of finding an optimal way to approximate an unknown decision rule in a given set of functions.However,this learning paradigm doesn’t consider the influence of some potential factors on the learner model.Learning using privileged information(LUPI),which considers about the guiding role of teachers in the human teaching model,applies teacher-student interaction mechanism to machine learning.It uses some potential hidden information related to the machine learning model as privileged information to guide the model training,in order to increase the diversity of training data and improve the model generalization.However,the existing machine learning algorithms combined with the LUPI paradigm are all batch algorithms,which has the biggest problem that it is difficult to determine the optimal number of network nodes for different tasks.The optimal solution is usually found by trial and error,which consumes a lot of time and computing resources.Incremental learning is different from batch learning,which starts from the minimum structure of the network and gradually constructs network nodes until the model reaches an acceptable solution.This learning method solves the problem that traditional batch algorithms are difficult to determine the size of network nodes.However,even with incremental learning,the performance of the model may be related to some potentially privileged information in the process of constructing the network.This paper mainly studies the introduction of privileged information in incremental random weight neural network to improve the performance of the network and fill the gap of LUPI paradigm in incremental learning.The main work and innovations are as follows:(1)IRVFL+ learning method is proposed,which introduces LUPI paradigm into incremental random vector functional link(IRVFL)network.It introduces privileged information during incremental learning and two algorithm implementations of IRVFL+ are proposed for how to solve the output weights: local update and global update strategies.IRVFL-I+ is an algorithm based on the local construction of the original IRVFL,and IRVFL-II+ is an algorithm that globally updates the output weights considering that the original local construction method fails to consider the problem that the new node may lead to the change of the optimal solution of the model.Finally,two kinds of convergence of the two algorithms are analyzed theoretically and the two algorithms are verified to be better than the traditional IRVFL on the classification and regression data sets.This novel learning paradigm can add privileged information to guide the learning in the incremental learning process of the IRVFL network,which improves the generalization performance of the model.(2)SCN+ learning method is proposed,which introduces LUPI paradigm into stochastic configuration network(SCN).SCN uses the supervision constraints on the hidden layer nodes,which can effectively improve the convergence speed of the learning process,ensures that the parameters selected by each new hidden layer node strictly make the model converge and contribute the most to the drop error.SCN+introduces the LUPI learning paradigm,which not only solves the problem of slow convergence of the IRVFL+ model due to its strong randomness,but also extends SCN from the traditional machine learning mode to the learning mode using privileged information.This thesis analyzes the learning convergence of SCN+ and proves the universal approximation of SCN+.Finally,the comprehensive performances of SCN+ on classification and regression tasks are verified by experiments to be better than IRVFL+.SCN+ has both the fast and efficient characteristics of SCN and the superior generalization of LUPI and its performance are significantly improved compared to IRVFL+.(3)The proposed SCN+ is applied to photovoltaic(PV)power generation forecasting,which is able to accomplish the task of PV power generation forecasting brilliantly and efficiently.Existing PV prediction models can only use the directly collected real-time operating index data of PV power generation for modeling and cannot use some privileged information related to PV power generation.The SCN+proposed in this thesis can use privileged information modeling and has superior performance,which is very suitable for the needs of PV power generation forecasting.Finally,it is verified by simulation experiments that SCN+ can effectively complete the task of PV power generation prediction.