Network Structure Improvement Of Broad Learning System Based On Regularization Technology

Broad learning system is a new kind of neural network with simple structure,which is composed of feature layer,enhancement layer and output layer.Compared with the deep network,its structure expands the width.Broad learning system is widely used in many fields because of its simple structure,high computational efficiency and many kinds of incremental learning algorithms to update the model quickly.However,there are still some shortcomings in broad learning system,which cannot meet the higher demands of the actual tasks.Firstly,the structure of broad learning system is often redundant in the training process,which makes the model too complex and affects its generalization.Secondly,in the process of simplifying the broad network structure,only the correlation between nodes and outputs is considered,and the correlation between nodes is ignored,which leads to insufficient modeling information acquisition.Finally,when dealing with the multi-output regression problem,the original broad network cannot capture the correlation between input-output and multiple outputs at the same time,resulting in the inability to predict multiple output targets at the same time.In view of these problems,this paper carries on the research and proposes the corresponding solutions.The specific work is as follows:（1）Aiming at the problem of redundancy of network structure in broad learning system,this paper proposes a sparsity method for network structure of broad learning system based on lasso and elastic net.This method introduces two kinds of regularization technology based on broad learning system,including lasso and elastic net.It constrains the output weight of nodes and captures the correlation between each node and the output response in order to measure the importance of each node to the output prediction.Then,it retains the important nodes and removes the unimportant or even negative nodes from the network.Finally,a more sparse network structure is obtained.For the new objective function,this paper gives the optimization method.Finally,several regression datasets are used to verify the effectiveness of the proposed method.（2）To solve the problem that only the correlation between nodes and output response is considered and the correlation between nodes is ignored in the network simplification process,this paper proposes pruned broad learning system based on sparse ridge fusion.Sparse ridge fusion is introduced into the broad learning system,where sparse ridge fusion includes ₁L-norm and second-order fusion penalty terms.The ₁L-norm regularization term is responsible for capturing the correlation between nodes and output response,eliminating unimportant nodes and reducing the redundancy of network structure.The second-order fusion penalty term is responsible for constraining the square of the weight difference between the adjacent outputs,so as to capture the correlation between nodes,making nodes with high correlation automatically cluster into a group,and the weight of each node within the group presents smooth changes.The node groups can not only enrich the information of feature selection,but also make the selection of nodes more interpretative.For the newly constructed objective function,this paper designs the optimal solution method by using the alternative direction multiplier method.Finally,several common datasets are used to verify the effectiveness of the proposed method.（3）Aiming at the problem that broad learning system cannot handle the multi-output regression,this paper proposes multi-output broad learning system based on Frobenius and L_2,1 norm.In this method,the Frobenius and L_{2 1,}two matrix norms are introduced to capture the correlation between the input features and each output by using the advantages of matrix norm.Meanwhile,the mutual information between multiple outputs is explored to jointly model the input-output and the relationship between each output through this information,so as to predict multiple output targets simultaneously.In addition,in order to suppress the interference of outliers or noise to the modeling,this paper replaces the ₂L loss function in the standard broad learning system with the L_{2 1,}loss function,then constructs a new objective function,and carries out optimization solution to it.Finally,the proposed method is applied to several public multi-output datasets and a real industrial dataset,and the experimental results verify the effectiveness of the proposed method.