Structure Optimization Method For Stochastic Configuration Networks

Due to the rise of artificial intelligence,support vector machines,artificial neural networks(ANNs),decision tree and other machine learning technologies,have been widely applied to the fields of signal processing,pattern recognition,image processing and so on,especially ANNs with outstanding information processing ability.As an advanced random weight network(RWN)emerging in recent years,stochastic configuration networks(SCNs)have obvious advantages in network structure,learning efficiency,generalization performance and other aspects compared with the general RWNs.Random characteristic,however,is bound to generate low-quality hidden nodes,which increase network complexity and make little contribution to the model performance,even lead to over-fitting and instability.Therefore,based on the incremental structure method of SCNs,this paper studies the relationship between the network structure and the model performance.Using structure optimization method to improve the quality of model construction and make SCN more stable.The main work of this paper is summarized as follows:(1)An orthogonal stochastic configuration network(OSCN)is proposed.Due to the randomness of network,approximately linearly correlated nodes are inevitably generated in the process of stochastic configuration.These nodes have high redundancy and poor quality,which easily lead to ill-conditioned hidden output matrix affecting model generalization.To solve these problems,OSCN is proposed to optimize the network structure by introducing the Schmidt Orthogonalization technique.The universal approximation property of OSCN is proved and the output weight updating strategy is improved.The experiment results show that this proposed model can effectively reduce the iteration times of SCN,and have better generalization performance.(2)A stochastic configuration network based on stochastic sensitivity measure(SSM-SCN)is proposed.Due to unknown invisible samples are inevitable in modeling tasks,which greatly affect the generalization performance of the model.Moreover,the more complex the structure of SCN model is,the more sensitive to invisible samples.The generalization performace becomes worse and worse under this situation.To solve these problems,SSM-SCN is proposed by introducing stochastic sensitivity measure(SSM)into the processing of hidden nodes configuration and parameter optimization to enhance the quality of SCN construction and realize the structure optimization.The convergence of SSM-SCN with updated supervision mechanism is also proved.In addition,the output weight updating strategy of Cholesky factorization is introduced into SSM-SCN to improve learning efficiency.The experiment results show that this proposed model can help build a network structure with better generalization performance and raise the modeling efficiency.(3)The two improved SCN algorithms proposed in this paper are applied to an example of grinding industry.Through experimental research on industrial data of different scales,the results show that the two improved algorithms can meet different industrial needs.In summary,this paper focuses on the relationship between network structure and generalization performance of the SCNs to enhance the model generalization ability and proposes two optimized SCNs.Moreover,it helps select appropriate algorithms for different data sizes.The developed algorithms effectively improve the learning and generalization ability of the SCNs in practical application.