Recursive time- and order-update algorithms for radial basis function networks

Scope and method of study. This research develops online learning schemes that can construct small and parsimonious RBF networks. We have shown that this goal can be achieved by using the time- and order-update framework developed in this research. This framework is adopted by combining three schemes: time-update, order-update, and subset selection. Using this framework, two new algorithms, the Recursive Least Squares with Automatic Weight Selection (RLS-AWS) algorithm and the QR Recursive Least Squares with Automatic Weight Selection (QR-RLS-AWS), have been developed. Both algorithms are recursive in time and order. We first developed the subset selection mechanism of the algorithms based on the forward selection method. This technique allows useful RBF nodes to be added into the network sub-optimally and recursively. Later, we developed an improved subset selection mechanism based on the Efroymson method. This method has the capability of removing insignificant RBF nodes in addition to adding useful RBF nodes. Both recursive subset selection methods are new.; Findings and conclusions. The QR-RLS-AWS algorithm is numerical more accurate than the RLS-AWS algorithm. However, if numerical ill conditioning is not a problem, both algorithms yield the same solution. Both subset selection schemes, the recursive forward selection method and the recursive Efroymson method, have been adopted successfully. The results have shown that the recursive Efroymson method can produce smaller RBF network than the recursive forward selection method and the batch forward selection method. In conclusion, this research has successfully designed and implemented the recursive time- and order-update algorithms for online learning. Although the work described in this research has focused on small RBF networks, the algorithms can be applied to all linear models and all nonlinear models that have a linear-in-parameters structure, such as the fuzzy basis function network, functional-link network, polynomial network, and more.