Research On Multi-task Optimization Based On Multi-radial Basic Neural Network

At present,when using Bayesian optimization framework to solve expensive optimization problems(i.e.the problem that the fitness estimation cost is particularly high),the most commonly used surrogate model is Gaussian process model,mainly because it can provide estimation of fitness value(predicted average of objective function value)and estimation of uncertainty.(predicted variance of the objective function value).However,when the number of training samples increases,the amount of calculation of the covariance matrix in the Gaussian process becomes very large(O(N ~3),the N is the number of training data),and the calculation time for constructing the Gaussian process may become excessively long.On the other hand,by sharing information between different tasks and learning multiple tasks with correlations at the same time,each task can be avoided tabula rasa learning,and the related information between tasks can be used to promote better learning for each task.The study of multitasking is very meaningful.However,most existing multi-task learning methods add constraint items to the network's loss function to constrain the weights in the network,and use the constraint items to represent various possible relationships between tasks.The network structure of each task has not changed.By analyzing the existing multi-task learning network structure,this paper combines the networks of multiple tasks to learn.By adding the correlation layer in the network,proposed two new multi-task learning network structures.The main work of the thesis is as follows:(1)Study the Radial Basic Function(RBF)neural network,and apply it to Bayesian optimization framework according to its local approximation characteristics and fast learning speed,replace the commonly used Gaussian process model,thus avoiding the complicated covariance function calculation problem in Gaussian model.(2)This paper proposes a multi-task learning network model based on RBF neural network,and applies the multi-task RBF network model to the Bayesian optimization framework.Compared with the traditional multi-task learning network,our proposed network model includes relevance learning layer.Through the acquisition function,some new candidate points are selected according to different tasks in each iteration,and according to the data set characteristics of different models,the candidate points are selectively evaluated,and the data amount of the training level is rapidly expanded,which is very important for the training of the model.(3)Two multi-task learning networks are proposed for different applications.One is the multi-task learning network with single-input-multiple-output based on radial basic(SIMO-MT-RBF),the other is the multi-task learning network with multi-input-multiple-output based on radial basic function(MIMO-MT-RBF)model,where,for the MIMO-MT-RBF network model,we propose two different training methods.(4)Apply the proposed multitasking learning model to multiple benchmark optimization problems,comparing with the Bayesian optimization algorithm based on single-task learning model and the Bayesian optimization algorithm based on Gaussian process,in several cases,the experimental research shows that our proposed multi-task learning framework can achieve better performance.(5)Applying the proposed algorithm to the hyper-parameter optimization problem of complex neural network,and comparing it with the results of Bayesian optimization algorithm based on single-task learning.The experimental results show that the multi-task learning network model proposed in this paper can make multiple tasks share the information and learning the correlation between tasks,then finding a better hyperparameter group,thus improving the test accuracy of the neural network.