Research On Particle Swarm Optimization Algorithm Based On Neural Network And Reinforcement Learning

As a representative heuristic algorithm,swarm intelligence(SI)algorithm is constructed by simulating the behaviors of real-life biological populations in the natural world and their self-organizing interaction.As a typical SI algorithm,particle swarm optimization(PSO)algorithm has attracted extensive attention since proposed,and many PSO variants have been developed.However,there are still some problems in current particle swarm optimization algorithm.For example,PSO mostly depends on the priori knowledge and lacks search diversity.And it’s hard for PSO to obtain the global optima in the face of complex optimization problems.In view of the above problems,this paper introduces neural network and reinforcement learning to guide the search of particles,so as to improve the optimization performance.The main research contents in this paper are given as:1.The parameter settings of particle swarm optimization algorithm mostly depend on human experience(via trial and error),so it is difficult to obtain appropriate parameters.this paper proposes a particle swarm optimization guided by neural network(PSO-NN).PSO-NN designs a neural network as a guidance to adjust the acceleration coefficient of each individual separately based on its performance,rather than making all the particles in the population follow the same search manner.Then reinforcement learning is introduced to update the network parameters.Specifically,PSO-NN takes the historical experience of each particle,that is,the change of fitness value in a period of time,as the input and outputs the corresponding adjustment action for the acceleration coefficient of each particle.Then the network is trained with the feedback obtained from the change of particle fitness value.PSO-NN is able to achieve an effective setting of parameters automatically.Through the experimental results on all 28 functions of CEC2013 test suite,it can be seen that PSO-NN obtains good performance in the search performance.2.Faced with the problem of lacking learning diversity due to the simple global learning target in PSO.This paper proposes a particle swarm optimization based on role division and neural network(PSO-RDNN).In PSO-RDNN,the particles in the population will be divided into three roles,including leader,follower and rambler,according to the fitness value.After that,particles will follow different learning behaviors according to their different roles,that is,they will learn from different targets.Moreover,PSO-RDNN designs a neural network to adjust the parameters for each particle,and a neural network will be trained for the particles of the same role,which maps the historical performance of particles with similar fitness values to the parameter adjustment actions.PSO-NN designs a role transformation method for particles to realize the re-division of particle roles,and proposes a candidate particle mechanism to assist the role transformation process of particles.Experimental results on all 30 functions of CEC2017 test suite show that PSO-NN outperforms other competitors in terms of optimization accuracy.3.To cope with dynamic optimization problems,this paper proposes a behavior decision neural network-based particle swarm optimization(SBDNN-PSO),which aims to make behavior decisions for the two search actions of each individual in the population including the localization of the learning target and adjustment of the acceleration coefficient while maintaining the swarm diversity.In order to maintain swarm diversity in the whole search process,firstly,a subswarm division method is designed.For each input individual,the subswarm it belongs to is located and the particle that can represent the search characteristics of the subswarm are selected as the subswarm center,and each subswarm center is distant from each other.The designed subswarm division method realizes the division of whole population by selecting the appropriate subswarm center.Particles in different subswarms search under the guidance of their respective subswarm centers,showing various search characteristics and improving the swarm diversity.The input of SBDNN is the current position of each particle in the population,and then the hidden node is set by the obtained subswarm center.In order to adjust the acceleration coefficient of particles,the adjustment action of the current input particles is output in the SBDNN output layer.Finally,in order to train SBDNN,reinforcement learning is also introduced to reward or punish the actions determined by the network(including locating the learning target and adjusting the acceleration coefficient),so as to effectively adjust the hidden nodes(representing the learning target of the particles in the subswarm)and the attached weight to the action node(controlling the acceleration coefficient of the particles),so that SBDNN-PSO can appropriately guide the search action of particles to better solve the DOPs.Through the experimental results on the various MPB(a widely used dynamic benchmark function)instances,it can be seen that SBDNN-PSO can better solve DOPs.