Hyperparameter Optimization Method Based On Improved DQN Algorithm With Its Application In Fault Diagnosis

With the development of intelligent manufacturing,the fault diagnosis method based on convolution neural network(CNN)has become a research hotspot.As one of the most important super-parameters of deep learning,learning rate has a great impact on the final performance of the CNN model.The traditional method of adjusting the learning rate is time-consuming and over-dependent on the experience of experts.Because of the fixed change law,the existing learning rate optimization algorithms are difficult to guarantee their generalization performance between different data sets.In this paper,a new learning rate scheduler based on deep reinforcement learning(DRL)is proposed,which is used to deal with the CNN model of fault diagnosis and verified in an engineering case.First of all,aiming at the problems existing in the original depth Q network(DQN),three improvements are made.Based on the idea of double network,the maximization operation is divided into action selection and action evaluation through target network and prediction network.The competitive network architecture is introduced to decompose the optimal action value function into the optimal state value function and the optimal dominance function to learn the state value function more effectively.Priority experience playback is introduced to distinguish the samples that are meaningful and meaningless to the training process,so as to improve the learning efficiency of reinforcement learning(RL).Tested in the simulation environment on Open AIGYM,the results show that compared with the original DQN,the improved DQN can converge faster and show higher stability.Secondly,aiming at the CNN classification model,a learning rate optimization method based on improved DQN(PERD3QN-CNN)is proposed.The environment,state space,action space and reward function of the learning rate scheduler based on DRL are defined.The state space of CNN is represented by feature vectors with six state features,and five action spaces are defined to adjust the learning rate without any predefined boundary constraints.Finally,the model is used to control the learning rate.Compared with other learning rate attenuation strategies and adaptive gradient descent algorithms on MNIST and CIFAR-10 image recognition data sets,the results show that the proposed method converges faster and the recognition effect is better.Then,aiming at the case of case Western Reserve University bearing data set,a fault diagnosis method based on PERD3QN-CNN is proposed.The results show that PERD3QN-CNN can automatically control the learning rate within a reasonable boundary.In some cases,PERD3QN-CNN has also discovered state-of-the-art artificially designed learning rate schedulers.Compared with the traditional deep learning and machine learning methods,the results show that the result of PERD3QN-CNN is the best.Finally,the work of this paper is summarized and prospected.