Research On Unbalanced Classification Method Based On Reinforcement Learning

In application problems such as disease diagnosis,spam detection,and software defect detection,the number of normal and abnormal samples in the dataset is often unbalanced due to their low frequency and small overall number of abnormal samples.Deep reinforcement learning algorithms learn strategies for solving specific tasks by interacting directly with a given environment and adjusting the next actions through feedback from the environment.By constructing different classification environments and reward functions,deep reinforcement learning algorithms can be applied to different types of data with different degrees of imbalance.Therefore,in order to meet the practical application requirements,this paper conducts research on the imbalance classification methods based on reinforcement learning at the data level and at the model level,respectively.The main research contents and innovative work of this paper are as follows:(1)In the unbalanced binary classification problem,the quality of generated samples from different up-sampling algorithms varies,some of the poor quality generated samples will force the distribution of the minority class to deviate from the true distribution,reduce the classification performance of the model.To solve this problem,this paper proposed an unbalanced samples classification model combining transfer learning and reinforcement learning.Firstly,the model generated samples for minority class based on the up-sampling method.The generated sample set was regarded as the source domain,and the original unbalanced data set was regarded as the target domain.And calculated the contribution degree of each sample in the source domain on the unbalanced classification problem to obtain the prior knowledge of the source domain.Secondly,established a reinforcement learning agent.The agent selected the generated samples with prior knowledge and selected the samples with large contribution into the training set.In the newly established training set,the inferior generated samples were deleted by introducing the generated samples after agent selection for minority classes,which not only amplified the amount of data,but also improved the category representation of minority classes.Experimental results show that the proposed method has good generalization ability and can effectively improve the performance of the classifier on unbalanced classification problems.In addition,based on the transmission line fault data collected in actual working conditions,the application value of this method in engineering is verified.(2)The existing imbalance classification algorithm DQNImb based on reinforcement learning did not consider the imbalance in the experience pool,which leaded to the experience samples obeying the imbalance distribution when the experience replay strategy was used to optimize the network,and the classification effect needed to be improved.To address this problem,this paper proposed an imbalanced classification model IDQNImb based on improved reinforcement learning.Firstly,constructed a Markov decision process for the imbalanced binary classification problem.On the one hand,set different reward functions for majority class samples and minority class samples,a larger reward and penalty value was set for the minority class samples,and the current round of classification training was ended when the agent classified minority class samples incorrectly,which improved the sensitivity of the model to minority class samples;on the other hand,constructed the experience pools of majority class samples and minority class samples separately to ensure a better balance of the extracted batch experience samples and improved the efficiency of model parameter optimization.The experimental results show that the proposed model can improve the classification effect on different types of data sets with different imbalance degrees,which indicates that the proposed model has better generalization ability and stronger robustness.