The Imbalance Problem In Object Detection Based On Deep Learning

Object detection is a classic problem in computer vision with many important ap-plications such as autonomous driving,surveillance and so on.The objective of object detection is to locate and classify objects of interest in an image or video sequence.Due to the great success of deep learning in the field of object detection,various object detec-tion algorithms based on deep learning continue to emerge,which has greatly promoted the development of this field.Currently,the mainstream object detection algorithms are based on deep learning technology.However,the data used to train deep learning mod-els often faces the class imbalance problem,namely the number of samples belong to some categories is much larger than the number of samples belong to other categories.These imbalanced training data will significantly degrade the performance of deep learn-ing models.In the object detection task,the proportion of background far outweigh that of the foreground on the entire image(between-class imbalance)? At the same time,the struc-ture of the deep learning model itself will lead to samples with different complexity.The number of easy samples is often larger than the number of complex samples(within-class imbalance)? In addition,the object detection task actually contains two sub-tasks,namely localization and classification.These two sub-tasks correspond to different loss functions.The contribution of the task's loss function to the total loss is different,and the loss of the classification task is often greater than the loss of localization(objective imbalance).The existence of these imbalance problems can significantly decrease the performance of ob-ject detectors based on deep learning.At present,some methods have been proposed to solve the class imbalance problems,but these methods either require considerable addi-tional memory and computation,and are difficult to train,or there is essentially no gain on the two-stage detector.Therefore,it is a very valuable and challenging work to study and solve the imbalance problem in object detection based on deep learning.This article proposes appropriate solutions from the data and algorithm levels,re-spectively.The main work and innovations are as follows:1.Design new data augmentation methods in detection scenarios: object detectors based on deep learning are more likely to overfit when facing with the imbalanced data,so this paper use small blocks and a universal adversarial perturbation for the training data to generate adversarial samples.2.Design a module to assist with the proposal training: The existing mainstream detection framework r-cnn series network structures has a large number of background areas in the candidate proposals generated in the first stage,and the second stage con-trols the foreground and background proposals by random sampling.But this can cause the gradient to be dominated by a large number of simple samples during training.This paper considers the use of a module to assist in the second-stage sampling,which assign certain focus to candidate object proposals generated in object detectors,for filling the consequence caused by the uneven distribution of complex samples and simple samples.3.Design constraints to optimize the imbalanced classification: In the categories with few samples,the classifier is easy to classify depend on irrelevant features due to the contingency in the training set,which always leads to the over-fitting problem.Increas-ing the variance between features can make the model's feature distribution wider and reduce the redundancy of model features.Therefore,a loss function for variance penalty is designed in this paper to make the training of the model more controllable.