Evolutionary Neural Architecture Search Based On Improved ConvNet And Attention Mechanism

Convolutional Neural Networks(CNNs)due to excellent capability of feature extraction has gained prominent achievements in the field of computer vision.The performance of CNNs highly relies upon their network architectures.Designing CNNs with exceptional performance requires rich expert experience,and needs to constantly adjust the network architecture as well as parameters to obtain a model that satisfies real applications,which is difficult for most users to achieve.Neural architecture search aims to design a neural architecture that achieves the best possible performance.Neural architecture search based on evolutionary algorithms aims to automatically design and adaptively optimize CNNs using limited computing resources with minimal human intervention,achieving advanced performance beyond that of artificial neural networks,and has become one of the hot topics in the field of deep learning.This thesis based on evolutionary algorithms automatically designs convolutional neural network architectures,with the following main research components.1.Evolutionary neural architecture search algorithms are in the spotlight due to strong robustness and superior performance.This thesis summarizes the research progress of evolutionary neural architectures in recent years based on different optimization strategies.The key techniques to improve the performance of evolutionary neural networks are summarized,including network coding,modular search spaces,fitness evaluation and attention mechanisms.2.To search for the network architecture with exceptional performance and further improve search efficiency,an efficient evolutionary convolutional neural network architecture search algorithm based on group whitening residual building blocks and large kernel subspace attention module is proposed.The architecture search space is designed as a chained space based on group whitening residual building blocks,ensuring the promising performance and lightweight structure of generated architecture;the corresponding evolutionary operators are designed to navigate the evolutionary direction of architectures and achieve adaptive optimization of the network architecture;a novel but effective ”Large Kernel Subspace Attention Module”(LKSAM)is proposed to enhance the efficiency and representational capacity of neural networks.LKSAM is capable of extracting the complex interaction information between diverse channels with few parameters while degrades the redundancy of feature maps in spatial and channel dimension.Through inferring individual attention maps for each feature subspace,the multi-scale and multi-frequency feature representation is effectively achieved.The proposed algorithm conducted extensive experiments on the image classification benchmark dataset CIFAR and was compared with current state-of-the-art classification algorithms,achieving 96.90% classification accuracy with 3.96 M parameters on CIFAR10 dataset;On CIFAR100 dataset,the optimal network architecture has an accuracy of 79.36%,the number of parameters is 4.94 M,and the algorithm only consumes 0.75 and 1.15 GPU days,respectively,The results demonstrate the proposed algorithm is not only computationally efficient but also highly competitive in terms of performance.3.To further verify the promising performance of the proposed algorithm in real application scenarios,the algorithm in this paper is applied to the traffic sign recognition task,and achieves 99.60% recognition accuracy on the GTSRB dataset with only 3.76 M parameters.Excellent classification performance is obtained while achieves the lightweight design of the model,and the effectiveness of the proposed algorithm designing architectures for specific tasks is verified.Meanwhile,the proposed large kernel subspace attention module is integrated into the search space of classical neural architecture search algorithms and compare the search results,the results indicate that the proposed attention model has excellent flexibility and adaptability with negligible computational costs incurred.A promising network model can be designed with low computational overheads using the evolutionary algorithm-based architectural search method in this thesis.