Multi-objective Deep Neural Network Architecture Search Based On Evolutionary Algorithm

The rapid development of deep learning technology has become a key driver for breakthroughs in fields such as computer vision and natural language processing technology,and has driven a new round of explosive development of artificial intelligence.Convolutional Neural Networks(CNN)is one of the representative methods in the field of deep learning.Due to its topology,hyperparameters and other key technologies,CNN has achieved excellent performance in various tasks.However,in practical production applications,the neural network architecture still relies on expert empirical knowledge for design,and requires repeated adjustments and modifications to the network architecture based on model performance and other factors,which consumes a lot of human and material resources.Therefore,the use of neural network architecture search techniques to automate the construction and design of promising network models has become a research hotspot in the field of deep learning.The main contributions and results are as follows.1.To address the lack of uncertainty measures in current neural network architectures,by introducing bayesian neural networks,this thesis proposes a neural architecture search algorithm based on bayesian convolutional neural networks.The algorithm constructs the search space by a fixed-length integer encoding scheme and uses an evolutionary algorithm as the search strategy to deeply explore the influence of relevant parameters such as the convolutional kernel size on the network architecture.Meanwhile,an early stopping strategy is used in the performance evaluation phase to reduce the time loss caused by individual evaluation.The optimal models searched by the proposed algorithm are evaluated on the CIFAR10 and CIFAR100 datasets,and the experimental results verified the effectiveness of the proposed algorithm.2.To address the problem that training and evaluating candidate neural networks consume a lot of computational resources and time,this thesis designs an evolutionary neural architecture search algorithm based on the assistance of surrogate models.The algorithm constructs a cell-based search space through an adjacency matrix encoding scheme,uses an evolutionary algorithm as a search strategy to guide the optimization of network models,and incorporates an surrogate model as a performance predictor to evaluate candidate network architectures thus greatly reducing computational costs to improve the efficiency of neural network architecture search.The advanced performance of the models searched by the proposed algorithm is demonstrated by comparing experiments with 14 other NAS algorithms on the CIFAR10 and CIFAR100 datasets.3.Most neural network architecture search methods optimize only a single objective for the core objectives of model parameters,search time and image classification accuracy.To solve this problem,this thesis embeds the idea of multi-objective algorithm into the neural network architecture search to optimize multiple conflicting objectives simultaneously,so that the best model searched by the algorithm has a better comprehensive performance.