Multiobjective Evolving Deep Neural Networkmodels And Applications

Artificial neural networks are one type of computational intelligence methods that mimic neurons and their connections in brain to solve various data processing and learning problems.With the development of big data and increase of computational capabilities,deep neural networks as deeper,more complex,and better modeling ability neural networks,have achieved many breakthroughs in many big data processing problems,especially for remote sensing imagery.Remote sensing imagery is difficult to handle because of high dimension,high redundancy,and unstructured representation.While deep neural networks have the properties of deep abstraction,automatic representation learning,and efficient modeling,and thus are suitable to deal with remote sensing imagery.In this dissertation,we first research on the modeling and optimization problems in deep neural networks.Then focusing on remote sensing imagery,we attempt to solve the problems in change detection.The researches in this dissertation are listed as follows:(1)Focusing on the difficulty of the balance between representation ability and sparsity in unsupervised sparse feature learning models,we propose a multiobjective sparse feature learning model.In this model,the representation ability of network and sparsity of features are respectively modeled as two objective functions.Then by optimizing this model,we can obtain a set of solutions that have different tradeoffs between the two objectives.Due to the large scale of the neural network,we design a multiobjective induced learning procedure based on an improved multiobjective differential evolutionary algorithm and finally obtain a solution that can well balance the tradeoff between the two objectives.The experiments demonstrate that the designed learning procedure is efficient and the features learned greatly improves the performance of feature learning models.(2)In the connecting structure learning problem,there also exists the problem of the balance between network representation ability and sparsity of connections.We directly model the network connection structure without considering the weights and biases.The basic principle is to represent the input data with as less connections as possible.Then a multiobjective model is constructed.We design an improved multiobjective evolutionary algorithm by considering the properties of input data.With the layer-wise optimization,the optimal structure is obtained and then the weights and biases are trained by back-propagation.The experiments demonstrate the structure learning capability of the proposed model.(3)Based on the above two models,we use deep neural networks to solve the problem of change detection in remote sensing images.Unsupervised sparse feature learning models can remove redundant information and learn the structured feature of images.Therefore they are robust to speckle noise in radar images.The sparse features can improve generative ability of neural networks,and thus can relieve the impact of rare labeled samples.Meanwhile,the connecting structure can well represent the data structure.Thus the accuracy of change detection can be improved.(4)We further increase the difficulty of change detection,i.e.,change detection in heterogeneous multisource images.Most of the change detection methods are for homogeneous images,i.e.,the compared images are from the same kind of sensor,and thus the images can be compared directly.While heterogeneous multisource images are from different kind of sensors which results in unstructured data.Most change detection methods for heterogeneous images are supervised that learn the relationship between different representations.Therefore,focusing on the different data structure,we design a deep convolutional coupling neural network and define an objective function to minimize the difference of unchanged regions.Then the changed regions will be highlighted.The experiments demonstrate that the proposed network is able to detect changes from both heterogeneous and homogeneous images accurately.(5)Most change detection methods are implemented based on the hypothesis that compared images are accurately co-registered.In other words,the pixels at the same position in the compared images are also at the same geo-location.So that they can be compared easily.While in many cases,the co-registration cannot accurately register the images,such as images from different view angles,corrupted images by noise and whether conditions,and problems in co-registration methods.Therefore,we define a bipartite difference neural network to compare the global features which can avoid the detection errors caused by unregistered local objects.This network is based on the architecture of convolutional neural network.The change information is obtained by optimizing network parameters and change disguise maps defined by us.We design a new network architecture,a novel objective function,and corresponding optimization method.The new method achieves the change detection in unregistered images which is of great difficulty for traditional change detection methods.The researches in this dissertation start from the modeling and optimization theories of deep neural networks,then focus on the practical problem of change detection in remote sensing images.We gradually increase the difficulty of problems and attempt to solve challenging problems.