Neural network training algorithms utilizing periodic activation and relaxation

In this dissertation, neural network fundamentals and Fourier series theory are reviewed, followed by a presentation of new design techniques of network architectures. A new objective function for neural net classifier design is presented, which has more free parameters than the classical objective function. An iterative minimization technique for the objective function is derived which requires the solution of multiple sets of numerically ill-conditioned linear equations. A numerically stable solution to the neural network design equations, which utilizes the conjugate gradient algorithm and a relaxation algorithm, is presented. The design method is applied to networks used to classify remote sensing and shaping imaginary.;To transfer Fourier series into a neural network, it is necessary to convert N-dimensional Fourier series into a convenient form in which each term has a sine and a cosine. This is a multilayer network with trigonometric activations and weight sharing. Thresholds in the hidden layer are not necessary. A series of experiments on the mapping and classification applications from the new developed network shows that its performance is close to that of the multilayer perceptron network, but with a smaller number of weights. The network training performance, with the new architecture, is sometimes better than that of the MLP.