Learning, adaptation and optimization: The nonnegative Boltzmann machine and the tunneling salesman algorithm

This thesis presents two novel concepts; the first, in the field of Machine Learning, where a probabilistic generative model, the Nonnegative Boltzmann Machine, is developed for the specific case of modeling continuous nonnegative data; and the second in the area of Global Optimization, where a novel algorithm, the Tunneling Salesman Algorithm, is presented that exploits the mathematics of quantum mechanics (QM).; The Nonnegative Boltzmann machine (NNBM) is a recurrent neural network model that can describe multimodal nonnegative data. Application of maximum likelihood estimation to this model gives a learning rule that is analogous to the binary Boltzmann machine. We examine the utility of the mean field approximation for the NNBM, and describe the use of Monte Carlo sampling techniques to learn the model parameters. Reflective slice sampling is particularly well-suited for this distribution, and can efficiently be implemented to sample the distribution. A second-order mean-field approximation for the Nonnegative Boltzmann Machine model, obtained using a "high-temperature" expansion is also presented. The two approaches to learning the model are tested on learning a bimodal 2-dimensional model, a high-dimensional translationally-invariant distribution, a generative model for handwritten digits, and a generative model for face images.; Machine learning problems frequently reduce to search for an optimum of some cost-function on the parameters of the model in question. In the novel optimization algorithm we propose, the groundstate of a quantum system is approximated using a method based on perturbation theory. By choosing the mass of the particle in the system to be sufficiently large, the ground state wave function of the system becomes localized to the global minimum of its potential.; Hence, a given optimization problem involving the minimisation of a cost function can be restated as the problem of finding the ground state of the QM system with a potential specified by that cost function. In particular, computationally 'hard' integer problems, including factoring and the travelling salesman problem, can be cast as QM problems involving a quartic polynomial potential. In this thesis an optimization algorithm based on these ideas is demonstrated in 1 and 2-dimensions, for visualization, and then applied to the problem of factoring biprimes of up to 12 bits, and 4- and 5-City Travelling Salesman problems.