Acoustic feature design for speech recognition, a statistical information-theoretic approach

Bayesian classifiers rely on models of the a priori and class-conditional feature distributions; the classifier is trained by optimizing these models to best represent features observed in a training corpus according to a certain criterion. In many problems of interest, the true class-conditional feature probability density function (PDF) is not a member of the set of PDFs the classifier can represent.; This dissertation addresses this model mismatch problem. We formulate it as the problem of minimizing the relative entropy between the true conditional probability density function and the hypothesized probabilistic model. Based on this formulation, we provide a computationally efficient solution to the problem based on volume-preserving maps; existing linear transform designs are shown to be special cases of the proposed solution. We apply this approach to automatic speech recognition (ASR) systems. We describe an iterative algorithm to estimate the parameters of both a class of nonlinear volume-preserving feature transforms and the hidden Markov model (HMM) that jointly optimize the objective function for an HMM-based ASR system.; In the second part of this work we present a generalization of linear discriminant analysis (LDA) that optimizes a discriminative criterion and solves the problem in the lower-dimensional subspace. We start with showing that the calculation of the LDA projection matrix is a maximum mutual information estimation problem in the lower-dimensional space with some constraints on the model of the joint conditional and unconditional PDFs of the features, and then, by relaxing these constraints, we develop a dimensionality reduction approach that maximizes the conditional mutual information between the class identity and the feature vector in the lower-dimensional space given the recognizer model.