Compression algorithms for distributed classification with applications to distributed speech recognition

With the wide proliferation of mobile devices coupled with the explosion of new multimedia applications, there is a need for adopting a client-server architecture to enable clients with low complexity/memory to support complex multimedia applications. In these client-server systems compression is vital to minimize the communication channel bandwidth requirements by compressing the transmitted data. Traditionally, compression techniques have been designed to minimize perceptual distortion, i.e., the compressed data was intended to be heard/viewed by humans. However, recently there has been an emergence of applications in which the compressed data is processed by an algorithm. Examples include distributed estimation or classification. In these applications, for best system performance, rather than minimizing perceptual distortion, the compression algorithm should be optimized to have the least effect on the estimation/classification capability of the processing algorithm. In this work novel compression techniques optimized for classification are proposed.; The first application considered is remote speech recognition, where the speech recognizer uses compressed data to recognize the spoken utterance. For this application, a scalable encoder designed to maximize recognition performance is proposed. The scalable encoder is shown to have superior rate-recognition performance compared to conventional speech encoders. Additionally, a scalable recognition system capable of trading off recognition performance for reduced complexity is also proposed. These are useful in distributed speech recognition systems where several clients are accessing a single server and efficient server design becomes important to both reduce the computational complexity and the bandwidth requirement at the server.; The second application considered is distributed classification, where the classifier operates on the compressed and transmitted data to make the class decision. A novel algorithm is proposed which is shown to significant reduce the misclassification penalty with a small sacrifice in distortion performance. The generality of this algorithm is demonstrated by extending it to improve the performance of table-lookup encoders. It is shown that by designing product vector quantizers (PVQ) to approximate a higher dimension vector quantizer (VQ), a significant improvement in PSNR performance over conventional PVQ design is possible while not increasing the encoding time significantly over conventional table-lookup encoding.; Finally, a new distortion metric, mutual information (MI) loss , is proposed for designing quantizers in distributed classification applications. It is shown that the MI loss optimized quantizers are able to provide significant improvements in classification performance when compared to mean square error optimized quantizers. Empirical quantizer design and rate allocation algorithms are provided to optimize quantizers for minimizing MI loss. Additionally, it is shown that the MI loss metric can be used to design quantizers operating on low dimension vectors. This is a vital requirement in classification systems employing high dimension classifiers as it enables design of optimal and practical minimum MI loss quantizers implementable on low complexity/memory clients.