Alternate sensor based speech systems for speaker assessment and robust human communication

This study explores the suitability of an alternate sensor and advancements in speech system algorithm development especially for stress detection and speaker verification systems based on the use of the Physiological Microphone (PMIC). By virtue of its proximity to the glottal excitation source and its robustness to ambient noise, PMIC based systems for stress detection and speaker verification outperforms the traditional close-talk microphone (CTM). The combined system designed, with stress detection as the front-end to the speaker verification system will boost overall performance of speaker verification under unknown stress conditions. As a result, easy, fast and reliable deployment of speaker verification with or without the PMIC sensor is possible. The captured PMIC speech signal is muffled since it is a skin surface vibration wave and not a traditional acoustic signal, and requires significantly higher listener effort. Thus, for human listeners it has lower speech intelligibility and may introduce fatigue over extended time periods. A statistical algorithm proposed in this thesis overcomes this issue by normalizing the PMIC signal to more closely represent an acousti speech signal. Evaluations performed over two databases (UT-Scope and IEEE) indicate the effectiveness of the statistical projection to improve speech quality and intelligibility of the PMIC signal. The non-acoustic cues captured by the PMIC sensor are also exploited to create awareness of the speakers physiological status, indicating the presence of speech under physical stress with the help of breathing pattern parameters, minute volume and tidal volume. Finally, the PMIC sensor is employed within an automatic classification algorithm used to indicate driver fatigue with extended cellphone use while driving. The work presented in this thesis demonstrates the potential of incorporating non-acoustic/alternate sensors for improving overall robustness of speech system performance.