A Silent-Speech Interface using Electro-Optical Stomatography

Speech technology is a major and growing industry that enriches the lives of technologically-minded people in a number of ways. Many potential users are, however, excluded: Namely, all speakers who cannot easily, or even at all, produce speech. Silent-Speech Interfaces offer a way to communicate with a machine through a convenient speech recognition interface without the need for acoustic speech. To that end, the speech movements need to be captured and mapped to either text or acoustic speech. This dissertation proposes a new Silent-Speech Interface based on a newly developed measurement technology called Electro-Optical Stomatography and a novel parametric vocal tract model to facilitate real-time speech synthesis based on the measured data. The hardware was used to conduct command word recognition studies reaching state-of-the-art intra- and inter-individual performance. Furthermore, a study on using the hardware to control the vocal tract model in a direct articulation-to-speech synthesis loop was also completed. While the intelligibility of consonants and connected speech was quite poor, the intelligibility of synthesized vowels was high. Promising ways to improve the system are discussed in the outlook.