Listening with your eyes: towards a practical visual speech recognition system

This thesis develops visual speech feature extraction methods using the most advanced computer vision and machine learning techniques. First, a visual and audio-visual speech recognition system, which combines both grey and depth information, is proposed. This system is expected to provide the research community with a new perspective to overcome the limitations of grey-level visual speech features and boost visual and audio-visual speech accuracy. Second, several automatic deep visual feature learning techniques are also introduced. Experimental results show that these proposed techniques outperform the performance of the commonly used handcrafted features.