Audio-visual biometrics using reliability-based late fusion and deep neural networks

Recently data acquisition for audio-visual biometric systems has been facilitated by the advancement of mobile phone technology. The captured data may be of poor quality due to various factors such as variation in environment, pose and illumination. Although a quality based fusion approach may be used to deal with this, measuring the quality at the signal level is difficult, particularly for the visual inputs. This thesis presents a reliability-based late fusion framework as a method of dealing with noisy Input signals. In addition, a novel three-step algorithm is proposed to train a multimodal deep neural network.