Model learning for structured predictions from complementary data sources

This dissertation explores novel model learning approaches to advance human brain-like capabilities of solving intricate tasks by simultaneously processing multiple information sources. It proposes an efficient self-supervised classification technique for set-based samples with no training requirements. It also develops a robust deep model for improved multimodal classification. Furthermore, it proposes a learning-based method to estimate any classifier's confidence and uses it for enhanced multimodal analyses. Lastly, a novel approach is presented for direct cross-modality feature matching. The developed techniques are evaluated for numerous applications including face recognition, surveillance, coral classification, emotion analysis and image pose localization, with significant performance gains.