Robust RGB-D face recognition using Kinect sensor

B.Y.L. Li, M. Xue, Ajmal Mian, W. Liu, A. Krishna

    Research output: Contribution to journalArticle

    10 Citations (Scopus)

    Abstract

    © 2016 Elsevier B.V.
    In this paper we propose a robust face recognition algorithm for low resolution RGB-D Kinect data. Many techniques are proposed for image preprocessing due to the noisy depth data. First, facial symmetry is exploited based on the 3D point cloud to obtain a canonical frontal view image irrespective of the initial pose and then depth data is converted to XYZ normal maps. Secondly, multi-channel Discriminant Transforms are then used to project RGB to DCS (Discriminant Color Space) and normal maps to DNM (Discriminant Normal Maps). Finally, a Multi-channel Robust Sparse Coding method is proposed that codes the multiple channels (DCS or DNM) of a test image as a sparse combination of training samples with different pixel weighting. Weights are calculated dynamically in an iterative process to achieve robustness against variations in pose, illumination, facial expressions and disguise. In contrast to existing techniques, our multi-channel approach is more robust to variations. Reconstruction errors of the test image (DCS and DNM) are normalized and fused to decide its identity. The proposed algorithm is evaluated on four public databases. It achieves 98.4% identification rate on CurtinFaces, a Kinect database with 4784 RGB-D images of 52 subjects. Using a first versus all protocol on the Bosphorus, CASIA and FRGC v2 databases, the proposed algorithm achieves 97.6%, 95.6% and 95.2% identification rates respectively. To the best of our knowledge, these are the highest identification rates reported so far for the first three databases.
    Original languageEnglish
    Pages (from-to)93-108
    JournalNeurocomputing
    Volume214
    Early online date17 Jun 2016
    DOIs
    Publication statusPublished - 19 Nov 2016

    Fingerprint

    Face recognition
    Databases
    Color
    Sensors
    Facial Expression
    Lighting
    Weights and Measures
    Pixels
    Facial Recognition
    Mathematical transformations

    Cite this

    Li, B.Y.L. ; Xue, M. ; Mian, Ajmal ; Liu, W. ; Krishna, A. / Robust RGB-D face recognition using Kinect sensor. In: Neurocomputing. 2016 ; Vol. 214. pp. 93-108.
    @article{0963b53cceda406cb533cbe213dea5e6,
    title = "Robust RGB-D face recognition using Kinect sensor",
    abstract = "{\circledC} 2016 Elsevier B.V. In this paper we propose a robust face recognition algorithm for low resolution RGB-D Kinect data. Many techniques are proposed for image preprocessing due to the noisy depth data. First, facial symmetry is exploited based on the 3D point cloud to obtain a canonical frontal view image irrespective of the initial pose and then depth data is converted to XYZ normal maps. Secondly, multi-channel Discriminant Transforms are then used to project RGB to DCS (Discriminant Color Space) and normal maps to DNM (Discriminant Normal Maps). Finally, a Multi-channel Robust Sparse Coding method is proposed that codes the multiple channels (DCS or DNM) of a test image as a sparse combination of training samples with different pixel weighting. Weights are calculated dynamically in an iterative process to achieve robustness against variations in pose, illumination, facial expressions and disguise. In contrast to existing techniques, our multi-channel approach is more robust to variations. Reconstruction errors of the test image (DCS and DNM) are normalized and fused to decide its identity. The proposed algorithm is evaluated on four public databases. It achieves 98.4{\%} identification rate on CurtinFaces, a Kinect database with 4784 RGB-D images of 52 subjects. Using a first versus all protocol on the Bosphorus, CASIA and FRGC v2 databases, the proposed algorithm achieves 97.6{\%}, 95.6{\%} and 95.2{\%} identification rates respectively. To the best of our knowledge, these are the highest identification rates reported so far for the first three databases.",
    author = "B.Y.L. Li and M. Xue and Ajmal Mian and W. Liu and A. Krishna",
    year = "2016",
    month = "11",
    day = "19",
    doi = "10.1016/j.neucom.2016.06.012",
    language = "English",
    volume = "214",
    pages = "93--108",
    journal = "Neurocomputing",
    issn = "0925-2312",
    publisher = "Pergamon",

    }

    Robust RGB-D face recognition using Kinect sensor. / Li, B.Y.L.; Xue, M.; Mian, Ajmal; Liu, W.; Krishna, A.

    In: Neurocomputing, Vol. 214, 19.11.2016, p. 93-108.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Robust RGB-D face recognition using Kinect sensor

    AU - Li, B.Y.L.

    AU - Xue, M.

    AU - Mian, Ajmal

    AU - Liu, W.

    AU - Krishna, A.

    PY - 2016/11/19

    Y1 - 2016/11/19

    N2 - © 2016 Elsevier B.V. In this paper we propose a robust face recognition algorithm for low resolution RGB-D Kinect data. Many techniques are proposed for image preprocessing due to the noisy depth data. First, facial symmetry is exploited based on the 3D point cloud to obtain a canonical frontal view image irrespective of the initial pose and then depth data is converted to XYZ normal maps. Secondly, multi-channel Discriminant Transforms are then used to project RGB to DCS (Discriminant Color Space) and normal maps to DNM (Discriminant Normal Maps). Finally, a Multi-channel Robust Sparse Coding method is proposed that codes the multiple channels (DCS or DNM) of a test image as a sparse combination of training samples with different pixel weighting. Weights are calculated dynamically in an iterative process to achieve robustness against variations in pose, illumination, facial expressions and disguise. In contrast to existing techniques, our multi-channel approach is more robust to variations. Reconstruction errors of the test image (DCS and DNM) are normalized and fused to decide its identity. The proposed algorithm is evaluated on four public databases. It achieves 98.4% identification rate on CurtinFaces, a Kinect database with 4784 RGB-D images of 52 subjects. Using a first versus all protocol on the Bosphorus, CASIA and FRGC v2 databases, the proposed algorithm achieves 97.6%, 95.6% and 95.2% identification rates respectively. To the best of our knowledge, these are the highest identification rates reported so far for the first three databases.

    AB - © 2016 Elsevier B.V. In this paper we propose a robust face recognition algorithm for low resolution RGB-D Kinect data. Many techniques are proposed for image preprocessing due to the noisy depth data. First, facial symmetry is exploited based on the 3D point cloud to obtain a canonical frontal view image irrespective of the initial pose and then depth data is converted to XYZ normal maps. Secondly, multi-channel Discriminant Transforms are then used to project RGB to DCS (Discriminant Color Space) and normal maps to DNM (Discriminant Normal Maps). Finally, a Multi-channel Robust Sparse Coding method is proposed that codes the multiple channels (DCS or DNM) of a test image as a sparse combination of training samples with different pixel weighting. Weights are calculated dynamically in an iterative process to achieve robustness against variations in pose, illumination, facial expressions and disguise. In contrast to existing techniques, our multi-channel approach is more robust to variations. Reconstruction errors of the test image (DCS and DNM) are normalized and fused to decide its identity. The proposed algorithm is evaluated on four public databases. It achieves 98.4% identification rate on CurtinFaces, a Kinect database with 4784 RGB-D images of 52 subjects. Using a first versus all protocol on the Bosphorus, CASIA and FRGC v2 databases, the proposed algorithm achieves 97.6%, 95.6% and 95.2% identification rates respectively. To the best of our knowledge, these are the highest identification rates reported so far for the first three databases.

    U2 - 10.1016/j.neucom.2016.06.012

    DO - 10.1016/j.neucom.2016.06.012

    M3 - Article

    VL - 214

    SP - 93

    EP - 108

    JO - Neurocomputing

    JF - Neurocomputing

    SN - 0925-2312

    ER -