Dense 3D Face Correspondence

Syed Zulqarnain Gilani, Ajmal Mian, Faisal Shafait, Ian Reid

    Research output: Contribution to journalArticle

    14 Citations (Scopus)

    Abstract

    We present an algorithm that automatically establishes dense correspondences between a large number of 3D faces. Starting from automatically detected sparse correspondences on the outer boundary of 3D faces, the algorithm triangulates existing correspondences and expands them iteratively by matching points of distinctive surface curvature along the triangle edges. After exhausting keypoint matches, further correspondences are established by generating evenly distributed points within triangles by evolving level set geodesic curves from the centroids of large triangles. A deformable model (K3DM) is constructed from the dense corresponded faces and an algorithm is proposed for morphing the K3DM to fit unseen faces. This algorithm iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model. We have extensively evaluated the proposed algorithms on synthetic data and real 3D faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using quantitative and qualitative benchmarks. Our algorithm achieved dense correspondences with a mean localisation error of 1.28mm on synthetic faces and detected 14 anthropometric landmarks on unseen real faces from the FRGCv2 database with 3mm precision. Furthermore, our deformable model fitting algorithm achieved 98.5% face recognition accuracy on the FRGCv2 and 98.6% on Bosphorus database. Our dense model is also able to generalize to unseen datasets.

    Original languageEnglish
    Pages (from-to)1584-1598
    JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
    Volume40
    Issue number7
    DOIs
    Publication statusPublished - 1 Jul 2018

    Fingerprint

    Correspondence
    Face
    Deformable Models
    Triangle
    Morphing
    Triangulate
    Model Fitting
    Landmarks
    Synthetic Data
    Centroid
    Face recognition
    Face Recognition
    Iterate
    Level Set
    Expand
    Geodesic
    Alignment
    Curvature
    Benchmark
    Generalise

    Cite this

    @article{4d1cc4ea09c4435a82a70a3d12a6dde0,
    title = "Dense 3D Face Correspondence",
    abstract = "We present an algorithm that automatically establishes dense correspondences between a large number of 3D faces. Starting from automatically detected sparse correspondences on the outer boundary of 3D faces, the algorithm triangulates existing correspondences and expands them iteratively by matching points of distinctive surface curvature along the triangle edges. After exhausting keypoint matches, further correspondences are established by generating evenly distributed points within triangles by evolving level set geodesic curves from the centroids of large triangles. A deformable model (K3DM) is constructed from the dense corresponded faces and an algorithm is proposed for morphing the K3DM to fit unseen faces. This algorithm iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model. We have extensively evaluated the proposed algorithms on synthetic data and real 3D faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using quantitative and qualitative benchmarks. Our algorithm achieved dense correspondences with a mean localisation error of 1.28mm on synthetic faces and detected 14 anthropometric landmarks on unseen real faces from the FRGCv2 database with 3mm precision. Furthermore, our deformable model fitting algorithm achieved 98.5{\%} face recognition accuracy on the FRGCv2 and 98.6{\%} on Bosphorus database. Our dense model is also able to generalize to unseen datasets.",
    keywords = "3D face, Databases, deformable model, Deformable models, Dense correspondence, Face, Face recognition, geodesic curves, keypoint detection, level sets, morphing, Shape, Solid modeling, Three-dimensional displays",
    author = "{Zulqarnain Gilani}, Syed and Ajmal Mian and Faisal Shafait and Ian Reid",
    year = "2018",
    month = "7",
    day = "1",
    doi = "10.1109/TPAMI.2017.2725279",
    language = "English",
    volume = "40",
    pages = "1584--1598",
    journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
    issn = "0162-8828",
    publisher = "IEEE, Institute of Electrical and Electronics Engineers",
    number = "7",

    }

    Dense 3D Face Correspondence. / Zulqarnain Gilani, Syed; Mian, Ajmal; Shafait, Faisal; Reid, Ian.

    In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, No. 7, 01.07.2018, p. 1584-1598.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Dense 3D Face Correspondence

    AU - Zulqarnain Gilani, Syed

    AU - Mian, Ajmal

    AU - Shafait, Faisal

    AU - Reid, Ian

    PY - 2018/7/1

    Y1 - 2018/7/1

    N2 - We present an algorithm that automatically establishes dense correspondences between a large number of 3D faces. Starting from automatically detected sparse correspondences on the outer boundary of 3D faces, the algorithm triangulates existing correspondences and expands them iteratively by matching points of distinctive surface curvature along the triangle edges. After exhausting keypoint matches, further correspondences are established by generating evenly distributed points within triangles by evolving level set geodesic curves from the centroids of large triangles. A deformable model (K3DM) is constructed from the dense corresponded faces and an algorithm is proposed for morphing the K3DM to fit unseen faces. This algorithm iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model. We have extensively evaluated the proposed algorithms on synthetic data and real 3D faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using quantitative and qualitative benchmarks. Our algorithm achieved dense correspondences with a mean localisation error of 1.28mm on synthetic faces and detected 14 anthropometric landmarks on unseen real faces from the FRGCv2 database with 3mm precision. Furthermore, our deformable model fitting algorithm achieved 98.5% face recognition accuracy on the FRGCv2 and 98.6% on Bosphorus database. Our dense model is also able to generalize to unseen datasets.

    AB - We present an algorithm that automatically establishes dense correspondences between a large number of 3D faces. Starting from automatically detected sparse correspondences on the outer boundary of 3D faces, the algorithm triangulates existing correspondences and expands them iteratively by matching points of distinctive surface curvature along the triangle edges. After exhausting keypoint matches, further correspondences are established by generating evenly distributed points within triangles by evolving level set geodesic curves from the centroids of large triangles. A deformable model (K3DM) is constructed from the dense corresponded faces and an algorithm is proposed for morphing the K3DM to fit unseen faces. This algorithm iterates between rigid alignment of an unseen face followed by regularized morphing of the deformable model. We have extensively evaluated the proposed algorithms on synthetic data and real 3D faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using quantitative and qualitative benchmarks. Our algorithm achieved dense correspondences with a mean localisation error of 1.28mm on synthetic faces and detected 14 anthropometric landmarks on unseen real faces from the FRGCv2 database with 3mm precision. Furthermore, our deformable model fitting algorithm achieved 98.5% face recognition accuracy on the FRGCv2 and 98.6% on Bosphorus database. Our dense model is also able to generalize to unseen datasets.

    KW - 3D face

    KW - Databases

    KW - deformable model

    KW - Deformable models

    KW - Dense correspondence

    KW - Face

    KW - Face recognition

    KW - geodesic curves

    KW - keypoint detection

    KW - level sets

    KW - morphing

    KW - Shape

    KW - Solid modeling

    KW - Three-dimensional displays

    UR - http://www.scopus.com/inward/record.url?scp=85023633865&partnerID=8YFLogxK

    U2 - 10.1109/TPAMI.2017.2725279

    DO - 10.1109/TPAMI.2017.2725279

    M3 - Article

    VL - 40

    SP - 1584

    EP - 1598

    JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

    JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

    SN - 0162-8828

    IS - 7

    ER -