Facial self similarity for sketch to photo matching

Z. Khan, Yiqun Hu, Ajmal Mian

    Research output: Chapter in Book/Conference paperConference paper

    5 Citations (Scopus)

    Abstract

    Automatic recognition of suspects from forensic sketches is of considerable interest to the law enforcement agencies. However, this task is complex due to the heterogenous nature of face sketches and photographs. To address this challenge, previous approaches generally learn a transformation of a sketch to photo or a photo to sketch at the image or feature level in order to reduce the modality gap. Such a transformation may be indeterministic and if learned from training data, is likely to over-fit the sketch artist's drawing technique. Instead, we formulate the problem in the context of matching local self similarities which are independently computed from a face sketch and a photo. The proposed Facial Self Similarity (FSS) descriptor is obtained by correlation of a small face patch with its local neighborhood. Thus, our approach avoids the need of a modality transformation, while implicitly reducing the inter-modality gap. The proposed FSS descriptor is evaluated on the CUHK Face Sketch database using sketch-photo pairs of 311 subjects. The FSS descriptor demonstrates high recognition accuracy of 99.53% and outperforms current techniques. We also evaluate the robustness of the descriptor to anomalies such as matching sketches to blurred photographs.
    Original languageEnglish
    Title of host publication2012 International Conference on Digital Image Computing Techniques and Applications
    Place of PublicationUSA
    PublisherIEEE, Institute of Electrical and Electronics Engineers
    Pages1-7
    ISBN (Print)9781467321815
    DOIs
    Publication statusPublished - 2012
    EventFacial self similarity for sketch to photo matching - Fremantle, Australia
    Duration: 1 Jan 2012 → …

    Conference

    ConferenceFacial self similarity for sketch to photo matching
    Period1/01/12 → …

    Fingerprint

    Law enforcement

    Cite this

    Khan, Z., Hu, Y., & Mian, A. (2012). Facial self similarity for sketch to photo matching. In 2012 International Conference on Digital Image Computing Techniques and Applications (pp. 1-7). USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/DICTA.2012.6411670
    Khan, Z. ; Hu, Yiqun ; Mian, Ajmal. / Facial self similarity for sketch to photo matching. 2012 International Conference on Digital Image Computing Techniques and Applications. USA : IEEE, Institute of Electrical and Electronics Engineers, 2012. pp. 1-7
    @inproceedings{a060e8cb8cb5414da2590d42735ecf7d,
    title = "Facial self similarity for sketch to photo matching",
    abstract = "Automatic recognition of suspects from forensic sketches is of considerable interest to the law enforcement agencies. However, this task is complex due to the heterogenous nature of face sketches and photographs. To address this challenge, previous approaches generally learn a transformation of a sketch to photo or a photo to sketch at the image or feature level in order to reduce the modality gap. Such a transformation may be indeterministic and if learned from training data, is likely to over-fit the sketch artist's drawing technique. Instead, we formulate the problem in the context of matching local self similarities which are independently computed from a face sketch and a photo. The proposed Facial Self Similarity (FSS) descriptor is obtained by correlation of a small face patch with its local neighborhood. Thus, our approach avoids the need of a modality transformation, while implicitly reducing the inter-modality gap. The proposed FSS descriptor is evaluated on the CUHK Face Sketch database using sketch-photo pairs of 311 subjects. The FSS descriptor demonstrates high recognition accuracy of 99.53{\%} and outperforms current techniques. We also evaluate the robustness of the descriptor to anomalies such as matching sketches to blurred photographs.",
    author = "Z. Khan and Yiqun Hu and Ajmal Mian",
    year = "2012",
    doi = "10.1109/DICTA.2012.6411670",
    language = "English",
    isbn = "9781467321815",
    pages = "1--7",
    booktitle = "2012 International Conference on Digital Image Computing Techniques and Applications",
    publisher = "IEEE, Institute of Electrical and Electronics Engineers",
    address = "United States",

    }

    Khan, Z, Hu, Y & Mian, A 2012, Facial self similarity for sketch to photo matching. in 2012 International Conference on Digital Image Computing Techniques and Applications. IEEE, Institute of Electrical and Electronics Engineers, USA, pp. 1-7, Facial self similarity for sketch to photo matching, 1/01/12. https://doi.org/10.1109/DICTA.2012.6411670

    Facial self similarity for sketch to photo matching. / Khan, Z.; Hu, Yiqun; Mian, Ajmal.

    2012 International Conference on Digital Image Computing Techniques and Applications. USA : IEEE, Institute of Electrical and Electronics Engineers, 2012. p. 1-7.

    Research output: Chapter in Book/Conference paperConference paper

    TY - GEN

    T1 - Facial self similarity for sketch to photo matching

    AU - Khan, Z.

    AU - Hu, Yiqun

    AU - Mian, Ajmal

    PY - 2012

    Y1 - 2012

    N2 - Automatic recognition of suspects from forensic sketches is of considerable interest to the law enforcement agencies. However, this task is complex due to the heterogenous nature of face sketches and photographs. To address this challenge, previous approaches generally learn a transformation of a sketch to photo or a photo to sketch at the image or feature level in order to reduce the modality gap. Such a transformation may be indeterministic and if learned from training data, is likely to over-fit the sketch artist's drawing technique. Instead, we formulate the problem in the context of matching local self similarities which are independently computed from a face sketch and a photo. The proposed Facial Self Similarity (FSS) descriptor is obtained by correlation of a small face patch with its local neighborhood. Thus, our approach avoids the need of a modality transformation, while implicitly reducing the inter-modality gap. The proposed FSS descriptor is evaluated on the CUHK Face Sketch database using sketch-photo pairs of 311 subjects. The FSS descriptor demonstrates high recognition accuracy of 99.53% and outperforms current techniques. We also evaluate the robustness of the descriptor to anomalies such as matching sketches to blurred photographs.

    AB - Automatic recognition of suspects from forensic sketches is of considerable interest to the law enforcement agencies. However, this task is complex due to the heterogenous nature of face sketches and photographs. To address this challenge, previous approaches generally learn a transformation of a sketch to photo or a photo to sketch at the image or feature level in order to reduce the modality gap. Such a transformation may be indeterministic and if learned from training data, is likely to over-fit the sketch artist's drawing technique. Instead, we formulate the problem in the context of matching local self similarities which are independently computed from a face sketch and a photo. The proposed Facial Self Similarity (FSS) descriptor is obtained by correlation of a small face patch with its local neighborhood. Thus, our approach avoids the need of a modality transformation, while implicitly reducing the inter-modality gap. The proposed FSS descriptor is evaluated on the CUHK Face Sketch database using sketch-photo pairs of 311 subjects. The FSS descriptor demonstrates high recognition accuracy of 99.53% and outperforms current techniques. We also evaluate the robustness of the descriptor to anomalies such as matching sketches to blurred photographs.

    U2 - 10.1109/DICTA.2012.6411670

    DO - 10.1109/DICTA.2012.6411670

    M3 - Conference paper

    SN - 9781467321815

    SP - 1

    EP - 7

    BT - 2012 International Conference on Digital Image Computing Techniques and Applications

    PB - IEEE, Institute of Electrical and Electronics Engineers

    CY - USA

    ER -

    Khan Z, Hu Y, Mian A. Facial self similarity for sketch to photo matching. In 2012 International Conference on Digital Image Computing Techniques and Applications. USA: IEEE, Institute of Electrical and Electronics Engineers. 2012. p. 1-7 https://doi.org/10.1109/DICTA.2012.6411670