Detection of Malignant Melanoma Using Artificial Intelligence: An Observational Study of Diagnostic Accuracy

Mike Phillips, Jack Greenhalgh, Helen Marsden, Ioulios Palamaras

Research output: Contribution to journalArticle

Abstract

Background: Malignant melanoma can most successfully be cured when diagnosed at an early stage in the natural history. However, there is controversy over screening programs and many advocate screening only for high-risk individuals.

Objectives: This study aimed to evaluate the accuracy of an artificial intelligence neural network (Deep Ensemble for Recognition of Melanoma [DERM]) to identify malignant melanoma from dermoscopic images of pigmented skin lesions and to show how this compared to doctors' performance assessed by meta-analysis.

Methods: DERM was trained and tested using 7,102 dermoscopic images of both histologically confirmed melanoma (24%) and benign pigmented lesions (76%). A meta-analysis was conducted of studies examining the accuracy of naked-eye examination, with or without dermoscopy, by specialist and general physicians whose clinical diagnosis was compared to histopathology. The meta-analysis was based on evaluation of 32,226 pigmented lesions including 3,277 histopathology-confirmed malignant melanoma cases. The receiver operating characteristic (ROC) curve was used to examine and compare the diagnostic accuracy.

Results: DERM achieved a ROC area under the curve (AUC) of 0.93 (95% confidence interval: 0.92-0.94), and sensitivity and specificity of 85.0% and 85.3%, respectively. Avoidance of false-negative results is essential, so different decision thresholds were examined. At 95% sensitivity DERM achieved a specificity of 64.1% and at 95% specificity the sensitivity was 67%. The meta-analysis showed primary care physicians (10 studies) achieve an AUC of 0.83 (95% confidence interval: 0.79-0.86), with sensitivity and specificity of 79.9% and 70.9%; and dermatologists (92 studies) 0.91 (0.88-0.93), 87.5%, and 81.4%, respectively.

Conclusions: DERM has the potential to be used as a decision support tool in primary care, by providing dermatologist-grade recommendation on the likelihood of malignant melanoma.

Original languageEnglish
Article numbere2020011
JournalDermatology Practical & Conceptual
Volume10
Issue number1
DOIs
Publication statusPublished - 2020

Fingerprint

Artificial Intelligence
Observational Studies
Melanoma
Meta-Analysis
Sensitivity and Specificity
ROC Curve
Area Under Curve
Confidence Intervals
Dermoscopy
Primary Care Physicians
Natural History
Primary Health Care
Physicians
Skin

Cite this

Phillips, Mike ; Greenhalgh, Jack ; Marsden, Helen ; Palamaras, Ioulios. / Detection of Malignant Melanoma Using Artificial Intelligence : An Observational Study of Diagnostic Accuracy. In: Dermatology Practical & Conceptual. 2020 ; Vol. 10, No. 1.
@article{129f40a3b3504ab6a408ac120723714d,
title = "Detection of Malignant Melanoma Using Artificial Intelligence: An Observational Study of Diagnostic Accuracy",
abstract = "Background: Malignant melanoma can most successfully be cured when diagnosed at an early stage in the natural history. However, there is controversy over screening programs and many advocate screening only for high-risk individuals.Objectives: This study aimed to evaluate the accuracy of an artificial intelligence neural network (Deep Ensemble for Recognition of Melanoma [DERM]) to identify malignant melanoma from dermoscopic images of pigmented skin lesions and to show how this compared to doctors' performance assessed by meta-analysis.Methods: DERM was trained and tested using 7,102 dermoscopic images of both histologically confirmed melanoma (24{\%}) and benign pigmented lesions (76{\%}). A meta-analysis was conducted of studies examining the accuracy of naked-eye examination, with or without dermoscopy, by specialist and general physicians whose clinical diagnosis was compared to histopathology. The meta-analysis was based on evaluation of 32,226 pigmented lesions including 3,277 histopathology-confirmed malignant melanoma cases. The receiver operating characteristic (ROC) curve was used to examine and compare the diagnostic accuracy.Results: DERM achieved a ROC area under the curve (AUC) of 0.93 (95{\%} confidence interval: 0.92-0.94), and sensitivity and specificity of 85.0{\%} and 85.3{\%}, respectively. Avoidance of false-negative results is essential, so different decision thresholds were examined. At 95{\%} sensitivity DERM achieved a specificity of 64.1{\%} and at 95{\%} specificity the sensitivity was 67{\%}. The meta-analysis showed primary care physicians (10 studies) achieve an AUC of 0.83 (95{\%} confidence interval: 0.79-0.86), with sensitivity and specificity of 79.9{\%} and 70.9{\%}; and dermatologists (92 studies) 0.91 (0.88-0.93), 87.5{\%}, and 81.4{\%}, respectively.Conclusions: DERM has the potential to be used as a decision support tool in primary care, by providing dermatologist-grade recommendation on the likelihood of malignant melanoma.",
author = "Mike Phillips and Jack Greenhalgh and Helen Marsden and Ioulios Palamaras",
year = "2020",
doi = "10.5826/dpc.1001a11",
language = "English",
volume = "10",
journal = "Dermatology Practical & Conceptual",
issn = "2160-9381",
publisher = "International Dermoscopy Society",
number = "1",

}

Detection of Malignant Melanoma Using Artificial Intelligence : An Observational Study of Diagnostic Accuracy. / Phillips, Mike; Greenhalgh, Jack; Marsden, Helen; Palamaras, Ioulios.

In: Dermatology Practical & Conceptual, Vol. 10, No. 1, e2020011, 2020.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Detection of Malignant Melanoma Using Artificial Intelligence

T2 - An Observational Study of Diagnostic Accuracy

AU - Phillips, Mike

AU - Greenhalgh, Jack

AU - Marsden, Helen

AU - Palamaras, Ioulios

PY - 2020

Y1 - 2020

N2 - Background: Malignant melanoma can most successfully be cured when diagnosed at an early stage in the natural history. However, there is controversy over screening programs and many advocate screening only for high-risk individuals.Objectives: This study aimed to evaluate the accuracy of an artificial intelligence neural network (Deep Ensemble for Recognition of Melanoma [DERM]) to identify malignant melanoma from dermoscopic images of pigmented skin lesions and to show how this compared to doctors' performance assessed by meta-analysis.Methods: DERM was trained and tested using 7,102 dermoscopic images of both histologically confirmed melanoma (24%) and benign pigmented lesions (76%). A meta-analysis was conducted of studies examining the accuracy of naked-eye examination, with or without dermoscopy, by specialist and general physicians whose clinical diagnosis was compared to histopathology. The meta-analysis was based on evaluation of 32,226 pigmented lesions including 3,277 histopathology-confirmed malignant melanoma cases. The receiver operating characteristic (ROC) curve was used to examine and compare the diagnostic accuracy.Results: DERM achieved a ROC area under the curve (AUC) of 0.93 (95% confidence interval: 0.92-0.94), and sensitivity and specificity of 85.0% and 85.3%, respectively. Avoidance of false-negative results is essential, so different decision thresholds were examined. At 95% sensitivity DERM achieved a specificity of 64.1% and at 95% specificity the sensitivity was 67%. The meta-analysis showed primary care physicians (10 studies) achieve an AUC of 0.83 (95% confidence interval: 0.79-0.86), with sensitivity and specificity of 79.9% and 70.9%; and dermatologists (92 studies) 0.91 (0.88-0.93), 87.5%, and 81.4%, respectively.Conclusions: DERM has the potential to be used as a decision support tool in primary care, by providing dermatologist-grade recommendation on the likelihood of malignant melanoma.

AB - Background: Malignant melanoma can most successfully be cured when diagnosed at an early stage in the natural history. However, there is controversy over screening programs and many advocate screening only for high-risk individuals.Objectives: This study aimed to evaluate the accuracy of an artificial intelligence neural network (Deep Ensemble for Recognition of Melanoma [DERM]) to identify malignant melanoma from dermoscopic images of pigmented skin lesions and to show how this compared to doctors' performance assessed by meta-analysis.Methods: DERM was trained and tested using 7,102 dermoscopic images of both histologically confirmed melanoma (24%) and benign pigmented lesions (76%). A meta-analysis was conducted of studies examining the accuracy of naked-eye examination, with or without dermoscopy, by specialist and general physicians whose clinical diagnosis was compared to histopathology. The meta-analysis was based on evaluation of 32,226 pigmented lesions including 3,277 histopathology-confirmed malignant melanoma cases. The receiver operating characteristic (ROC) curve was used to examine and compare the diagnostic accuracy.Results: DERM achieved a ROC area under the curve (AUC) of 0.93 (95% confidence interval: 0.92-0.94), and sensitivity and specificity of 85.0% and 85.3%, respectively. Avoidance of false-negative results is essential, so different decision thresholds were examined. At 95% sensitivity DERM achieved a specificity of 64.1% and at 95% specificity the sensitivity was 67%. The meta-analysis showed primary care physicians (10 studies) achieve an AUC of 0.83 (95% confidence interval: 0.79-0.86), with sensitivity and specificity of 79.9% and 70.9%; and dermatologists (92 studies) 0.91 (0.88-0.93), 87.5%, and 81.4%, respectively.Conclusions: DERM has the potential to be used as a decision support tool in primary care, by providing dermatologist-grade recommendation on the likelihood of malignant melanoma.

U2 - 10.5826/dpc.1001a11

DO - 10.5826/dpc.1001a11

M3 - Article

VL - 10

JO - Dermatology Practical & Conceptual

JF - Dermatology Practical & Conceptual

SN - 2160-9381

IS - 1

M1 - e2020011

ER -