TY - JOUR
T1 - Deep Learning to Detect OCT-derived Diabetic Macular Edema from Color Retinal Photographs
T2 - A Multicenter Validation Study
AU - Liu, Xinle
AU - Ali, Tayyeba K.
AU - Singh, Preeti
AU - Shah, Ami
AU - McKinney, Scott Mayer
AU - Ruamviboonsuk, Paisan
AU - Turner, Angus W.
AU - Keane, Pearse A.
AU - Chotcomwongse, Peranut
AU - Nganthavee, Variya
AU - Chia, Mark
AU - Huemer, Josef
AU - Cuadros, Jorge
AU - Raman, Rajiv
AU - Corrado, Greg S.
AU - Peng, Lily
AU - Webster, Dale R.
AU - Hammel, Naama
AU - Varadarajan, Avinash V.
AU - Liu, Yun
AU - Chopra, Reena
AU - Bavishi, Pinal
N1 - Funding Information:
P.A.K.: Consultant – DeepMind, Roche, Novartis, Apellis; Equity owner – Big Picture Medical; Speaker fees – Heidelberg Engineering, Topcon, Allergan, Bayer; Support – Moorfields Eye Charity Career Development Award (R190028A), UK Research & Innovation Future Leaders Fellowship (MR/T019050/1).
Publisher Copyright:
© 2022 American Academy of Ophthalmology
PY - 2022/5
Y1 - 2022/5
N2 - Purpose: To validate the generalizability of a deep learning system (DLS) that detects diabetic macular edema (DME) from 2-dimensional color fundus photographs (CFP), for which the reference standard for retinal thickness and fluid presence is derived from 3-dimensional OCT. Design: Retrospective validation of a DLS across international datasets. Participants: Paired CFP and OCT of patients from diabetic retinopathy (DR) screening programs or retina clinics. The DLS was developed using data sets from Thailand, the United Kingdom, and the United States and validated using 3060 unique eyes from 1582 patients across screening populations in Australia, India, and Thailand. The DLS was separately validated in 698 eyes from 537 screened patients in the United Kingdom with mild DR and suspicion of DME based on CFP. Methods: The DLS was trained using DME labels from OCT. The presence of DME was based on retinal thickening or intraretinal fluid. The DLS's performance was compared with expert grades of maculopathy and to a previous proof-of-concept version of the DLS. We further simulated the integration of the current DLS into an algorithm trained to detect DR from CFP. Main Outcome Measures: The superiority of specificity and noninferiority of sensitivity of the DLS for the detection of center-involving DME, using device-specific thresholds, compared with experts. Results: The primary analysis in a combined data set spanning Australia, India, and Thailand showed the DLS had 80% specificity and 81% sensitivity, compared with expert graders, who had 59% specificity and 70% sensitivity. Relative to human experts, the DLS had significantly higher specificity (P = 0.008) and noninferior sensitivity (P < 0.001). In the data set from the United Kingdom, the DLS had a specificity of 80% (P < 0.001 for specificity of >50%) and a sensitivity of 100% (P = 0.02 for sensitivity of > 90%). Conclusions: The DLS can generalize to multiple international populations with an accuracy exceeding that of experts. The clinical value of this DLS to reduce false-positive referrals, thus decreasing the burden on specialist eye care, warrants a prospective evaluation.
AB - Purpose: To validate the generalizability of a deep learning system (DLS) that detects diabetic macular edema (DME) from 2-dimensional color fundus photographs (CFP), for which the reference standard for retinal thickness and fluid presence is derived from 3-dimensional OCT. Design: Retrospective validation of a DLS across international datasets. Participants: Paired CFP and OCT of patients from diabetic retinopathy (DR) screening programs or retina clinics. The DLS was developed using data sets from Thailand, the United Kingdom, and the United States and validated using 3060 unique eyes from 1582 patients across screening populations in Australia, India, and Thailand. The DLS was separately validated in 698 eyes from 537 screened patients in the United Kingdom with mild DR and suspicion of DME based on CFP. Methods: The DLS was trained using DME labels from OCT. The presence of DME was based on retinal thickening or intraretinal fluid. The DLS's performance was compared with expert grades of maculopathy and to a previous proof-of-concept version of the DLS. We further simulated the integration of the current DLS into an algorithm trained to detect DR from CFP. Main Outcome Measures: The superiority of specificity and noninferiority of sensitivity of the DLS for the detection of center-involving DME, using device-specific thresholds, compared with experts. Results: The primary analysis in a combined data set spanning Australia, India, and Thailand showed the DLS had 80% specificity and 81% sensitivity, compared with expert graders, who had 59% specificity and 70% sensitivity. Relative to human experts, the DLS had significantly higher specificity (P = 0.008) and noninferior sensitivity (P < 0.001). In the data set from the United Kingdom, the DLS had a specificity of 80% (P < 0.001 for specificity of >50%) and a sensitivity of 100% (P = 0.02 for sensitivity of > 90%). Conclusions: The DLS can generalize to multiple international populations with an accuracy exceeding that of experts. The clinical value of this DLS to reduce false-positive referrals, thus decreasing the burden on specialist eye care, warrants a prospective evaluation.
KW - artificial intelligence
KW - deep learning
KW - diabetic macular edema
KW - diabetic retinopathy
KW - optical coherence tomography
UR - http://www.scopus.com/inward/record.url?scp=85125527776&partnerID=8YFLogxK
U2 - 10.1016/j.oret.2021.12.021
DO - 10.1016/j.oret.2021.12.021
M3 - Article
C2 - 34999015
AN - SCOPUS:85125527776
SN - 2468-6530
VL - 6
SP - 398
EP - 410
JO - Ophthalmology Retina
JF - Ophthalmology Retina
IS - 5
ER -