Performance of multilabel machine learning models and risk stratification schemas for predicting stroke and bleeding risk in patients with non-valvular atrial fibrillation

Background: Appropriate anticoagulant therapy for patients with atrial fibrillation (AF) requires assessment of stroke and bleeding risks. However, risk stratification schemas such as CHA₂DS₂-VASc and HAS-BLED have modest predictive capacity for patients with AF. Multilabel machine learning (ML) techniques may improve predictive performance and support decision-making for anticoagulant therapy. We compared the performance of multilabel ML models with the currently used risk scores for predicting outcomes in AF patients. Methods: This was a retrospective cohort study of 9670 patients, mean age 76.9 years, 46% women, who were hospitalized with non-valvular AF, and had 1-year follow-up. The outcomes were ischemic stroke (167), major bleeding (430) admissions, all-cause death (1912) and event-free survival (7387). Discrimination and calibration of ML models were compared with clinical risk scores by area under the curve (AUC). Risk stratification was assessed using net reclassification index (NRI). Results: Multilabel gradient boosting classifier chain provided the best AUCs for stroke (0.685 95% CI 0.676, 0.694), major bleeding (0.709 95% CI 0.703, 0.716) and death (0.765 95% CI 0.763, 0.768) compared to multi-layer neural networks and classifier chain using support vector machine. It provided modest performance improvement for stroke compared to AUC of CHA₂DS₂-VASc (0.652, NRI = 3.2%, p-value = 0.1), but significantly improved major bleeding prediction compared to AUC of HAS-BLED (0.522, NRI = 22.8%, p-value < 0.05). It also achieved greater discriminant power for death compared with AUC of CHA₂DS₂-VASc (0.606, p-value < 0.05). ML models identified additional risk features such as hemoglobin level, renal function. Conclusions: Multilabel ML models can outperform clinical risk stratification scores for predicting the risk of major bleeding and death in non-valvular AF patients.