Machine learning-based prediction of heart failure readmission or death: implications of choosing the right model and the right metrics

Research output: Contribution to journalArticle

Abstract

Aims: Machine learning (ML) is widely believed to be able to learn complex hidden interactions from the data and has the potential in predicting events such as heart failure (HF) readmission and death. Recent studies have revealed conflicting results likely due to failure to take into account the class imbalance problem commonly seen with medical data. We developed a new ML approach to predict 30 day HF readmission or death and compared the performance of this model with other commonly used prediction models. Methods and results: We identified all Western Australian patients aged above 65 years admitted for HF between 2003 and 2008 in the linked Hospital Morbidity Data Collection. Taking into consideration the class imbalance problem, we developed a multi-layer perceptron (MLP)-based approach to predict 30 day HF readmission or death and compared the predictive performances using the performance metrics, that is, area under the receiver operating characteristic curve (AUC), area under the precision–recall curve (AUPRC), sensitivity and specificity with other ML and regression models. Out of the 10 757 patients with HF, 23.6% were readmitted or died within 30 days of hospital discharge. We observed an AUC of 0.55, 0.53, 0.58, and 0.54 while an AUPRC of 0.39, 0.38, 0.46, and 0.38 for weighted random forest, weighted decision trees, logistic regression, and weighted support vector machines models, respectively. The MLP-based approach produced the highest AUC (0.62) and AUPRC (0.46) with 48% sensitivity and 70% specificity. Conclusions: We show that for the medical data with class imbalance, the proposed MLP-based approach is superior to other ML and regression techniques for the prediction of 30 day HF readmission or death.

Original languageEnglish
Pages (from-to)428-435
Number of pages8
JournalESC Heart Failure
Volume6
Issue number2
DOIs
Publication statusPublished - 1 Apr 2019

Fingerprint

Area Under Curve
Heart Failure
Neural Networks (Computer)
Sensitivity and Specificity
Decision Trees
ROC Curve
Machine Learning
Logistic Models
Morbidity

Cite this

@article{4871b44bb12641008ced1c2c41d18cd6,
title = "Machine learning-based prediction of heart failure readmission or death: implications of choosing the right model and the right metrics",
abstract = "Aims: Machine learning (ML) is widely believed to be able to learn complex hidden interactions from the data and has the potential in predicting events such as heart failure (HF) readmission and death. Recent studies have revealed conflicting results likely due to failure to take into account the class imbalance problem commonly seen with medical data. We developed a new ML approach to predict 30 day HF readmission or death and compared the performance of this model with other commonly used prediction models. Methods and results: We identified all Western Australian patients aged above 65 years admitted for HF between 2003 and 2008 in the linked Hospital Morbidity Data Collection. Taking into consideration the class imbalance problem, we developed a multi-layer perceptron (MLP)-based approach to predict 30 day HF readmission or death and compared the predictive performances using the performance metrics, that is, area under the receiver operating characteristic curve (AUC), area under the precision–recall curve (AUPRC), sensitivity and specificity with other ML and regression models. Out of the 10 757 patients with HF, 23.6{\%} were readmitted or died within 30 days of hospital discharge. We observed an AUC of 0.55, 0.53, 0.58, and 0.54 while an AUPRC of 0.39, 0.38, 0.46, and 0.38 for weighted random forest, weighted decision trees, logistic regression, and weighted support vector machines models, respectively. The MLP-based approach produced the highest AUC (0.62) and AUPRC (0.46) with 48{\%} sensitivity and 70{\%} specificity. Conclusions: We show that for the medical data with class imbalance, the proposed MLP-based approach is superior to other ML and regression techniques for the prediction of 30 day HF readmission or death.",
keywords = "Heart failure, Machine learning, Prediction, Readmission",
author = "Awan, {Saqib Ejaz} and Mohammed Bennamoun and Ferdous Sohel and Sanfilippo, {Frank Mario} and Girish Dwivedi",
year = "2019",
month = "4",
day = "1",
doi = "10.1002/ehf2.12419",
language = "English",
volume = "6",
pages = "428--435",
journal = "ESC Heart Failure",
issn = "2055-5822",
publisher = "The Heart Failure Association of the European Society of Cardiology",
number = "2",

}

TY - JOUR

T1 - Machine learning-based prediction of heart failure readmission or death

T2 - implications of choosing the right model and the right metrics

AU - Awan, Saqib Ejaz

AU - Bennamoun, Mohammed

AU - Sohel, Ferdous

AU - Sanfilippo, Frank Mario

AU - Dwivedi, Girish

PY - 2019/4/1

Y1 - 2019/4/1

N2 - Aims: Machine learning (ML) is widely believed to be able to learn complex hidden interactions from the data and has the potential in predicting events such as heart failure (HF) readmission and death. Recent studies have revealed conflicting results likely due to failure to take into account the class imbalance problem commonly seen with medical data. We developed a new ML approach to predict 30 day HF readmission or death and compared the performance of this model with other commonly used prediction models. Methods and results: We identified all Western Australian patients aged above 65 years admitted for HF between 2003 and 2008 in the linked Hospital Morbidity Data Collection. Taking into consideration the class imbalance problem, we developed a multi-layer perceptron (MLP)-based approach to predict 30 day HF readmission or death and compared the predictive performances using the performance metrics, that is, area under the receiver operating characteristic curve (AUC), area under the precision–recall curve (AUPRC), sensitivity and specificity with other ML and regression models. Out of the 10 757 patients with HF, 23.6% were readmitted or died within 30 days of hospital discharge. We observed an AUC of 0.55, 0.53, 0.58, and 0.54 while an AUPRC of 0.39, 0.38, 0.46, and 0.38 for weighted random forest, weighted decision trees, logistic regression, and weighted support vector machines models, respectively. The MLP-based approach produced the highest AUC (0.62) and AUPRC (0.46) with 48% sensitivity and 70% specificity. Conclusions: We show that for the medical data with class imbalance, the proposed MLP-based approach is superior to other ML and regression techniques for the prediction of 30 day HF readmission or death.

AB - Aims: Machine learning (ML) is widely believed to be able to learn complex hidden interactions from the data and has the potential in predicting events such as heart failure (HF) readmission and death. Recent studies have revealed conflicting results likely due to failure to take into account the class imbalance problem commonly seen with medical data. We developed a new ML approach to predict 30 day HF readmission or death and compared the performance of this model with other commonly used prediction models. Methods and results: We identified all Western Australian patients aged above 65 years admitted for HF between 2003 and 2008 in the linked Hospital Morbidity Data Collection. Taking into consideration the class imbalance problem, we developed a multi-layer perceptron (MLP)-based approach to predict 30 day HF readmission or death and compared the predictive performances using the performance metrics, that is, area under the receiver operating characteristic curve (AUC), area under the precision–recall curve (AUPRC), sensitivity and specificity with other ML and regression models. Out of the 10 757 patients with HF, 23.6% were readmitted or died within 30 days of hospital discharge. We observed an AUC of 0.55, 0.53, 0.58, and 0.54 while an AUPRC of 0.39, 0.38, 0.46, and 0.38 for weighted random forest, weighted decision trees, logistic regression, and weighted support vector machines models, respectively. The MLP-based approach produced the highest AUC (0.62) and AUPRC (0.46) with 48% sensitivity and 70% specificity. Conclusions: We show that for the medical data with class imbalance, the proposed MLP-based approach is superior to other ML and regression techniques for the prediction of 30 day HF readmission or death.

KW - Heart failure

KW - Machine learning

KW - Prediction

KW - Readmission

UR - http://www.scopus.com/inward/record.url?scp=85062341113&partnerID=8YFLogxK

U2 - 10.1002/ehf2.12419

DO - 10.1002/ehf2.12419

M3 - Article

VL - 6

SP - 428

EP - 435

JO - ESC Heart Failure

JF - ESC Heart Failure

SN - 2055-5822

IS - 2

ER -