A quest for a one-size-fits-all neural network: Early prediction of students at risk in online courses

David Monllao Olive, Du Huynh, Mark Reynolds, Martin Dougiamas, Damyon Wiese

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

A significant amount of research effort has been put into finding variables that can identify students at risk based on activity records available in Learning Management Systems (LMS). These variables often depend on the context for example, the course structure, how the activities are assessed or whether the course is entirely online or a blended course. To the best of our knowledge, a predictive model that can generalise well to many different types of courses using data available in the LMS does not currently exist in the Learning Analytics literature. In this study, early prediction of students at risk is tackled by training a number of neural networks to predict which students would likely submit their assignments on time based on their activity up to 2 days before assignments' due dates. 5 different datasets that cover a total of 78,722 student enrolments in 5,487 courses have been used in our study. In order to improve how well the neural networks generalise, our networks can perform different forms of feature engineering using course peers data. The different architectures of these networks have been compared to find the one with more predictive power. To validate the models trained from the networks, both new datasets and unseen examples extracted from the same datasets have been used for training. Our research show that adding contextual information results in better prediction accuracies and F1 scores. Our networks are able to give predictions with accuracies in the 67.46-81.63% range and F1 scores in the 71.30-83.09% range.

Original languageEnglish
Pages (from-to)171-183
Number of pages13
JournalIEEE Transactions on Learning Technologies
DOIs
Publication statusE-pub ahead of print - 15 Apr 2019

Fingerprint

neural network
Students
Neural networks
student
learning
predictive model
management
engineering

Cite this

@article{1dfb0f0fa86b4f8b813c360aaf480447,
title = "A quest for a one-size-fits-all neural network: Early prediction of students at risk in online courses",
abstract = "A significant amount of research effort has been put into finding variables that can identify students at risk based on activity records available in Learning Management Systems (LMS). These variables often depend on the context for example, the course structure, how the activities are assessed or whether the course is entirely online or a blended course. To the best of our knowledge, a predictive model that can generalise well to many different types of courses using data available in the LMS does not currently exist in the Learning Analytics literature. In this study, early prediction of students at risk is tackled by training a number of neural networks to predict which students would likely submit their assignments on time based on their activity up to 2 days before assignments' due dates. 5 different datasets that cover a total of 78,722 student enrolments in 5,487 courses have been used in our study. In order to improve how well the neural networks generalise, our networks can perform different forms of feature engineering using course peers data. The different architectures of these networks have been compared to find the one with more predictive power. To validate the models trained from the networks, both new datasets and unseen examples extracted from the same datasets have been used for training. Our research show that adding contextual information results in better prediction accuracies and F1 scores. Our networks are able to give predictions with accuracies in the 67.46-81.63{\%} range and F1 scores in the 71.30-83.09{\%} range.",
keywords = "Learning management systems, Neural Networks, Predictive models, Predictive models' portability",
author = "{Monllao Olive}, David and Du Huynh and Mark Reynolds and Martin Dougiamas and Damyon Wiese",
year = "2019",
month = "4",
day = "15",
doi = "10.1109/TLT.2019.2911068",
language = "English",
pages = "171--183",
journal = "IEEE Transactions on Learning Technologies",
issn = "1939-1382",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",

}

A quest for a one-size-fits-all neural network : Early prediction of students at risk in online courses. / Monllao Olive, David; Huynh, Du; Reynolds, Mark; Dougiamas, Martin; Wiese, Damyon.

In: IEEE Transactions on Learning Technologies, 15.04.2019, p. 171-183.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A quest for a one-size-fits-all neural network

T2 - Early prediction of students at risk in online courses

AU - Monllao Olive, David

AU - Huynh, Du

AU - Reynolds, Mark

AU - Dougiamas, Martin

AU - Wiese, Damyon

PY - 2019/4/15

Y1 - 2019/4/15

N2 - A significant amount of research effort has been put into finding variables that can identify students at risk based on activity records available in Learning Management Systems (LMS). These variables often depend on the context for example, the course structure, how the activities are assessed or whether the course is entirely online or a blended course. To the best of our knowledge, a predictive model that can generalise well to many different types of courses using data available in the LMS does not currently exist in the Learning Analytics literature. In this study, early prediction of students at risk is tackled by training a number of neural networks to predict which students would likely submit their assignments on time based on their activity up to 2 days before assignments' due dates. 5 different datasets that cover a total of 78,722 student enrolments in 5,487 courses have been used in our study. In order to improve how well the neural networks generalise, our networks can perform different forms of feature engineering using course peers data. The different architectures of these networks have been compared to find the one with more predictive power. To validate the models trained from the networks, both new datasets and unseen examples extracted from the same datasets have been used for training. Our research show that adding contextual information results in better prediction accuracies and F1 scores. Our networks are able to give predictions with accuracies in the 67.46-81.63% range and F1 scores in the 71.30-83.09% range.

AB - A significant amount of research effort has been put into finding variables that can identify students at risk based on activity records available in Learning Management Systems (LMS). These variables often depend on the context for example, the course structure, how the activities are assessed or whether the course is entirely online or a blended course. To the best of our knowledge, a predictive model that can generalise well to many different types of courses using data available in the LMS does not currently exist in the Learning Analytics literature. In this study, early prediction of students at risk is tackled by training a number of neural networks to predict which students would likely submit their assignments on time based on their activity up to 2 days before assignments' due dates. 5 different datasets that cover a total of 78,722 student enrolments in 5,487 courses have been used in our study. In order to improve how well the neural networks generalise, our networks can perform different forms of feature engineering using course peers data. The different architectures of these networks have been compared to find the one with more predictive power. To validate the models trained from the networks, both new datasets and unseen examples extracted from the same datasets have been used for training. Our research show that adding contextual information results in better prediction accuracies and F1 scores. Our networks are able to give predictions with accuracies in the 67.46-81.63% range and F1 scores in the 71.30-83.09% range.

KW - Learning management systems

KW - Neural Networks

KW - Predictive models

KW - Predictive models' portability

UR - http://www.scopus.com/inward/record.url?scp=85064594773&partnerID=8YFLogxK

U2 - 10.1109/TLT.2019.2911068

DO - 10.1109/TLT.2019.2911068

M3 - Article

SP - 171

EP - 183

JO - IEEE Transactions on Learning Technologies

JF - IEEE Transactions on Learning Technologies

SN - 1939-1382

ER -