A supervised learning framework: using assessment to identify students at risk of dropping out of a MOOC

David Monllaó Olivé, Du Q. Huynh, Mark Reynolds, Martin Dougiamas, Damyon Wiese

Research output: Contribution to journalArticle

Abstract

Both educational data mining and learning analytics aim to understand learners and optimise learning processes of educational settings like Moodle, a learning management system (LMS). Analytics in an LMS covers many different aspects: finding students at risk of abandoning a course or identifying students with difficulties before the assessments. Thus, there are multiple prediction models that can be explored. The prediction models can target at the course also. For instance, will this activity assessment engage learners? To ease the evaluation and usage of prediction models in Moodle, we abstract out the most relevant elements of prediction models and develop an analytics framework for Moodle. Apart from the software framework, we also present a case study model which uses variables based on assessments to predict students at risk of dropping out of a massive open online course that has been offered eight times from 2013 to 2018, including a total of 46,895 students. A neural network is trained with data from past courses and the framework generates insights about students at risk in ongoing courses. Predictions are then generated after the first, the second, and the third quarters of the course. The average accuracy that we achieve is 88.81% with a 0.9337 F1 score and a 73.12% of the area under the ROC curve.

Original languageEnglish
JournalJournal of Computing in Higher Education
DOIs
Publication statusPublished - 24 May 2019

Fingerprint

learning
student
educational setting
management
neural network
learning process
evaluation
software

Cite this

@article{6a756103f54342a284cb6f489868cf19,
title = "A supervised learning framework: using assessment to identify students at risk of dropping out of a MOOC",
abstract = "Both educational data mining and learning analytics aim to understand learners and optimise learning processes of educational settings like Moodle, a learning management system (LMS). Analytics in an LMS covers many different aspects: finding students at risk of abandoning a course or identifying students with difficulties before the assessments. Thus, there are multiple prediction models that can be explored. The prediction models can target at the course also. For instance, will this activity assessment engage learners? To ease the evaluation and usage of prediction models in Moodle, we abstract out the most relevant elements of prediction models and develop an analytics framework for Moodle. Apart from the software framework, we also present a case study model which uses variables based on assessments to predict students at risk of dropping out of a massive open online course that has been offered eight times from 2013 to 2018, including a total of 46,895 students. A neural network is trained with data from past courses and the framework generates insights about students at risk in ongoing courses. Predictions are then generated after the first, the second, and the third quarters of the course. The average accuracy that we achieve is 88.81{\%} with a 0.9337 F1 score and a 73.12{\%} of the area under the ROC curve.",
keywords = "Assessment, Educational data mining, Learning analytics, Learning management systems, Machine learning, Moodle, Neural networks",
author = "{Monlla{\'o} Oliv{\'e}}, David and Huynh, {Du Q.} and Mark Reynolds and Martin Dougiamas and Damyon Wiese",
year = "2019",
month = "5",
day = "24",
doi = "10.1007/s12528-019-09230-1",
language = "English",
journal = "Journal of Computing in Higher Education",
issn = "1042-1726",
publisher = "Paideia Publishers",

}

A supervised learning framework : using assessment to identify students at risk of dropping out of a MOOC. / Monllaó Olivé, David; Huynh, Du Q.; Reynolds, Mark; Dougiamas, Martin; Wiese, Damyon.

In: Journal of Computing in Higher Education, 24.05.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A supervised learning framework

T2 - using assessment to identify students at risk of dropping out of a MOOC

AU - Monllaó Olivé, David

AU - Huynh, Du Q.

AU - Reynolds, Mark

AU - Dougiamas, Martin

AU - Wiese, Damyon

PY - 2019/5/24

Y1 - 2019/5/24

N2 - Both educational data mining and learning analytics aim to understand learners and optimise learning processes of educational settings like Moodle, a learning management system (LMS). Analytics in an LMS covers many different aspects: finding students at risk of abandoning a course or identifying students with difficulties before the assessments. Thus, there are multiple prediction models that can be explored. The prediction models can target at the course also. For instance, will this activity assessment engage learners? To ease the evaluation and usage of prediction models in Moodle, we abstract out the most relevant elements of prediction models and develop an analytics framework for Moodle. Apart from the software framework, we also present a case study model which uses variables based on assessments to predict students at risk of dropping out of a massive open online course that has been offered eight times from 2013 to 2018, including a total of 46,895 students. A neural network is trained with data from past courses and the framework generates insights about students at risk in ongoing courses. Predictions are then generated after the first, the second, and the third quarters of the course. The average accuracy that we achieve is 88.81% with a 0.9337 F1 score and a 73.12% of the area under the ROC curve.

AB - Both educational data mining and learning analytics aim to understand learners and optimise learning processes of educational settings like Moodle, a learning management system (LMS). Analytics in an LMS covers many different aspects: finding students at risk of abandoning a course or identifying students with difficulties before the assessments. Thus, there are multiple prediction models that can be explored. The prediction models can target at the course also. For instance, will this activity assessment engage learners? To ease the evaluation and usage of prediction models in Moodle, we abstract out the most relevant elements of prediction models and develop an analytics framework for Moodle. Apart from the software framework, we also present a case study model which uses variables based on assessments to predict students at risk of dropping out of a massive open online course that has been offered eight times from 2013 to 2018, including a total of 46,895 students. A neural network is trained with data from past courses and the framework generates insights about students at risk in ongoing courses. Predictions are then generated after the first, the second, and the third quarters of the course. The average accuracy that we achieve is 88.81% with a 0.9337 F1 score and a 73.12% of the area under the ROC curve.

KW - Assessment

KW - Educational data mining

KW - Learning analytics

KW - Learning management systems

KW - Machine learning

KW - Moodle

KW - Neural networks

UR - http://www.scopus.com/inward/record.url?scp=85066916478&partnerID=8YFLogxK

U2 - 10.1007/s12528-019-09230-1

DO - 10.1007/s12528-019-09230-1

M3 - Article

JO - Journal of Computing in Higher Education

JF - Journal of Computing in Higher Education

SN - 1042-1726

ER -