Cognitive interviews for reviewing multiple-choice items in mathematics

Joan Burfitt

Research output: Contribution to journalArticle

Abstract

During the development of a test of multiple-choice items, cognitive interviews were conducted with students from lower secondary school. The purpose of these interviews was to confirm that the prospective respondents' interpretation of mathematics test items was consistent with the interpretation intended by the item writer. The conversations with the students provided unexpected and intriguing insights into the way students think and speak about mathematical concepts: ideas worthy of comment and further research. The benefits and limitations of using cognitive interviews to improve assessment items, and the recommended procedures for conducting such interviews, had been identified from earlier research studies. With these findings, a framework was developed for designing and implementing a series of interviews with ten Year 9 students in Western Australia. The items reviewed by these students were intended for an online test of the skills and understandings essential for the development of sound proportional reasoning for Year 8 students. The responses to the interview questions not only provided useful suggestions on how to improve the test items and on the suitability of the test instrument, but they also provided valuable information about the students' understanding and communication of what they had learned.

Original languageEnglish
Pages (from-to)346-362
Number of pages17
JournalIssues in Educational Research
Volume29
Issue number2
Publication statusPublished - 2019

Cite this

@article{ea813af1daa84b4884358ba183785140,
title = "Cognitive interviews for reviewing multiple-choice items in mathematics",
abstract = "During the development of a test of multiple-choice items, cognitive interviews were conducted with students from lower secondary school. The purpose of these interviews was to confirm that the prospective respondents' interpretation of mathematics test items was consistent with the interpretation intended by the item writer. The conversations with the students provided unexpected and intriguing insights into the way students think and speak about mathematical concepts: ideas worthy of comment and further research. The benefits and limitations of using cognitive interviews to improve assessment items, and the recommended procedures for conducting such interviews, had been identified from earlier research studies. With these findings, a framework was developed for designing and implementing a series of interviews with ten Year 9 students in Western Australia. The items reviewed by these students were intended for an online test of the skills and understandings essential for the development of sound proportional reasoning for Year 8 students. The responses to the interview questions not only provided useful suggestions on how to improve the test items and on the suitability of the test instrument, but they also provided valuable information about the students' understanding and communication of what they had learned.",
keywords = "QUESTIONS, VALIDITY, SCALE",
author = "Joan Burfitt",
year = "2019",
language = "English",
volume = "29",
pages = "346--362",
journal = "Issues in Educational Research",
issn = "1837-6290",
publisher = "WESTERN AUSTRALIAN INST EDUCATIONAL RESEARCH INC",
number = "2",

}

Cognitive interviews for reviewing multiple-choice items in mathematics. / Burfitt, Joan.

In: Issues in Educational Research, Vol. 29, No. 2, 2019, p. 346-362.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Cognitive interviews for reviewing multiple-choice items in mathematics

AU - Burfitt, Joan

PY - 2019

Y1 - 2019

N2 - During the development of a test of multiple-choice items, cognitive interviews were conducted with students from lower secondary school. The purpose of these interviews was to confirm that the prospective respondents' interpretation of mathematics test items was consistent with the interpretation intended by the item writer. The conversations with the students provided unexpected and intriguing insights into the way students think and speak about mathematical concepts: ideas worthy of comment and further research. The benefits and limitations of using cognitive interviews to improve assessment items, and the recommended procedures for conducting such interviews, had been identified from earlier research studies. With these findings, a framework was developed for designing and implementing a series of interviews with ten Year 9 students in Western Australia. The items reviewed by these students were intended for an online test of the skills and understandings essential for the development of sound proportional reasoning for Year 8 students. The responses to the interview questions not only provided useful suggestions on how to improve the test items and on the suitability of the test instrument, but they also provided valuable information about the students' understanding and communication of what they had learned.

AB - During the development of a test of multiple-choice items, cognitive interviews were conducted with students from lower secondary school. The purpose of these interviews was to confirm that the prospective respondents' interpretation of mathematics test items was consistent with the interpretation intended by the item writer. The conversations with the students provided unexpected and intriguing insights into the way students think and speak about mathematical concepts: ideas worthy of comment and further research. The benefits and limitations of using cognitive interviews to improve assessment items, and the recommended procedures for conducting such interviews, had been identified from earlier research studies. With these findings, a framework was developed for designing and implementing a series of interviews with ten Year 9 students in Western Australia. The items reviewed by these students were intended for an online test of the skills and understandings essential for the development of sound proportional reasoning for Year 8 students. The responses to the interview questions not only provided useful suggestions on how to improve the test items and on the suitability of the test instrument, but they also provided valuable information about the students' understanding and communication of what they had learned.

KW - QUESTIONS

KW - VALIDITY

KW - SCALE

M3 - Article

VL - 29

SP - 346

EP - 362

JO - Issues in Educational Research

JF - Issues in Educational Research

SN - 1837-6290

IS - 2

ER -