Controlling Bias in Both Constructed Response and Multiple-Choice Items When Analyzed With the Dichotomous Rasch Model

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Even though guessing biases difficulty estimates as a function of item difficulty in the dichotomous Rasch model, assessment programs with tests which include multiple-choice items often construct scales using this model. Research has shown that when all items are multiple-choice, this bias can largely be eliminated. However, many assessments have a combination of multiple-choice and constructed response items. Using vertically scaled numeracy assessments from a large-scale assessment program, this article shows that eliminating the bias on estimates of the multiple-choice items also impacts on the difficulty estimates of the constructed response items. This implies that the original estimates of the constructed response items were biased by the guessing on the multiple-choice items. This bias has implications for both defining difficulties in item banks for use in adaptive testing composed of both multiple-choice and constructed response items, and for the construction of proficiency scales.

Original languageEnglish
Pages (from-to)281-307
Number of pages27
JournalJournal of Educational Measurement
Volume55
Issue number2
DOIs
Publication statusPublished - 1 Jun 2018

Fingerprint

trend
Research
bank

Cite this

@article{d6a32fdb94654a05a2491e5c64a3f6fa,
title = "Controlling Bias in Both Constructed Response and Multiple-Choice Items When Analyzed With the Dichotomous Rasch Model",
abstract = "Even though guessing biases difficulty estimates as a function of item difficulty in the dichotomous Rasch model, assessment programs with tests which include multiple-choice items often construct scales using this model. Research has shown that when all items are multiple-choice, this bias can largely be eliminated. However, many assessments have a combination of multiple-choice and constructed response items. Using vertically scaled numeracy assessments from a large-scale assessment program, this article shows that eliminating the bias on estimates of the multiple-choice items also impacts on the difficulty estimates of the constructed response items. This implies that the original estimates of the constructed response items were biased by the guessing on the multiple-choice items. This bias has implications for both defining difficulties in item banks for use in adaptive testing composed of both multiple-choice and constructed response items, and for the construction of proficiency scales.",
author = "David Andrich and Ida Marais",
year = "2018",
month = "6",
day = "1",
doi = "10.1111/jedm.12176",
language = "English",
volume = "55",
pages = "281--307",
journal = "Journal of Educational Measurement (JEM)",
issn = "0274-838X",
publisher = "Wiley-Blackwell",
number = "2",

}

TY - JOUR

T1 - Controlling Bias in Both Constructed Response and Multiple-Choice Items When Analyzed With the Dichotomous Rasch Model

AU - Andrich, David

AU - Marais, Ida

PY - 2018/6/1

Y1 - 2018/6/1

N2 - Even though guessing biases difficulty estimates as a function of item difficulty in the dichotomous Rasch model, assessment programs with tests which include multiple-choice items often construct scales using this model. Research has shown that when all items are multiple-choice, this bias can largely be eliminated. However, many assessments have a combination of multiple-choice and constructed response items. Using vertically scaled numeracy assessments from a large-scale assessment program, this article shows that eliminating the bias on estimates of the multiple-choice items also impacts on the difficulty estimates of the constructed response items. This implies that the original estimates of the constructed response items were biased by the guessing on the multiple-choice items. This bias has implications for both defining difficulties in item banks for use in adaptive testing composed of both multiple-choice and constructed response items, and for the construction of proficiency scales.

AB - Even though guessing biases difficulty estimates as a function of item difficulty in the dichotomous Rasch model, assessment programs with tests which include multiple-choice items often construct scales using this model. Research has shown that when all items are multiple-choice, this bias can largely be eliminated. However, many assessments have a combination of multiple-choice and constructed response items. Using vertically scaled numeracy assessments from a large-scale assessment program, this article shows that eliminating the bias on estimates of the multiple-choice items also impacts on the difficulty estimates of the constructed response items. This implies that the original estimates of the constructed response items were biased by the guessing on the multiple-choice items. This bias has implications for both defining difficulties in item banks for use in adaptive testing composed of both multiple-choice and constructed response items, and for the construction of proficiency scales.

UR - http://www.scopus.com/inward/record.url?scp=85048049121&partnerID=8YFLogxK

U2 - 10.1111/jedm.12176

DO - 10.1111/jedm.12176

M3 - Article

VL - 55

SP - 281

EP - 307

JO - Journal of Educational Measurement (JEM)

JF - Journal of Educational Measurement (JEM)

SN - 0274-838X

IS - 2

ER -