Applying a Thurstonian, Two-Stage Method in the Standardized Assessment of Writing

Joshua Aaron McGrane, Stephen Mark Humphry, Sandra Heldsinger

Research output: Contribution to journalArticlepeer-review

10 Citations (Scopus)


National standardized assessment programs have increasingly included extended written performances, amplifying the need for reliable, valid, and efficient methods of assessment. This article examines a two-stage method using comparative judgments and calibrated exemplars as a complement and alternative to existing methods of assessing writing. Written performances were taken from Australia’s National Assessment Program—Literacy and Numeracy, which included both narrative and persuasive performances from students aged 8 to 15. In Stage 1, assessors performed comparative judgments on 160 performances to form a scale of 36 calibrated exemplars. These comparative judgments showed a very high level of reliability and concurrent validity. In Stage 2, assessors scored 2,380 new performances by matching them to the most similar calibrated exemplar. These matching judgments showed a generally high level of reliability and concurrent validity and were reasonably efficient after a familiarization period. Further research is suggested to enhance Stage 2 by simplifying the exemplar scale and scaffolding it with detailed descriptors. Overall, the findings support the use of the method in standardized writing assessment and its application to various learning areas.

Original languageEnglish
Pages (from-to)297-311
Number of pages15
JournalApplied Measurement in Education
Issue number4
Publication statusPublished - 2 Oct 2018


Dive into the research topics of 'Applying a Thurstonian, Two-Stage Method in the Standardized Assessment of Writing'. Together they form a unique fingerprint.

Cite this