Using pairwise comparison and ordered exemplars as a basis for a novel method of standard setting in narrative writing

Stephen Humphry, Lenore Adie, Carolyn Maxwell, Sonia Sappl

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

Standard setting of extended performances is fraught with difficulties due to the subjective nature of the scoring decision. This article introduces and reports results of a novel systematic method for setting standards for assessments that require extended performances: the Scaled Exemplar Standard Setting method (SESS). This two-stage method brings together (1) pairwise comparison of samples to scale performances and (2) standard setting involving multiple standards with tasks presented to judges in a validated order. By utilizing paired comparison methodology in the first stage, the judgment is made more manageable by chunking it into a series of pairs. This method produces scaled performances that can be checked for internal consistency of the judgments. The validated, ordered performances are used in the standard setting stage, making the task for judges as transparent and straightforward as possible. Recommendations and implications for standard setting are discussed in light of the results.
Original languageEnglish
Article number1183695
Number of pages13
JournalFrontiers in Education
Volume8
Early online date2 Aug 2023
DOIs
Publication statusPublished - 2023

Fingerprint

Dive into the research topics of 'Using pairwise comparison and ordered exemplars as a basis for a novel method of standard setting in narrative writing'. Together they form a unique fingerprint.

Cite this