Raters’ perceptions of assessment criteria relevance

Stephen Humphry, Sandy Heldsinger

Research output: Contribution to journalArticlepeer-review

12 Citations (Scopus)

Abstract

This study adopts a novel approach to investigate perceptions of assessment criteria relevance in differentiating writing performance levels. Experienced writing assessors were asked to directly compare pairs of performances. For each comparison, assessors were asked to determine which performance was better and to record which aspects of writing were used to make determinations. To do so, assessors referred to a marking rubric containing 10 criteria. By triangulating direct/explicit qualitative comparisons of performances against indirect/implied comparisons using a rubric, the study aims to provide unique insights into the perceived relevance of criteria. Whereas the rubric presupposes relatively equal relevance of all criteria, in direct comparisons participating assessors perceive certain criteria relevant more often than others. In particular, conventions of writing, such as punctuation and spelling, were perceived relevant less often than authorial aspects. In combination with descriptions of qualitative gradations contained in the rubric itself, the information obtained from the study aims to assist with identifying and delineating criteria perceived relevant to developmental levels by the expert assessors. These and other findings have potential implications for selecting which criteria to use in narrative writing rubrics and they suggest advantages of granting assessors discretion in their selection and application of criteria.

Original languageEnglish
Pages (from-to)1-13
Number of pages13
JournalAssessing Writing
Volume41
DOIs
Publication statusPublished - 1 Jul 2019

Fingerprint

Dive into the research topics of 'Raters’ perceptions of assessment criteria relevance'. Together they form a unique fingerprint.

Cite this