Can Proficiency Criteria Be Accurately Identified During Real-Time Fundamental Movement Skill Assessment?

Research output: Contribution to journalArticle

Abstract

Purpose: Fundamental movement skill (FMS) assessors in education environments rely upon real-time FMS assessment; however, the recognition of individual proficiency criteria during real-time process-oriented FMS assessment may be problematic. Few studies consider the accuracy of identifying individual proficiency criteria in process-oriented FMS assessment, even though criteria are relied upon for intervention planning. This study aimed to further understand assessors’ ability to recognize proficiency criteria during real-time FMS assessment and the impact of assessor experience on assessment accuracy. Methods: 10 primary teachers, and 7 pediatric professionals assessed 10 performances of four FMSs (Jump, Hop, Kick, Throw) presented in videos and point-light displays using the Test of Gross Motor Development-2. Results: Accuracy in identifying proficiency criteria was moderate for both pediatric professionals (74.73%) and primary teachers (69.58%), with no differences between groups. In contrast, reliability of overall proficiency scores was good to excellent (ICC>0.8) in both groups. Some individual criteria may be more difficult to assess, evidenced by large average accuracy ranges within skills (e.g., 46% difference between Throw criteria 1 (34%) and 2 (80%)). Conclusions: The study reinforces the difficulty of observing proficiency criteria during real-time FMS assessment regardless of assessor experience. Results suggest that assessors can accurately score overall FMS proficiency, whilst accurate identification of proficiency criteria is problematic. Accurate criterion identification is crucial to understand skill deficiencies and inform subsequent intervention. Attentional demands during real-time assessment may be too great to allow accurate criterion identification, even by experienced assessors, which presents an important consideration for test administrators and developers.

Original languageEnglish
Number of pages9
JournalResearch Quarterly for Exercise and Sport
DOIs
Publication statusE-pub ahead of print - 3 Sep 2019

Fingerprint

Pediatrics
Humulus
Administrative Personnel
Education
Light

Cite this

@article{6b09dc1a9b274b1da038c442a8aa5de1,
title = "Can Proficiency Criteria Be Accurately Identified During Real-Time Fundamental Movement Skill Assessment?",
abstract = "Purpose: Fundamental movement skill (FMS) assessors in education environments rely upon real-time FMS assessment; however, the recognition of individual proficiency criteria during real-time process-oriented FMS assessment may be problematic. Few studies consider the accuracy of identifying individual proficiency criteria in process-oriented FMS assessment, even though criteria are relied upon for intervention planning. This study aimed to further understand assessors’ ability to recognize proficiency criteria during real-time FMS assessment and the impact of assessor experience on assessment accuracy. Methods: 10 primary teachers, and 7 pediatric professionals assessed 10 performances of four FMSs (Jump, Hop, Kick, Throw) presented in videos and point-light displays using the Test of Gross Motor Development-2. Results: Accuracy in identifying proficiency criteria was moderate for both pediatric professionals (74.73{\%}) and primary teachers (69.58{\%}), with no differences between groups. In contrast, reliability of overall proficiency scores was good to excellent (ICC>0.8) in both groups. Some individual criteria may be more difficult to assess, evidenced by large average accuracy ranges within skills (e.g., 46{\%} difference between Throw criteria 1 (34{\%}) and 2 (80{\%})). Conclusions: The study reinforces the difficulty of observing proficiency criteria during real-time FMS assessment regardless of assessor experience. Results suggest that assessors can accurately score overall FMS proficiency, whilst accurate identification of proficiency criteria is problematic. Accurate criterion identification is crucial to understand skill deficiencies and inform subsequent intervention. Attentional demands during real-time assessment may be too great to allow accurate criterion identification, even by experienced assessors, which presents an important consideration for test administrators and developers.",
keywords = "assessment reliability, Fundamental movement skills, movement skill assessment, skill observation",
author = "Brodie Ward and Ashleigh Thornton and Brendan Lay and Nigel Chen and Michael Rosenberg",
year = "2019",
month = "9",
day = "3",
doi = "10.1080/02701367.2019.1646852",
language = "English",
journal = "Research Quarterly for Exercise and Sport",
issn = "0270-1367",
publisher = "AAHPERD",

}

TY - JOUR

T1 - Can Proficiency Criteria Be Accurately Identified During Real-Time Fundamental Movement Skill Assessment?

AU - Ward, Brodie

AU - Thornton, Ashleigh

AU - Lay, Brendan

AU - Chen, Nigel

AU - Rosenberg, Michael

PY - 2019/9/3

Y1 - 2019/9/3

N2 - Purpose: Fundamental movement skill (FMS) assessors in education environments rely upon real-time FMS assessment; however, the recognition of individual proficiency criteria during real-time process-oriented FMS assessment may be problematic. Few studies consider the accuracy of identifying individual proficiency criteria in process-oriented FMS assessment, even though criteria are relied upon for intervention planning. This study aimed to further understand assessors’ ability to recognize proficiency criteria during real-time FMS assessment and the impact of assessor experience on assessment accuracy. Methods: 10 primary teachers, and 7 pediatric professionals assessed 10 performances of four FMSs (Jump, Hop, Kick, Throw) presented in videos and point-light displays using the Test of Gross Motor Development-2. Results: Accuracy in identifying proficiency criteria was moderate for both pediatric professionals (74.73%) and primary teachers (69.58%), with no differences between groups. In contrast, reliability of overall proficiency scores was good to excellent (ICC>0.8) in both groups. Some individual criteria may be more difficult to assess, evidenced by large average accuracy ranges within skills (e.g., 46% difference between Throw criteria 1 (34%) and 2 (80%)). Conclusions: The study reinforces the difficulty of observing proficiency criteria during real-time FMS assessment regardless of assessor experience. Results suggest that assessors can accurately score overall FMS proficiency, whilst accurate identification of proficiency criteria is problematic. Accurate criterion identification is crucial to understand skill deficiencies and inform subsequent intervention. Attentional demands during real-time assessment may be too great to allow accurate criterion identification, even by experienced assessors, which presents an important consideration for test administrators and developers.

AB - Purpose: Fundamental movement skill (FMS) assessors in education environments rely upon real-time FMS assessment; however, the recognition of individual proficiency criteria during real-time process-oriented FMS assessment may be problematic. Few studies consider the accuracy of identifying individual proficiency criteria in process-oriented FMS assessment, even though criteria are relied upon for intervention planning. This study aimed to further understand assessors’ ability to recognize proficiency criteria during real-time FMS assessment and the impact of assessor experience on assessment accuracy. Methods: 10 primary teachers, and 7 pediatric professionals assessed 10 performances of four FMSs (Jump, Hop, Kick, Throw) presented in videos and point-light displays using the Test of Gross Motor Development-2. Results: Accuracy in identifying proficiency criteria was moderate for both pediatric professionals (74.73%) and primary teachers (69.58%), with no differences between groups. In contrast, reliability of overall proficiency scores was good to excellent (ICC>0.8) in both groups. Some individual criteria may be more difficult to assess, evidenced by large average accuracy ranges within skills (e.g., 46% difference between Throw criteria 1 (34%) and 2 (80%)). Conclusions: The study reinforces the difficulty of observing proficiency criteria during real-time FMS assessment regardless of assessor experience. Results suggest that assessors can accurately score overall FMS proficiency, whilst accurate identification of proficiency criteria is problematic. Accurate criterion identification is crucial to understand skill deficiencies and inform subsequent intervention. Attentional demands during real-time assessment may be too great to allow accurate criterion identification, even by experienced assessors, which presents an important consideration for test administrators and developers.

KW - assessment reliability

KW - Fundamental movement skills

KW - movement skill assessment

KW - skill observation

UR - http://www.scopus.com/inward/record.url?scp=85071947749&partnerID=8YFLogxK

U2 - 10.1080/02701367.2019.1646852

DO - 10.1080/02701367.2019.1646852

M3 - Article

JO - Research Quarterly for Exercise and Sport

JF - Research Quarterly for Exercise and Sport

SN - 0270-1367

ER -