The relative predictive value of undergraduate versus graduate selection tools in two Australian medical schools

Context: Monash University and the University of Western Australia admit both school-leavers and graduates into their Bachelor of Medicine and Bachelor of Surgery (MBBS) courses. The Undergraduate Medicine and Health Sciences Admission Test (UMAT) and the Graduate Medical Schools Admissions Test (GAMSAT) are used for selection, along with an academic score and an interview score. The aim of this study was to compare the relative predictive validity of the selected components in the two entry streams, particularly UMAT versus GAMSAT. Methods: Aggregated scores for course outcomes were calculated in the categories of knowledge, clinical and total scores, at four-time points. A path analysis was conducted based on multivariate regressions with model constraint parameters defined across the outcome variables to investigate change over time. Results: Academic scores were the strongest predictors of knowledge scores and end of course results. Interview scores had a small positive increasing effect, being stronger for clinical than knowledge outcomes. The effect size for GAMSAT was greater than for UMAT. Conclusions: Aptitude tests and interview scores added small but significant incremental predictive value to previous academic achievement. GAMSAT showed larger predictive value on outcomes than UMAT, for which one section (UMAT 3) had a negative effect.