Response Dependence and the Measurement of Change

Because of confounding effects that can mask change when persons respond to the same items on more than one occasion, the measurement of change is a challenge. The specific effect on change studied in this paper is that observed when responses of persons to items at time 2 are dependent statistically on their responses at time 1. In addition, because this response dependence may affect the change differently for different locations of items relative to persons at time 1, the initial targeting of persons to items was studied. For a specific change in means of persons, dichotomous data were simulated according to the Rasch model with varying degrees of dependence and varying initial targeting of persons to items. Data were analysed, also using the Rasch model, in two ways: firstly, by treating items used at time 1 and time 2 as distinct ones (rack analysis) and, secondly, by treating persons at time 1 and time 2 as distinct ones (stack analysis). With the rack analysis the change is revealed through the item parameters and with the stack analysis the change is revealed through the person parameters. With no response dependence the two analyses gave equivalent and correct measures of change. With increasing dependence change was increasingly masked or increasingly amplified, depending on the targeting of items to persons at time 1. Response dependence affected the measurement of change in both analyses, but not always in the same way. The paper serves as a warning against undetected dependence and also considers evidence that can be used in the analysis of real data sets for detecting the presence of dependence when measuring change.