English

ERIC ED476923: Context Effects in Pretesting: Impact on Item Statistics and Examinee Scores.

In this study, the effect of appended pretesting was evaluated with regard to item statistics and examinee scores for groups of items that were pretested as part of a large-scale operational testing program. In appended pretesting, items are administered in a separately timed section at the end of an operational test battery. Two evaluations were conducted: one using a pretest unit consisting of a reading passage and the other using a pretest unit consisting of mathematics items, most of which were discrete. Sample sizes were: (1) 634 for the original pretesting group; (2) 294,637 for the operational group; (3) 1,007 for the re-pretest as operational group; and (4) 1,021 for the re-pretest as original group. In the ideal case, if items are pretested in exactly the context in which they will appear operationally, there should be no context effects. Results from the simulations in this paper showed some small-to-moderate negative effects on scores when misspecified parameters were used for item response theory scoring. Larger negative effects occurred at score points where fewer examinees score. Score bias could be smaller or larger under different conditions from those observed in t

Sign in with your free account to read, download EPUBs or listen.