Towards standardisation: a comparison of two versions of an academic literacy test

Abstract

All versions of a standardised test should be at similar difficulty levels. In this article, we investigate whether two versions of TAG ("Toets van Akademiese Geletterdheidsvlakke") are valid and whether they are at the same difficulty level. A group of students wrote two versions of the test within a ten-week period. We first investigated their validity in terms of sampling, scoring and construct evidence. Before comparing the tests, we checked whether the classes the study population attended had any effect on the results of the second test. We then compared the scores of the tests by means of a Rasch analysis, an equipercentile measure and a Bland-Altman plot. Test 2 proved to be easier than Test 1. Various factors may have contributed to this, and although it is hard to achieve in practice, it was clear that further work is required to ensure that TAG tests are at more or less similar difficulty levels.