'n Evaluering van bestaande metodes en modelle vir die moderering en gelykstelling van eksamenstandaarde met spesiale verwysing na die gebruik van itembanktoetse

Abstract

As a result of historical circumstances and socio-political developments, nine different examination bodies in the R.S.A. set their own matriculation papers each year. The following questions need to be answered: *Do the examination papers represent the same standard of measurement? *How can it be determined whether they do in fact represent the same standard or not? *If the examination papers are not at the same standard, how can the examination marks of the pupils of the different examination bodies be equated or made comparable? The problem of equating standards has three facets: equating standards (i) between different examination bodies; (ii) between different subjects, and (iii) between consecutive years. This study was mainly concerned with the first of these facets. The aims of the present study were: *to evaluate methods and models of equating standards or making standards comparable *to evaluate methods of determining cut-off scores *to evaluate the use of item bank tests as external criteria for setting standards *to evaluate the use of a well-defined norms group for scaling the marks of all groups of pupils to a common scale A review of literature relevant to these issues was undertaken. An empirical investigation was conducted by applying an item bank test (multiple-choice questions) during the final matriculation examinations in Physical Science Higher Grade. A norms group was drawn by taking 16,7% of the candidates of each of six departments of education. The mean scores and standard deviations of the pupils in the norms group (in the anchor test and year-end examination papers) were used as a standard for the linear scaling of the scores of pupils of each department. Descriptive and inferential statistics were calculated. It was found from the literature that cut-off scores should be determined by subject specialists and by the utilization of objective methods. The equating of standards is necessary since it is impossible to set different examination papers of equal difficulty. If the conditions for equating cannot be met, the scores should be made comparable. Methods of equating scores can be classified into two groups: (i) those related to item response theory, and (ii) those related to classical methods. It is more suitable to equate tests consisting of multiple-choice items by means of methods related to item response theory, while it is more suitable to equate the conventional examination papers by means of classical methods. The scores of all pupils were transformed by linear methods to the scale of the norms group. It was found that the examination standards of some departments were too high while others were too low. The reliability and validity coefficients for the item bank tests were highly satisfactory. From the results of a factor analysis of groups of items, it can be concluded that the item bank tests have high construct validity. Finally, it was shown in this study that if national cut-off scores (for the norms group) are given (as a result of decisions made by subject specialists) valid cut-off scores for each department can be determined objectively by using an item bank test.