The validation of a rating scale for the assessment of compositions in ESL

Abstract

This study aimed to develop and validate a rating scale for assessing English First Additional Language essays at Grade 12 level for the final National Senior Certificate examination. The importance of writing as a communicative skill is emphasised with the re-introduction of writing as Paper 3 of the English First Additional Language examination at the end of Grade 12 in South Africa. No empirical evidence, however, is available to support claims of validity for the current rating scale. The literature on the concept of validity and the process of validation was surveyed. Theoretical models and validation frameworks were evaluated to establish a theoretical base for the development and validation of a rating scale for assessing writing. The adopted framework was used to evaluate the adequacy of the current rating scale used for assessing Grade 12 writing in South Africa. The current scale was evaluated in terms of the degree to which it offers an appropriate means of assessing Grade 12 Level essay writing while adhering to requirements of the National Curriculum Statement. It was found lacking and the need for a new, validated rating scale was established. Various approaches to scale development were considered in consideration of factors that impact scores directly, viz. the type of rating scale, rater characteristics, scoring procedures and rater training. A new scale was developed and validated following an empirical procedure comprising four phases. The empirical process was based on an analysis of actual performances of Grade 12 English learner writing. A combination of quantitative and qualitative methods was used in each of the four phases to ensure the validity of the instrument. The outcome of this project was an empirically developed and validated multiple trait rating scale to assess Grade 12 essay writing. The proposed scale distinguishes five criteria assessed by means of a seven-point scale.