Pertinence and alignment of educational level requirements in assessment : the case of chartered accountancy programmes in South Africa
Abstract
It has been shown that accounting graduates need a broad set of skills for entry into the profession. Students have to be able to apply knowledge in the workplace instead of merely accumulating knowledge during their years of study. The accounting profession requires critical thinking and application of knowledge, but students often struggle to perform at higher cognitive levels. Higher education requires that students should be tested for higher-order cognitive skills, yet the majority of questions often only test the lower order skills, focusing on recall of facts instead of processing skills. These skills should be reflected in the learning outcomes. Learning outcomes should include the analysis, synthesis and evaluation levels of Bloom’s Taxonomy, as these encourage critical thinking. Learning outcomes in the training of South African Chartered Accountants (CA(SA)) are, however, also influenced by the competency framework of the South African Institute of Chartered Accountants (SAICA), as this prescribes curricular content which is much more specific regarding the knowledge expected of the candidates. Other stakeholders in this process include the Council on Higher Education (CHE) and the policies and guidelines of the universities themselves.
Learning outcomes also serve as the starting point of constructive alignment, and constructive alignment can be successful only if the learning outcomes are stated clearly and precisely. Constructive alignment implies that the whole process, including the module outcomes, learning outcomes, teaching activities and assessments, are aligned and follow one on the other. Previous research has shown that modules which are constructively aligned improves student learning. The alignment of the learning outcomes with the assessments is the basic principle of constructive alignment. Constructive alignment, therefore, requires learning outcomes to be consistent with how the content of the module is assessed. Despite the importance of constructive alignment, educators often do not pay enough attention to the alignment of their modules, and even professional accounting bodies do not always align their stated learning outcomes and assessments.
The aim of this study was threefold: i) to provide an understanding of the process of module development in CA programmes at South African universities, ii) to evaluate the cognitive levels of the stated learning outcomes and the summative assessments of these universities, and iii) to evaluate the constructive alignment within MAF modules at SAICA-accredited universities.
The study aimed to provide a framework explaining the process, as well as the institutional frameworks and educational-level descriptors that influence the elements within the process. This will ensure that the elements of the process are at the appropriate educational levels. Furthermore, given the apparent focus on lower-level cognitive skills in learning outcomes, this study aimed to investigate the cognitive level of the knowledge-related learning outcomes of Management Accounting and Financial Management (MAF) modules presented in professional accounting programmes at South African SAICA-accredited universities. A total of 1 838 learning outcomes and 418 assessment items of six South African SAICA-accredited universities were analysed and categorised according to the cognitive levels of the revised Bloom’s Taxonomy. The data were analysed in four ways: (i) An average cognitive level was calculated based on the revised Bloom’s Taxonomy, (ii) the distributions between cognitive levels were established, (iii) the distributions of the Bloom levels were compared for the academic year levels and for Management Accounting and Financial Management, and (iv) the frequency of the verbs used in the learning outcomes was determined. A framework for the interpretation of the revised Bloom’s Taxonomy for the discipline of MAF, including the level, typical knowledge and method requirements, and the typical action verbs used for the specific level, was developed. The coding categories were labelled as: 1) remember, 2) understand, 3) apply, 4) analyse, 5) evaluate, and 6) create. The cognitive levels of the learning outcomes and assessment items were used to test the alignment between learning outcomes and assessment items in MAF modules in CA programmes.
The results varied across universities and showed that, although there is a shift in the cognitive levels of skills required from students as they progress from second year to fourth year, learning outcomes seem to be at too low cognitive levels of learning. The lower-order skills (remembering and understanding) and application of knowledge dominated the learning outcomes of most universities. This is not in line with the expectations of the industry which expects critical thinking and other higher-order cognitive skills from candidates.
The results also showed little alignment between the stated learning outcomes and assessment items of the MAF modules. The assessments were at higher cognitive levels than the learning outcomes. The university assessments were also more closely aligned to the learning outcomes of SAICA than to the university’s learning outcomes. This implies that universities tend to focus more on the outcomes provided in the competency framework of SAICA in setting their assessments than on university learning outcomes.
The study offers some insights into the educational-level requirements applicable to modules in CA programmes at South African universities and the application of the revised Bloom’s Taxonomy in the subject area of MAF. It also provides a framework that can be utilised for the development and classification of learning outcomes and assessments