Investigation of the optimal response scale for personality measurement : computer–based testing

Abstract

To be able to use personality tests in the most reliable and valid manner there are many considerations to be taken into account. Variables such as the population used, the culture of the test-takers, the mode of administration, whether pencil-and-paper or computer-based testing procedures, familiarity with computers when using computer-based tests and the response format to be used when administering the personality questionnaire are but some of the considerations. Within South Africa it is that much more important to consider the mode of administration, whether pencil-and-paper tests or computer-based tests, as there are many individual groups who have been historically disadvantaged when it comes to the use of computers as a testing method. It is just as important to consider the response scale to be utilised when administering personality testing as this may influence the results obtained and can influence the reliability and validity of these results. The objective of this study was to determine which response scale, dichotomous or polytomous, was the best to use when conducting computer-based personality testing. The questionnaire that was utilised was the South African Personality Inventory (SAPI) questionnaire; however, only items from the Soft-Heartedness cluster were employed as the objective was not to test the questionnaire but to test the most reliable and valid response scale to be used in conjunction with the questionnaire. A convenience sampling approach was utilised and the questionnaire was administered to students who were available and able to take the test (N = 724). Descriptive statistics, factor analysis and Cronbach Alpha coefficients were used to analyse the data obtained. By means of a literature review it was found that computer-based testing held many advantages when utilised with personality testing and that it can assist in better testing in the future. Literature also showed that polytomous or Likert-type response scales yielded better results than that of a dichotomous or forced-choice response scale. The factor loadings on the two different response scales also yielded evidence that the polytomous response scale had three distinct factors that measure Soft-Heartedness, namely generosity, compassion and appreciation, which is supported by the findings of the South African Personality Inventory (SAPI), whereas the dichotomous response scale only yielded one overall factor. The results indicated that the polytomous scale is a more optimal option than the dichotomous scale. It was found that the polytomous response scale had much better inter-item-correlations and item-factor loadings than that of the dichotomous response scale. It was also found that the polytomous response scale had a much higher internal consistency than that of the dichotomous response scale since its alpha coefficient is higher than 0,70, whereas the internal consistency of the dichotomous response scale was lower than 0,70. This study’s limitations were also identified and recommendations were made in terms of future research and for utilisation within the South African Personality Inventory (SAPI) questionnaire.