Homogenizing responses to different survey questions on the same topic: Proposal of a scale homogenization method using a reference distribution

Abstract

Survey data are often used for comparison purposes, such as comparisons across nations or comparisons over time. To be effective, this would require equivalent questions and equivalent responses options to the questions. Yet there is a lot of variation in the response scales used, which, for example, differ in the number of response options used and the labeling of these options. This is the case in happiness research, and as a result most of the research data in this field is incomparable. Several methods have been proposed to transform ratings on verbal response scales to a common numerical scale, typically ranging from 0 to 10. In this paper we give an overview of the progress made in those Scale Homogenization methods over time. We describe two early methods: Linear Stretch and the Semantic Judgement of Fixed Word Value Method. Next we discuss the Semantic Judgement of Word Value in Context Method in more detail. Based on these discussions we propose a new Reference Distribution Method. We apply the Semantic Judgement of Word Value in Context and the Reference Distribution Methods to data on happiness in The Netherlands for the years 1989–2009. We show that the Reference Distribution Method produces comparable time series on different questions and that it allows discontinuities in data to be corrected.