A critical review on the reporting of surveys in transdisciplinary research: a case study in Information Systems.
Abstract
Variability of goals and evolving research methodologies are fundamental characteristics
of transdisciplinary research. This integration of research strategies from different fields
complicates the evaluation of transdisciplinary research since the variability of goals
drives variability of criteria and quality indicators. The aim of this research is to
investigate the implications of using research methods across disciplinary boundaries by
drilling down into the use of one research strategy in one research context (Information
Systems) and a related sub-context (Human-Computer Interaction). Surveys with
questionnaires as data-capturing tools were selected as an established research method
which is widely used in transdisciplinary research. Questionnaires are one of the most
established data capturing tools and yet the validity of questionnaire-based findings have
often been questioned. The main problem areas have been identified as the sampling of
the data, the questionnaire design and the interpretation of the results. This paper looks
into questionnaire reporting practices - an essential determinant in the validity and
reliability of survey-based research. The field of Information Systems and Human-
Computer Interaction has been chosen as the research context. Information Systems
research is by nature interdisciplinary in focusing on social and organisational issues
regarding the development and use of software in organisations. Human-Computer
Interaction studies address the challenges of making computers and computations useful,
usable, and universally accessible to humans. Both Information Systems and Human-
Computer Interaction studies address complex, heterogeneous, real-world problems,
thereby meeting the first criteria to be classified as transdisciplinary research. The
research design entails document analysis of papers presented at conferences in
Computer Science and Information Systems over a three-year period to identify trends in
the reporting of survey results, especially the questionnaire design.
Transdisciplinary research methodology facilitates the application of research methods
across fields. However, if the constraints of the method are not recognised the validity of
the results may be compromised in a plethora of ways. While fusion of methods are
encouraged on a theoretical level in transdisciplinary research the findings of this study
are a warning about the dangers of interdisciplinary application of research strategies
without due diligence in observing best practices in the parent discipline. The paper aims
to advance the discussion on research design and practice beyond disciplinary research
and should be of interest to researchers and practitioners who deal with multidisciplinary,
interdisciplinary, and transdisciplinary research.