Towards designing an artefact evaluation strategy for human factors engineering: a lean implementation model case study

Abstract

Applying scientific methods to the evaluation of design science research artefacts is necessary to recognise the design process as design science research. Prior work has reported on this crucial evaluation component; however, limited information and guidance are available on the practices that should be followed. In this study, the framework for evaluation in design science and the elaborated action design research method in the design science research paradigm were used to develop an alternative evaluation strategy for a human factors engineering artefact. A different scientific method was used to design an evaluation episode for each of the elaborated action design research iterations: (1) a gap analysis, (2) a systematic literature review, (3) an applied thematic analysis, (4) a design requirements traceability matrix, and (5) the Delphi technique. The validity of the research design was proven using design science research guidelines, action design research principles, and a research validation matrix. This evaluation strategy indicated how the strategic use of different kinds of scientific evaluations assisted in establishing the quality of the knowledge delivered by the design science process. This study has contributed to the field of (human factor) engineering by providing a pragmatic approach to solving abstract, people-related problems in industryDie toepas van wetenskaplike metodes vir die evaluering van ontwerp navorsing artefakte is noodsaaklik om die ontwerpsproses as ʼn wetenskap te erken. Vorige werk in die verband is al publiseer, maar beperkte inligting en riglyne is beskikbaar. Hierdie artikel gebruik die riglyn vir evaluasie in ontwerpswetenskap en die uitgebreide aksie-ontwerp navorsingsmetode binne die ontwerpsnavorsing paradigma om ʼn alternatiewe strategie te onwikkel waarmee ʼn menslike faktore ingenieursartefak evalueer kan word. ʼn Ander wetenskaplike metode is gebruik om ʼn evaluasie episode te ontwerp vir elkeen van die uitgebreide aksie-ontwerp navorsingsmetode iterasies: (1) ʼn gapingsanalise, (2) ʼn sistematiese literatuurstudie, (3) ʼn toegepaste tematiese analise, (4) ʼn ontwerpvereiste naspeurbaarheidsmatriks en (5) die Delphitegniek. Die geldigheid van die navorsingsontwerp is bewys deur ontwerpswetenksap navorsingsriglyne te gebruik, aksie-ontwerp navorsingsbeginsels en ʼn navorsingvalidasie matriks. Die evaluasie strategie dui aan hoe die strategiese gebruik van verskillende wetenskaplike evaluasies bygedra het tot die bepaling van die gehalte van die kennis wat die navorsingsontwerp proses gelewer het. Die studie dra by tot die veld van (menslike faktor) ingenieurswese deur ʼn pragmatiese benadering tot die oplos van abstrakte, mense-verwante probleme in die industrie te verskaf