This article was downloaded by: [77.2.216.13] On: 02 July 2013, At: 01:02 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Theoretical Issues in Ergonomics Science Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ttie20 There are no qualitative methods – nor quantitative for that matter: the misleading rhetoric of the qualitative–quantitative argument a b Rodney Åsberg , Daniel Hummerdal & Sidney Dekker c a Department of Education, Gothenburg University, Gothenburg, Sweden b Centre de recherche sur les Risques et les Crises, École des Mines de Paris, Sophia-Antipolis, France c Key Centre for Ethics, Law, Justice and Governance, Griffith University, Brisbane, Australia Published online: 06 Jul 2011. To cite this article: Rodney Åsberg , Daniel Hummerdal & Sidney Dekker (2011): There are no qualitative methods – nor quantitative for that matter: the misleading rhetoric of the qualitative–quantitative argument, Theoretical Issues in Ergonomics Science, 12:5, 408-415 To link to this article: http://dx.doi.org/10.1080/1464536X.2011.559292 PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-andconditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material. Theoretical Issues in Ergonomics Science Vol. 12, No. 5, September–October 2011, 408–415 There are no qualitative methods – nor quantitative for that matter: the misleading rhetoric of the qualitative–quantitative argument Rodney Åsberga, Daniel Hummerdalb* and Sidney Dekkerc a Department of Education, Gothenburg University, Gothenburg, Sweden; bCentre de recherche sur les Risques et les Crises, École des Mines de Paris, Sophia-Antipolis, France; cKey Centre for Ethics, Law, Justice and Governance, Griffith University, Brisbane, Australia Downloaded by [77.2.216.13] at 01:02 02 July 2013 (Received 30 August 2010; final version received 19 January 2011) This article is about the comprehensive but meaningless rhetoric encircling the pseudo issue of quantitative–qualitative method. Two questions get clarification in this article. First, what does the term method mean, and second, what constraints should be attached to the term when moving into the theory and practice of science. Based on an examination of these two, we subsequently address the question of what the quantitative–qualitative argument is about. Our reasoning leads to the conclusion that the quantitative–qualitative argument should be removed from the agenda, so that significant and important a priori choices may be given due attention instead. Keywords: qualitative; quantitative; ontology; epistemology; critical theory; method 1. Introduction A decade ago, Xiao and Vicente (2000) pointed out that it is not very common for ergonomics research to spend much time on the epistemological foundations of its work. This article suggests that one of the reasons for a muted debate is the misleading rhetoric surrounding the pseudo issue of quantitative versus qualitative methods (Desnoyers 2004, Hancock and Szalma 2004, Kanis 2004). In both research and the debates surrounding it, the false choice between qualitative versus quantitative methods is often conflated with applied versus basic research. Advocates of the latter might question the truth value of findings by more interpretative methods of real-word settings. And those relying on qualitative methods can seek creditability through ecological appeals that implicitly or explicitly criticise the narrowness of quantitative approaches. In this article, we argue that the quantitative–qualitative argument masks more essential epistemological questions. By clarifying some of these, we hope that discussions about the status of one approach over another can be discarded so that more crucial issues can be tackled and progress can be made in our understanding of the multifaceted nature of ergonomic questions. *Corresponding author. Email: daniel.hummerdal@mines-paristech.fr ISSN 1464–536X online ß 2011 Taylor & Francis DOI: 10.1080/1464536X.2011.559292 http://www.informaworld.com Theoretical Issues in Ergonomics Science 409 Downloaded by [77.2.216.13] at 01:02 02 July 2013 2. What is ‘method’? Method, as a term, is derived from the Greek me´thodos (from meta: after and ho’dos: path) along a path. Hence, a method describes the steps along a path, how to go about it, an approach, or as defined in the English language: ‘A way or manner of doing something’ (The Advanced Learners Dictionary 1958, p. 792). When we say interview method, survey method and observation method, the term method stands for ‘the way one goes about it’ when collecting data through interviews, questionnaires and observation. Thus, these are different data collection processes. Here, we can already begin to see that a quantitative or qualitative way of doing things hardly exists. The problem is that it is not this sense of the word we use when we speak of hermeneutic ‘method’, phenomenological ‘method’, ethnographic ‘method’ or positivistic ‘approach’. The same data collection procedures (for e.g. interview or observation) can, of course, be used in several of these ‘methods’. Hence, what distinguishes them cannot be the method, in the sense of how data are collected, but rather what kind of knowledge is wanted, or even what knowledge is assumed to be or represent. This has to do with epistemological considerations. Such considerations are of a different nature, but this is seldom clarified. Often the use of the method concept is vague and broad, which creates confusion with respect to choices and positions to be taken in the research process (Holme and Solvang 1986, Patton 1990, Starrin and Svensson 1994, Hartman 1998, Denzin and Lincoln 2000). This confusion surfaces notably with da Silva and Wahlberg (1994, p. 41, t): The term ‘method’, which sometimes is used in an ambiguous and vague way, will hereafter be used as a synonym to ‘approach’, ‘perspective’, ‘strategy’ and ‘methodology’. But how does this reduce ambiguity? And how can method be synonymous with ‘methodology’? The obscurity of it all intensifies when they add the dichotomy quantitative–qualitative (da Silva and Wahlberg 1994, p. 51, t): As examples of specifically quantitative methods we find empirical observation, experiments and statistics. As examples of specifically qualitative methods there are participant observation, empathic understanding (the use of vivid realisation and empathy), in-depth interviews, hermeneutics, phenomenology, phenomenography, and ‘grounded theory’ as well as morphological or structural interpretation of myths . . . and existential analysis. The category of qualitative methods thus contains anything from phenomenology, hermeneutics and participant observation to in-depth interviews. The distinction quantitative–qualitative, however, usually tends to be grouped with the methodology issue. As such it is adopted as a natural starting point for reasoning around methods. For example, Hartman (1998, p. 14, t) writes in the preface: ‘In theory of methods one must make a distinction between two types of examinations; quantitative and qualitative’. The question is why we must make such a distinction. 3. Theory of science To clarify the departure points for later reasoning, we can divide the field of science into theoretical levels: from the level of ontology to the level of data. This is in order to underscore the specific questions and determinations to be addressed at each level. The borders between the levels are, however, not absolute. Both include discourse and determinations of an ontological as well as an epistemological nature. And the distinction idiographic – nomothetic may be traced partly to the level of method, and partly to the epistemological level. But the levels are still sufficiently clear in showing a way out of 410 R. Åsberg et al. Downloaded by [77.2.216.13] at 01:02 02 July 2013 unnecessary confusion. Below is a sketchy division: (1) Ontology, the question of ‘o’ntos’ (Greek) the nature of being, of reality: is about our perception of reality, our worldview. Idealism and materialism are perhaps the two most distinct approaches found here, existentialism and dialectics are two others. (2) Epistemology, the study of ‘Episteme’ (Greek; knowledge, knowing), the sources of knowledge and its validity: The main approaches are rationalism, that knowledge is based on reasoning and empiricism, that knowledge is based on observations. Kant’s studies (criticism) on the conditions and limitations of human knowledge, and his criticism of ‘pure reason’ are fundamental to this level. The realistic starting point, that there is a world independent of our consciousness and that it is possible to obtain knowledge about this, versus the idealistic, that the ideas, the mental representations are primary and knowledge relates only to these, have clear links to ontological determinations. Relativism, constructivism and structuralism represent different epistemological starting points, while hermeneutics, phenomenology and positivism represent different approaches to knowledge. (3.a) Methodology, the study of the ways we go about scientific investigations and how to establish scientific knowledge; critical distinctions include idiographic (looking for the uniqueness in individual cases) versus nomothetic (looking for law-like connections) approaches. Also in this category, we find inductive – deductive – abductive studies, or descriptive versus hypothesis- or theory-confirming. The so-called hypothetical–deductive method is therefore a determination of methodological nature. (3.b) Method, in the sense of a data collection process: when considering examples of data collection methods, such as interview, questionnaire or observation method, these distinctions relate to the approach taken (talk, write, observe) when collecting data on different phenomena. (4) Level of data, from data meaning facts, information, the plural of the Latin datum. Divisions exist between hard data, soft data, numerical (numbers), non-numeric (word) data or, if it should be necessary, quantitative and qualitative data. Data, which we are aware of but by deduction, mirrors thus, quantitative and qualitative characteristics of the phenomena or the objects we investigate. So the quantitative– qualitative distinction belongs to this level. But once the distinction has been put in place here, it becomes trivial. There is nothing more to say about it. The first two levels are of an a priori character and therefore polemical, i.e. consensus does not exist within the scientific community about these two. Therefore, there are crucial choices to be made about what kind of world (ontology) one is looking at, for what kind of knowledge (epistemology). Level three is more complementary. Particularly in the sense of the data collection process (3.b) as several methods can be used to establish empirical evidence for the same knowledge objective. Level 4, the data level, is totally complementary, since data in both numerical and non-numerical form are usually valid when we want to find out about something. It might be good if a fairly common terminology could be reached on the distinctions within the epistemological field. Not primarily to create a semantic straitjacket, and for that at all times use the ‘correct’ words, but to reduce some confusion of the terms and to surface critical choices to be made. Here is a suggestion: . The term ‘method’ is used for determinations on level 3, methodology. Theoretical Issues in Ergonomics Science 411 . The term knowledge approach is used for defining starting points associated with Level 2, epistemology. . The term ‘perception of reality’ or ‘ontological assumptions’ are used for distinctions at Level 1, the ontology. 4. Qualitative method? Let us analyse the consequence of taking the dichotomy quantitative–qualitative entered at the data level and add it to the term method. One example is found in this quote from Larsson (1994, p. 164, t) which seeks to clarify the significance of qualitative method: Downloaded by [77.2.216.13] at 01:02 02 July 2013 Qualitative method is about how to characterise something, how to represent it. The word comes from the Latin qualitas, which means ‘character, state, type’. Qualitative method is consequently systemised knowledge about how to go about to portraying the character of something. The phrase ‘Qualitative method is consequently systemised knowledge about how to go about portraying the character of something’ (emphasis added) is strange. The part that says ‘how to go about it’, is the very definition of method, after all. And the part that says ‘to depict the character of something’ denotes that method as qualitative. So the phrase actually means: ‘Qualitative method is systemised knowledge of qualitative method’, which of course is a tautology. But that is not all. Knowledge of how to go about data collection is comprehended by the concept of methodology (Logo: knowledge, learning (about) méthodos) but can hardly be captured by the term method. So it really says: Qualitative method is qualitative methodology! This is entirely meaningless. Qua’litas (Latin; quality) is about the properties, the nature of something. But this ‘something’ relates to the phenomena one is seeking knowledge about. It does not relate to the method, to the way one goes about collecting data, information or any other empirical evidence on the characteristics of certain phenomena. The method cannot be qualitative. Qua’ntitas (Latin; quantity) also refers to properties: properties in the form of, for example, volume, number and size. We sometimes speak of these as primary properties, properties of things that are independent of our perception of them. These are unlike the secondary characteristics (e.g. colour, smell, taste), which are not possessed by the phenomena themselves but which occur in our perception of something. But again, these are properties of the phenomenon about which we seek knowledge. Not characteristics of the method. The way we go about it, the method, cannot have quantity, or size or volume. The method itself cannot be quantitative. The dichotomy quantitative–qualitative method is thus an absurdity. Both qua’litas (quality) and qua’ntitas (quantity) refer to different characteristics of phenomena, which is what different types of data aim to illuminate. This is thus not the way we go about collecting – or for that matter analysing – such data. Methods (approaches, perspectives, etc.) cannot be quantitative or qualitative. Data, however, can be so. Data reflect, represent, assert or illustrate the qualitative and quantitative characteristics of phenomena. 5. Analyses cannot be quantitative or qualitative Sometimes there is hope that the dichotomy quantitative–qualitative could be saved with respect to what kind of analysis is carried out and that, if so, the question would be about something more sophisticated than the rather trivial fact that the data can include numbers 412 R. Åsberg et al. or words. For instance, Patel and Davidsson (1994) explain the difference between quantitative and qualitative research on the basis of what kind of analysis one carries out: Downloaded by [77.2.216.13] at 01:02 02 July 2013 Somewhat simplified one could say that the labels ‘quantitative’ and ‘qualitative’ refers to how one chooses to treat and analyse the information collected. By quantitative research one refers to such research that uses statistical methods to treat and analyse. By qualitative research one refers to such research that use verbal methods of analysis. (Patel and Davidsson 1994, p. 12, t) Our position is, of course, that it is not the analytical methods that are verbal, but rather the format of the data. Analytical methods can neither be quantitative nor qualitative – only data can. It is better, if at all necessary, to talk about the analysis of numerical data and analysis of non-numerical data. This is contained in the character of the data. Within the same study, data can take on the shape of words as well as numbers. If the quantitative–qualitative distinction is moved to the level where it belongs, the data level, the essential methodological issues need not be confusing. But if it has already been assumed that quantitative versus qualitative is an over-arching dichotomy under which everything else has to be subordinated, it is very likely that an almost endless rhetoric is maintained – taking lots of pages in methodological handbooks, essays and dissertations. But it is all about a pseudo choice. It can also stop when one realises that one ends up in obvious contradictions; that the arguments surrounding the differences in qualitative and quantitative method are entangled in absurdities. This is as much an absurdity as a convenient exit, however, a way out. The quantitative–qualitative argument should be linked to the data level, not to method. It is data that can be quantitative and qualitative, they can be compiled in the form of words and in terms of numbers. And indeed, ‘Numbers and words are both needed if we are to understand the world’. (Miles and Huberman 1994, p. 40). 6. The qualitative–quantitative smokescreen What if we start at the other end? This leads to contradictions and inconsistencies, which may prevent a genuine understanding of what the different a priori choices of epistemological and ontological nature mean. They get obscured by ‘The Gunsmoke in the quantitative/qualitative battlefield’ (Kvale 1989, p. 110). Hartman (1998) clearly sets out from a quantitative–qualitative dichotomisation and couples it to different epistemological starting points, to positivism versus hermeneutics: Quantitiative studies . . . are based on a positivistic theory of science . . . Qualitative studies . . . are characterised by the ambition to understand how people experience themselves and their surroundings. It is a method based on hermeneutics. (Hartman 1998, p. 14, t) Similarly, when we discuss different ‘principles and beliefs’ regarding ontology, epistemology and methodology under the label ‘qualitative researchers’ it amounts to something which ‘may be termed paradigm’: All qualitative researchers are . . . guided by highly abstract principles . . . These principles combine beliefs about ontology (What is the nature of reality?) . . . epistemology . . . and methodology (How do we know the world and gain knowledge of it?). These beliefs shape how the qualitative researcher sees the world and acts in it . . . The researcher is bound within a net . . . of epistemological and ontological premises . . . This net may be termed paradigm. (Denzin and Lincoln 1998, p. 26) But, of course, ‘these principles’ do not apply exclusively to how a qualitative researcher looks at and acts in the world, but applies to all researchers. There is no group, nor a paradigm, of qualitative a priori assumptions and another set of Theoretical Issues in Ergonomics Science 413 quantitative assumptions. There are different ontological and epistemological a priori assumptions that connect to the question for in what kind world (ontology) one is looking for what kind of knowledge (epistemology). But these can not in any meaningful way be associated with or subordinated by the dichotomy quantitative–qualitative. Consistent with an English-speaking tradition, Patton (1990, p. 37) claims that there is a longstanding ongoing epistemological debate on two global paradigms, on one hand a positivist that uses quantitative methodologies, and, in this case, a phenomenological paradigm that uses qualitative approaches: Downloaded by [77.2.216.13] at 01:02 02 July 2013 Philosophers of science and methodologists have been engaged in a long-standing epistemological debate about how best to conduct research. This debate has centered on the relative value of two fundamentally different and competing inquiry paradigms: (1) logical-positivism, which uses quantitative and experimental methods to test hypothetical-deductive generalisations, versus (2) phenomenological inquiry, using qualitative and naturalistic approaches to inductively and holistically understand human experience in context-specific settings. Even if Patton (1990, p. 38) confesses: ‘I prefer pragmatism to one sided paradigm allegiance’ he still discusses around a kind of qualitative paradigm and normatively attributes possibilities such as ‘holistic understanding’, ‘personal contact and insight’, and ‘flexibility’ (Patton 1990, p. 40, f). It seems to be clear that non-positivistic research, which basically means all other knowledge approaches except positivism, is considered qualitative research, which means not working with numeric data. And those who were engaged in that non-positivist research, what methods did they use? Well, they observed, described and recounted and interpreted. Which is also what positivists do. Positivism per se has no preference for quantitative data, but rather assumes that the knowable is equal to the observable. It emphasises empiricism rather than rationalism as the basis for scientific knowledge, ‘that imagination is subordinated observation’, as the father of positivism Comte (1979, p. 19) put it in 1844. 7. Conclusion In the end, the pseudo choice between qualitative and quantitative ‘methods’ may not be about that supposed choice at all. Instead, debates surrounding it invoke the question of the presumed scientific status of the ergonomic enterprise. Uncertainty about the provenance and objectivity of qualitative data seem to be born out of a lack of epistemological self-confidence of ergonomics as a science (Parasuraman et al. 2008). Not unlike the historical attempts to make psychology more into a hard science (from Freud to Watson and beyond), ergonomics may feel that a reliance on qualitative data mars its progress towards becoming a ‘normal’ science, in essential ways like the natural sciences. Most fields of scientific enquiry have wanted to emulate the ideal of physical science, where detached scientific observation and experimentation generate facts that are objective, time- and observation-independent, and value-free, and where a steady accumulation of such facts means ever more powerful science. To the question, then, why ergonomics has not lived up to this ideal, there are at least two possible answers (Flyvbjerg 2001): 7.1. The pre-paradigmatic answer Ergonomics is still a young science and there is nothing, in principle, to prevent it from becoming a ‘normal’ science in the future, other than a softening through too much 414 R. Åsberg et al. Downloaded by [77.2.216.13] at 01:02 02 July 2013 qualitative work. It has not been persuasively demonstrated, after all, that it is impossible for ergonomics to achieve the rigour and objectivity of the ‘hard’ sciences. It just takes more time, and more evidence, particularly of the quantitative kind. In the preparadigmatic view, it is hard science that lies at the core of any field’s scientific identity. Hard science functions as a kind of benchmark against which any other field’s epistemological confidence gets ranked. The problem, of course, is that even hard science has become relativised during the past decades through what has been called the ‘universality of hermeneutics’ (Feyerabend 1993). Hermeneutics, the study of interpretation, is no longer linked only to the study of humans, but to all sciences. Even natural sciences are now seen as historically conditioned and human constructed (Wallerstein 1996). They too must constantly re-assess what constitutes relevant facts, methods and theories, and what counts as ‘nature’. Interpretation in ergonomics (whether of qualitative or quantitative data) involves the taking (often implicitly) of a priori ontological positions that get obscured if the debate is merely cast as a choice between qualitative and quantitative. 7.2. The hermeneutic answer Giddens (1984) argues that the study of human activity must be based on people’s situational self-interpretation. Ergonomics is an activity in which humans study humans. Humans are self-reflecting actors, not objects in the natural world that do not answer back. For Giddens (1984), this involves a double hermeneutic. First, there are selfinterpretations among those people who are studied in ergonomics research. The second hermeneutic applies to the human researchers themselves, who are, of course, constituted in a particular context that offers a particular set of constructs and methods and techniques. Findings reported in any study, says Giddens (whether based on qualitative or quantitative data), are only as stable as their interpretations. They have no trans-historic truths that can progress towards a greater accumulation of facts or ‘science.’ It could be argued that none of this matters. After all, ergonomics is a field born out of pragmatic concerns, not out of a pre-occupation with ontology or epistemology. As long as ergonomics is able to produce tangible results, better displays, better work environments, we should perhaps not care about the extent to which it is ‘science,’ by whatever criteria. But when fundamentally different a priori assumptions are still brought together under the umbrella of a supposed choice between qualitative or quantitative research, there is a significant risk that essential and crucial questions are obscured. We might fail to see the world and its critical problems that need exploring. Instead we concentrate on the inside of the umbrella and are trapped in a futile rhetoric around a methodical pseudo question. References Comte, A., 1979. Om positivismen. Gothenburg, Sweden: Korpen. da Silva, A.B. and Wahlberg, V., 1994. Vetenskapsteoretisk grund för kvalitativ metod. In: B. Starrin and P.G. Svensson, eds. Kvalitativ metod och vetenskapsteori. Lund, Sweden: Studentlitteratur, 41–70. Denzin, N.K. and Lincoln, Y.S., 1998. The landscape of qualitative research. London: Sage. Denzin, N.K. and Lincoln, Y.S., eds., 2000. Handbook of qualitative research. London: Sage. Downloaded by [77.2.216.13] at 01:02 02 July 2013 Theoretical Issues in Ergonomics Science 415 Desnoyers, L., 2004. The role of qualitative methodology in ergonomics: a commentary. Theoretical Issues in Ergonomics Science, 5, 495–498. Feyerabend, P., 1993. Against method. 3rd ed. London: Verso Books. Flyvbjerg, B., 2001. Making social science matter: why social inquiry fails and how it can succeed again. Cambridge, UK: Cambridge University Press. Giddens, A., 1984. The constitution of society: outline of the theory of structuration. Berkeley, US: University of California Press. Hancock, P.A. and Szalma, J.L., 2004. On the relevance of qualitative methods for ergonomics. Theoretical Issues in Ergonomics Science, 5, 499–506. Hartman, J., 1998. Vetenskapligt tänkande. Lund, Sweden: Studentlitteratur. Holme, I.M. and Solvang, B.K., 1991. Forskningsmetodik. Om kvalitativa och kvantitativa metoder. Lund, Sweden: Studentlitteratur. Kanis, H., 2004. The quantitative–qualitative research dichotomy revisited. Theoretical Issues in Ergonomics Science, 5, 507–516. Kvale, S., 1989. Issues of validity in qualitative research. Lund, Sweden: Studentlitteratur. Larsson, S., 1994. Om kvalitetskriterier i kvalitativa studier. In: B. Starrin and P.G. Svensson, eds. Kvalitativ metod och vetenskapsteori. Lund, Sweden: Studentlitteratur, 163–189. Miles, M.B. and Huberman, A.M., 1994. Qualitative data analysis. Thousand Oaks: Sage. Parasuraman, R., Sheridan, T.B., and Wickens, C.D., 2008. Workload, and trust in automation: viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making, 2, 140–160. Patel, R. and Davidsson, B., 1994. Forskningsmetodikens grunder. Lund: Studentlitteratur. Patton, M.Q., 1990. Qualitative evaluation and research methods. Newbury Park: Sage Publications. Starrin, B. and Svensson, P.G., eds., 1994. Kvalitativ metod och vetenskapsteori. Lund, Sweden: Studentlitteratur. The Advanced Learners Dictionary, 1958. Oxford: Oxford University Press. Wallerstein, I., ed., 1996. Open the social sciences: report of the Gulbenkian Commission on the restructuring of the social sciences. Standford, CA: Stanford University Press. Xiao, Y. and Vicente, K.J., 2000. A framework for epistemological analysis in empirical (Laboratory and Field) studies. Human Factors, 42, 87–102. About the authors Rodney Åsberg was associate Professor at Gothenburg University and is currently involved in international education and aid projects. Daniel Hummerdal is PhD candidate at the Centre de recherche sur les Risques et les Crises at Ecole des Mines de Paris, France. Sidney Dekker (PhD, 1996, The Ohio State University) is Professor and Director of the Key Centre for Ethics, Law, Justice and Governance in Brisbane, Australia.