The current issue and full text archive of this journal is available on Emerald Insight at: www.emeraldinsight.com/0969-6474.htm TLO 26,5 Investigating unlearning and forgetting in organizations Research methods, designs and implications Annette Kluge and Arnulf Sebastian Schüffler 518 Received 16 September 2018 Revised 29 January 2019 13 March 2019 22 March 2019 23 April 2019 Accepted 6 May 2019 Work, Organisational and Business Psychology, Ruhr-Universität Bochum, Fakultät für Psychologie, Bochum, Germany, and Christof Thim, Jennifer Haase and Norbert Gronau Chair of Business Information Systems and Processes, Universitat Potsdam, Potsdam, Germany Abstract Purpose – Insight has grown that for an organization to learn and change successfully, forgetting and unlearning are required. The purpose of this paper is to summarize the relevant existing body of empirical research on forgetting and unlearning, to encourage research using a greater variety of methods and to contribute to a more complementary body of empirical work by using designs and instruments with a stronger reference to previous studies. Design/methodology/approach – As the number of theoretical papers clearly exceeds the number of empirical papers, the present paper deals with the main insights based on the empirical state of research on unlearning and forgetting. So far, these empirical results have shown relationships between unlearning and other organizational outcomes such as innovation on an organizational level, but many of the other proposed relationships have not been investigated. The authors presents suggestion to apply a larger variety of qualitative, quantitative and mixed methods in organizational research. Findings – Unlearning and forgetting research can benefit both from more diverse theoretical questions addressed in research and from a more complementary body of empirical work that applies methods, designs and instruments that refer to previous research designs and results. To understand and manage unlearning and forgetting, empirical work should relate to and expand upon previous empirical work to form a more coherent understanding of empirical results. Originality/value – The paper presents a variety of research designs and methods that can be applied within the research context of understanding the nature of organizational forgetting and unlearning. Additionally, it illustrates the potential for different methods, such as experience sampling methods, which capture the temporal aspects of forgetting and unlearning. Keywords Mixed-methods, Research design, Longitudinal studies, Experience sampling, Correlational designs, Quasi-experimental and experimental designs Paper type Conceptual paper The Learning Organization Vol. 26 No. 5, 2019 pp. 518-533 © Emerald Publishing Limited 0969-6474 DOI 10.1108/TLO-09-2018-0146 1. Introduction Insight has grown that for an organization to learn and change successfully, forgetting and unlearning are required, in addition to knowledge acquisition and dissemination (Grisold et al., 2017; Grisold and Kaiser, 2017; Fiol and O’Connor, 2017a, 2017b; Morais-Storz and Nguyen, 2017; Nguyen et al., 2018). While the term “organizational unlearning” was The research was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) with grant number KL2207/6-1, and GR 1846/21-1. introduced almost as early as the term “organizational learning” appeared in management research (Hedberg, 1981; Nystrom and Starbuck, 1984; Howells and Scholderer, 2016), the term “organizational forgetting” only began to crop up in the business and management literature some decades later, at the turn of the millennium (Argote, 2013; Easterby-Smith and Lyles, 2003, 2011; Martin de Holan et al., 2004; Martin de Holan and Phillips, 2004; Martin de Holan, 2011). Unlearning has been defined as discarding and replacing old routines (Huber, 1991; Tsang and Zahra, 2008), while forgetting has been defined as reducing the influence of old knowledge on cognitive and behavioral processes (Grisold et al., 2017; Kluge and Gronau, 2018), e.g. by ceasing to use knowledge (Hislop et al., 2014). However, an imbalance has emerged between the number of theoretical papers and the empirical testing thereof: in the organizational unlearning and forgetting literature, theories have dominated over empirical evidence (Kluge and Gronau, 2018). While some empirical studies have been conducted, these rather appear to stand in isolation. The foundation of this paper is a recently published review (Kluge and Gronau, 2018) describing the state of the art of theoretical concepts of intentional forgetting and unlearning in organizations, which was conducted in 2018 based on the guidelines of Tranfield et al. (2003). Leading electronic databases were used for the search, including peer-reviewed publications, conference proceedings and internet sources listed in GoogleScholar, PsycArticles, PsyINFO and Psyndex (via EBSCO) using the following keywords: organis(z) ational forgetting, intentional forgetting in organis(z)ations, organis(z)ational unlearning, organis(z)ational ignorance, knowledge management and forgetting and managing forgetting. Altogether, 246 publications were found. The 40 publications reviewed in Kluge and Gronau (2018) were included by examining the abstracts and in-depth reviews to identify core contributions. For the present paper, an additional search was conducted using the terms “organis/ztional unlearning/forgetting þ empirical” or “organis(z)ational unlearning/ forgetting þ study”. In total, 15 scientifically sound empirical studies (in addition to the 40 reviewed earlier), which were conducted in relevant organizational settings and published in scholarly journals, were identified and included in the present paper. The purpose of this paper is not to re-assess the number of theoretical concepts, but rather to illustrate several options for conducting unlearning and forgetting research and to encourage more empirical studies. We wish to emphasize that unlearning and forgetting research can benefit from a more complementary body of empirical work that applies methods, designs and instruments that refer to previous research designs and results. To achieve a deeper understanding of unlearning and forgetting, empirical work should relate to and expand upon previous empirical work to form a more coherent understanding of empirical results. 2. Theoretical background/foundation In the present paper, we refer both to the term unlearning and to the term forgetting. From a theoretical and conceptual perspective, most authors have agreed on the following definitions: unlearning (Hedberg, 1981; Huber, 1991; Tsang and Zahra, 2008; Reese, 2017; Visser, 2017; Tsang, 2017, Fiol and O’Connor, 2017a, 2017b; Starbuck, 2017) means discarding and replacing old routines (Huber, 1991) and is assumed to support the objective to install new routines (Tsang and Zahra, 2008). Unlearning of routines, which no longer serve the organizational objectives is required to successfully implement new routines, which do support the organizational goals in the present and future (Ellwart and Kluge, 2019). Forgetting refers to the facilitation of change, especially, when current knowledge is perceived as an obstruction and a competitor to new knowledge (Martin de Holan, 2011). Investigating unlearning and forgetting 519 TLO 26,5 520 During forgetting, managers work to forget established knowledge that was or is perceived to be a barrier to increased organizational effectiveness (Martin de Holan and Phillips, 2004). Forgetting in organizations involves processes that deliberately impede the recall of certain organizational memory items; to support an organization’s changed strategic goal achievement, these memory items and information elements are no longer provided in the case of a certain query (Kluge and Gronau, 2018). The aim of forgetting is to reduce the influence of old knowledge (Grisold et al., 2017) and to stop old knowledge from being used (Hislop et al., 2014). Nevertheless, in the reviewed studies, the deployment and operationalization of these terms are quite diverse. From a methodological perspective, the overall plan of empirical research is termed the research strategy. The strategy includes the research design and research method. The research design encompasses the concrete plan to test a hypothesis or to answer a research question. A research method is the choice of a concrete manner of data collection to implement the overall plan. Austin et al. (2002) and Scandura and Williams (2000) cluster aspects of a research design into the general setting (e.g. laboratory, field and simulation), study design (e.g. passive observation, experiment, case study and archival) and temporal aspects (e.g. cross-sectional, longitudinal and cohort). All of these research designs can include qualitative and quantitative research methods (Stone-Romero et al., 1995). Qualitative research (e.g. action research, archival data, case study, document interpretation, ethnography, grounded theory and interviewing, Aguinis et al., 2009) yields non-numerical data such as observations or personal accounts of experiences (Pistrang and Barker, 2012; Zedeck, 2014). Quantitative research (e.g. reaction times, tests, questionnaires, performance measures and log file data) relies on measuring variables using a numerical system with the aim of analyzing the measurements through the use of statistical methods (Zedeck, 2014). In Sections 3, we give examples of empirical research on organizational unlearning and forgetting. We group the work identified (see above) based on the distinction between individual, team and organization level analysis and on the type of method used. 3. Empirical research on organizational unlearning and forgetting 3.1 Survey method (using questionnaires) We start by presenting empirical research related to survey methods using questionnaires at different levels of analysis: the individual, team and organizational level. On an individual level, Gutiérrez et al. (2015) explored the influence of unlearning on the acquisition and assimilation of knowledge (by conducting questionnaires with 55 doctors and 62 nurses), the influence of acquisition and assimilation and how acquisition and assimilation can help home care units to align technology and physician-patient knowledge. Becker (2010) studied issues identified as potential influencers of unlearning. The authors developed a survey, which they administered in an Australian corporation (N = 189) that was undergoing large-scale change because of the implementation of an enterprise information system. Based on the findings, the following factors that are relevant to the unlearning process during times of change were identified: understanding the need for change, the level of organizational support and training, assessment of the change, positive experience and informal support, the organization’s history of change, individuals’ prior outlooks and individuals’ feelings and expectations. On the team level, Akgün et al. (2006) investigated unlearning as changes in beliefs and routines during team-based projects in new product development teams. To test the antecedents and consequences of a team unlearning model, 319 teams were investigated and the data were analyzed using structural equation modeling. The results showed that team crisis and anxiety have a direct impact on team unlearning; environmental turbulence also has a direct impact on team crisis, anxiety and team unlearning. Finally, after team beliefs and project routines have changed, implementing new knowledge or information positively affects new product success. On the organizational level, several studies have been conducted, which are additionally grouped according to higher-order themes and topics. A study by Becker et al. (2006), addressed the question of “who unlearns?” and showed that larger organizations give far more consideration to unlearning than do smaller organizations. Organizations with a high labor turnover focus less on unlearning than those with a more stable workforce. A study by Cegarra-Navarro and Moya (2005), focused on the relationship between unlearning on the individual and group level and organizational outcomes. The authors used structural equation modeling to test hypotheses on, for example, the relation between individual unlearning, group unlearning with respect to human capital and performance. The results indicated that “intellectual capital” depends on the unlearning among members of the company. Unlearning as a precondition for organizational outcomes was addressed by the following studies: Cegarra-Navarro and Dewhurst (2006) presented a structural equation model, which was validated through an empirical investigation of 139 small- and medium-sized enterprises (SMEs) in the Spanish optometry sector. The results showed that companies need to support unlearning as a first step; otherwise, unlearning does not have any significant effect on the creation of relational capital. Leal-Rodríguez et al. (2015) tested the mediating role of innovation outcomes on the relationship between organizational unlearning and overall performance by applying a conditional process model (structural equation modeling) using data from 45 firms from the Spanish automotive components manufacturing sector. They found that innovation outcomes partially mediate the influence of organizational unlearning on overall performance. Yang et al. (2014) investigated 193 sample firms from high-technology industries and showed that the change dimension of unlearning (as an internal process) positively affects radical innovation, whereas the forgetting dimension (forgetting by external partners) has a negative effect. Organizational unlearning was defined in terms of changes in routines and beliefs. The “forgetting” dimension mostly affects external suppliers and customers because these parties will lose the familiarity with or expectations of the firm in question that have accumulated for years and is assumed to be an outside-in dimension. Furthermore, work-life balance has been investigated as an outcome of unlearning. Cegarra-Navarro et al. (2016) argued that an unlearning context that fosters the updating of knowledge is likely to be essential for SMEs that are trying to implement a culture of worklife balance. The authors investigated 229 SMEs in the Spanish metal industry. The results showed that to strengthen a work-life balance culture and innovation-related outcomes, SMEs must meet the challenge of developing an unlearning context to counteract the negative effects of outdated knowledge in relevant areas and to facilitate the replacement of out-of-date or obsolete knowledge. Martelo-Landroguez et al. (2018) described in their research model how the complementary roles of absorptive capacity (direct effect) and the fostering of an organizational unlearning context (moderating effect) affect green customer capital within the Spanish automotive component manufacturing sector. Based on a survey (with 112 usable surveys) and path modeling, the empirical results showed that to create green Investigating unlearning and forgetting 521 TLO 26,5 522 customer capital, companies should absorb new knowledge and build a context of organizational unlearning. Finally, Wong et al. (2018) reported on a study that aimed to examine the factors affecting contractors’ organizational readiness for more extensive use of prefabrication in projects. As a conceptual framework that depicts the interrelationships among organizational readiness, unlearning and counterknowledge were proposed. Data were collected from a survey conducted in Australia. The results indicated that unlearning is positively correlated with organizational readiness. The fact that unlearning is an important mediator was demonstrated by CegarraNavarro et al. (2011), who examined the relationship between the exploration and exploitation of knowledge within an unlearning context and the effects of these two factors on the improvement in the performance of 229 SMEs in the Spanish metal sector. The results revealed that the effects of the exploration and exploitation of knowledge on organizational performance are mediated through an unlearning context. Huang et al. (2018) examined how organizational forgetting affects innovation performance under consideration of environmental turbulence as a moderating factor of the analysis framework. Based on a survey sample of 320 Chinese companies, the study validated a moderated mediating model of organizational forgetting. According to the findings, organizational forgetting is a critical determinant for improving innovation performance. In addition, organizational forgetting cannot promote an organization’s innovation performance without absorptive capacity and the mediating effect of absorptive capacity is more positive when turbulence is stronger. In summary, the main focus of unlearning and forgetting research using surveys lies on the organizational level and this research has mainly addressed the relationship between unlearning as a prerequisite and organizational outcomes such as radical innovation and culture, and the mediating role between innovation endeavors and organizational outcomes. Other results show that unlearning is itself a precondition for innovation and readiness for change. There are only isolated studies asking, which organizations unlearn and who in the organization is unlearning. Some research exists on the individual level, which was conducted during the implementation of new technology. On the team level, one study described the antecedents (crisis and anxiety) of team unlearning. As this research is descriptive and correlational in nature, none of the studies investigated what actually happens during unlearning, and how unlearning or forgetting can be effectively implemented. 3.2 Other methods Archival data plus formal theory was applied by Agrawal and Muthulingam (2015), who analyzed data on 2,732 quality improvement initiatives implemented by 295 vendors of a car manufacturer. They found that organizational forgetting affects quality gains obtained from learning by doing (autonomous learning) and from undertaking quality improvement initiatives (induced learning). To give examples of case studies, Fernandez and Sune (2009) used two qualitative case studies in higher education involving situations of organizational forgetting to derive propositions about the causes of forgetting. Usman et al. (2018) built mainly on social learning theory, using a single case study as research methodology and collecting data from 40 semi-structured interviews to understand how two key aspects of ethical leadership – accountability and honesty – facilitate the unlearning of destructive and inappropriate behaviors and practices. The goal of a study by Matsuo (2017) was to examine the managerial unlearning process upon promotion from senior manager to executive officer: analyzes of interview data on an individual level with 46 executive officers at medium-sized and large Japanese firms indicated that managers unlearned and learned their managerial skills in relation to “decision making”, “delegation and motivation” and “collecting information”. Specifically, decision-making skills switched from “short-term, analytic and partial” to “long-term, intuitive and holistic”. A study by Mehrizi and Lashkarbolouki (2016), constituted an exception, by applying a longitudinal research design (data used from the past six-eight years) and a mixed methods approach (observation, document analysis and formal interviews). Based on two longitudinal case studies, the authors proposed a process model that establishes four stages of business model unlearning, namely, “realizing,” “revitalizing,” “parallelizing” and “marginalizing.” They also discussed how unlearning dynamics help us to understand the importance of single- and double-loop unlearning, consider the double-faceted nature of business models and acknowledge the complex temporal dynamics of unlearning. Investigating unlearning and forgetting 523 4. Discussion of research methods and (potential) findings Tables I and II summarize categories of research designs (e.g. experimental, quasiexperimental and non-experimental), methods (e.g. qualitative, quantitative and mixed methods), strategies (e.g. formal theories/literature reviews, sample surveys, laboratory experiments, experimental simulation, field studies, field experiments or computer simulations) and the opportunities and challenges they bring. The summary is based on review articles by (in chronological order) Podsakoff and Dalton (1987), Stone-Romero et al. Research method Opportunities and challenges Case study In depth investigation of a single individual, event or other entity, e.g. to describe and understand the forgetting process of a single organizational unit, department and section Example: Fernandez and Sune (2009) þ Suited for capturing behaviors that were displayed in an authentic context þ Allows for intensive analysis of an issue þ The use of multiple case studies allows for more claims regarding generalizability Limited in the extent to which findings may be generalized Not well suited for maximizing generalizability with respect to populations Interviews A directed conversation in which a researcher intents to elicit specific information from an individual for research purposes, e.g. interviews with workers and managers on how they cope with the requirement of forgetting Example: Matsuo (2017) þ Capturing behaviors that have occurred in an authentic context Memory and self-serving biases might occur Reliability is a concern: more active participation in the situation, possible biases and impacts of personal judgments Analysis of archival records Information about past events and/or behaviors, that are stored in relative permanent form, e.g. books, journals, historical documents and other records, e.g. to understand the forgetting or fading of procedures, knowledge elements, that are not mentioned anymore and are removed from or exchanged in a document Example: Agrawal and Muthulingam (2015) þ Allows unobtrusive observation of human activity in a natural setting þ Effective in maximizing generalizability with respect to populations, enhancing precision in control/measurement of variables and/or capturing behaviors that have taken place in an authentic context Only past events are captured Causal inferences are more tentative than lab experiments Table I. Selection of qualitative research methods (case study and interviews) and archival data analysis adapted from and based on Turner et al. (2017) and Zedeck (2014) arranged according to aspects of intern and extern validity TLO 26,5 524 Table II. Selection of quantitative and formal methods adapted from and based on Turner et al. (2017) and Zedeck (2014) arranged according to aspects of intern and extern validity (Field)Surveys and Field Studies Study in which a group of subjects is selected from a population and some selected data are collected; collecting information on a specific topic in a relevant group or entity, in their natural environment; and more passive observation of relationships between variables, e.g. to understand the relationship between team and leadership variables and the support of perceived forgetting Examples: Akgün et al. (2006), Becker (2010), Cegarra-Navarro and Moya (2005), CegarraNavarro and Dewhurst (2006), Cegarra-Navarro et al. (2011), Cegarra-Navarro et al. (2016), Yang et al. (2014). Leal-Rodríguez et al. (2015), MarteloLandroguez et al. (2018), Huang et al. (2018) and Wong et al. (2018) þ Precision in control/measurement of variables and capturing behaviors that were displayed in an authentic context þ Already validated instruments can be used Only snapshot of current situation Possible memory biases Subjects can respond only to predefined items Challenges in extrapolation of findings to whole populations Causal relationship difficult to infer, only assumed relationships Field experiment Study outside the laboratory; subjects are not randomly selected and assigned to different conditions (independent variable); some active manipulation of variables, e.g. to investigate different intervention forms (e.g. workshop, trainings) to support forgetting þ Enhancing precision in control/measurement of variables and capturing behaviors that have occurred in an authentic context þ Incorporates mundane aspects of context Less options for experimental manipulation Possible confoundation with other variables, that are difficult to control for over a period of time Non-representative samples and settings Use of operational definitions of manipulation and measures of interest Lab experiment and experimental simulation Series of observations conducted under controlled conditions to study the relationship between predefined variables (independent and dependent variables). Includes random selection of participants and their random assignment to conditions; active manipulation of independent variable, e.g. to deliberately investigate microprocesses and cognitive processes of forgetting in teams and individuals Examples: Kluge et al. (2018), Schüffler et al. (2019) þ Suited for precision in control/measurement of variables þ Control over experimental manipulations þ Allows for causal inferences þ In case of simulations: capturing behaviors that have taken place in an authentic context Limited with respect to generalization Computational simulation Artificial creation of experimental data through the use of a mathematical or computer model to test the behavior or model under controlled conditions, e.g. to investigate forms of turn over or downsizing and organizational forgetting and renewal over a simulated period of time (e.g. decades) Examples: Jain and Kogut, 2014 and Bruderer and Singh (1996) þ Enhancing precision in control/ measurement of variables þ Effective in maximizing generalizability with respect to populations Limited with respect to in depth understanding Formal theory (mathematical) A model or set of rules used to understand and predict various behaviors in mathematical terms, e.g. to compare different forms of forgetting of different organizational structures, in combination with the comparison of different market conditions þ Enhances precision in control/measurement of variables and can be effective in maximizing generalizability with respect to populations Needs some empirical basis and data to be built on (1995), Scandura and Williams (2000), Austin et al. (2002), Aguinis et al. (2009), Cooper et al. (2012) and Aguinis et al. (2019). Table I describes the most commonly used qualitative methods and the use of archival data in organizational research (Aguinis et al., 2009; Turner et al., 2017) and illustrates possible applications for organizational forgetting and unlearning research. Examples of studies using archival data (Agrawal and Muthulingam, 2015), interviews (Matsuo, 2017) and case studies (Fernandez and Sune, 2009) were described above. Table II describes the most commonly used quantitative methods in organizational research in general (Aguinis et al., 2009; Turner et al., 2017) and illustrates possible applications for organizational forgetting and unlearning research. In Section 4.1, we elaborate in greater detail on methods that have been hitherto neglected in empirical unlearning and forgetting research. 4.1 Quasi-experimental design As examples of quasi-experimental designs are lacking to date, we give a hypothetical example of a quasi-experimental design. Moreover, we use the propositions by Martin de Holan (2011), who suggested that the amount and type of effort required to forget depend on the category of knowledge involved, and on the relationship between the new knowledge and the old knowledge (the distance between the new and old knowledge). Using a quasiexperimental design to test these hypotheses, two comparable organizational departments are required, which differ regarding the distance between the old and new knowledge (far versus near). Organizational members of both departments could rate the distance between the new and old knowledge and researchers could measure the rate or speed of forgetting and the speed of change in both departments over time. The results of the quasiexperimental design would then reveal assumed relationships between the independent variable (the distance between old and new knowledge) and the impact on the dependent variable (the speed of change). However, alternative explanations are difficult to rule out, as other variables which may differ, e.g. charismatic leadership or supportive group dynamics, could also serve as an explanation for the speed of change. 4.2 Randomized experimental design A randomized experimental design could either use a special- or a non-special-purpose setting (Stone-Romero, 2011) to investigate the influence of, for example, organizational actions as independent variables and their impact on unlearning and forgetting as dependent variables to measure effects on the organizational level. A special-purpose setting might be a laboratory setting that is designed as a production setting or shop floor or an industrial site that is used for experimental studies. Special-purpose settings cease to exist when research has been completed and are designed for intentional manipulation of the independent variable. For instance, a “learning factory” is a special-purpose setting with high physical and psychological fidelity. A study by Schüffler et al. (2019) demonstrated how a learning factory can be used to investigate the importance of eliminating retrieval cues for forgetting a knowledge-intensive multi-actor routine. In the controlled setting with four measurement times, it was shown that particular elements of a multi-actor routine are more difficult to forget if they have been well learned before, compared to less well-learned elements (Kluge et al., 2018). It follows that not all elements of a routine are forgotten at the same speed. Non-special-purpose settings (Stone-Romero, 2011), for instance, those used for field experiments, share similar challenges to those of quasi-experimental settings. They would, of course, include all organizational characteristics and their impact on unlearning and Investigating unlearning and forgetting 525 TLO 26,5 526 forgetting in parallel, such as organizational history, culture and values, human resource management practices, leadership, structure and technology (Cheung et al., 2017). If one wished to use a non-special-purpose setting to investigate, for example, the three phases of unlearning as proposed by Reese (2017), Phase 1: destabilization, crisis and mismatch; Phase 2: discarding, weathering and interruption; and Phase 3: experimenting, obsolescence and recovery, one could use two similar non-special-purpose settings, for instance, two production sites of one company in different countries, to investigate the impact of different leadership values that are displayed at these sites on workers’ and employees’ perceptions of the phases through which they have to go. 4.3 Computer simulations Computer simulations are model-based descriptions of the consequences of theoretical assumptions and side effects in a fast-forward mode; they allow for the observation of interdependencies and complex interactions between variables and their dynamics to investigate process aspects more closely (Runkel and McGrath, 1972; Turner et al., 2017; Zedeck, 2014). The results of a simulation conducted by Bruderer and Singh (1996) revealed that replacing inappropriate organizational routines helps in the quick discovery of a new, viable organizational form, which adapts better to a fast-changing environment. By using computer simulations, it is possible to observe extreme and unusual system states, which cannot be manipulated (for ethical reasons) in reality. Instead of direct observation, consequences can be modeled and inferred from the simulation results. Finally, several simulation runs can be implemented to vary system variables systematically in different combinations (Kluge and Schilling, 2004). For research on organizational forgetting, computer simulations could be used, for example, to model different forms of dynamic environments, several forms of interventions or organizational features that are assumed to support forgetting to observe the speed of forgetting and the success of change and adaptation in the environments (Jain and Kogut, 2014). 4.4 Mixed methods Finally, methods can be combined in mixed-methods approaches, which are based on the idea that the use of multiple, different research methods generates a better understanding of a given theory or phenomenon (Molina-Azorin et al., 2017; Turner et al., 2017); this can also be applied to research on unlearning and forgetting in organizations. As all methods have their limitations, a combination of different methods can compensate for the individual shortcomings of a single method alone. The integration of qualitative and quantitative methods as a mixed-methods approach in one study is an emerging trend, which matches the complexities of organizational phenomena (Molina-Azorin et al., 2017). Turner et al. (2017) offer a promising approach for a combination of different methods, namely, they developed a framework for mixed methods, e.g. the combination of archival methods, case studies, computer simulations, experimental simulations, field experiments, formal theory (mathematical, laboratory experiments and surveys, see Tables I and II) and provided several examples of benchmark studies using mixed methods. At the same time, they also pointed to challenges, e.g. the replication of findings, especially, when qualitative data are involved. Nevertheless, challenges depend on the study design and, for example, increase when mixed methods are applied to the same sample or organizational setting. 4.5 Summary of findings Taking into account the current state of the art of empirical research, from a content-related perspective, the above-cited findings can be summarized as follows: Individual- and team-level effects: not all elements of a routine are forgotten at the same speed. Particular elements of a multi-actor routine are more difficult to forget if they have been well learned before, compared to less well-learned elements (Kluge et al., 2018; Schüffler et al., 2019); individual unlearning of managers is discontinuous and occurs during the process of their promotion (Matsuo, 2017); ethical leadership supports individual unlearning (Usman et al., 2018); and crisis and anxiety are antecedents of team unlearning (Akgün et al., 2006). Organizational-level effects: replacing inappropriate organizational routines helps in the quick discovery of a new, viable organizational form, which adapts better to a fast-changing environment (Bruderer and Singh, 1996); unlearning is a precondition for relational capital (Cegarra-Navarro and Dewhurst, 2006); unlearning affects radical innovation (Yang et al., 2014); unlearning supports cultural change (Cegarra-Navarro et al., 2016); organizational forgetting supports quality improvement in autonomous learning (Agrawal and Muthulingam, 2015); unlearning has a positive relationship with organizational readiness (Wong et al., 2018); forgetting is a determinant for improving innovation in combination with absorptive capacity under the influence of turbulence (Huang et al., 2018). Mediator and moderator effects: the effects of exploration and exploitation of knowledge on performance are mediated by unlearning (Cegarra-Navarro et al., 2011); the relation between unlearning and performance is mediated by innovation outcomes (Leal-Rodríguez et al., 2015); and unlearning is a moderator of the relationship between absorptive capacity and creating green customer capital (Martelo-Landroguez et al., 2018). From a methodological perspective, the empirical state of the art is limited for several reasons: in relation to the large body of theoretical concepts, only a small number empirical studies exist, the studies seem to stand alone and the studies predominantly used one research method (cross-sectional survey data). What we can learn about unlearning and forgetting from these studies is limited to the conclusion that unlearning and forgetting matter as a predictor, mediator or moderator. However, what happens while managers, employees and workers are unlearning? Which organizational characteristics support or hinder unlearning and forgetting? To what extent does the technology used slow the unlearning process down? Can unlearning and forgetting to be managed? We assume that the understanding of organizational unlearning and forgetting can benefit from both a more coherent and interrelated empirical investigation and more diverse research in terms of research methods and strategies to foster the understanding of what happens in the unlearning and forgetting processes. As a suggestion, we see some innovative research potential in the development of smartphone use for online surveys. This Investigating unlearning and forgetting 527 TLO 26,5 528 overcomes some of the limitations of field studies (in terms of “snapshots”) and addresses the challenge of measuring the temporal aspects of forgetting and unlearning. In particular, experience sampling methods (ESM), in combination with more sophisticated statistical analysis such as multilevel analysis, render it possible to gather data over a longer time period (of forgetting or unlearning). ESM allows researchers to gather detailed data on organizational members’ daily experiences over time (Aguinis and Edwards, 2014). Moreover, it offers the potential to combine several approaches and methods of analysis, such as qualitative and quantitative methods and temporal aspects of forgetting such as cross-sectional and longitudinal designs. As such, ESM is able to acknowledge intra- and inter-individual forgetting and unlearning developments over time and reduces biases and errors, which are inherent in the global retrospective reporting of forgetting experiences. A further advantage lies in the possibility to study and capture the ongoing stream of forgetting behavior in its natural sequence and occurrence (instead of cross-sectionally). Finally, ESM data can be analyzed on an individual, team and organizational level (Fisher and To, 2012; Csikszentmihalyi and Larson, 2014; Uy et al., 2010). 5. Conclusion The outline of the current and existing empirical results on forgetting and unlearning showed that only a limited number of empirical studies exist. The majority of the studies used field surveys and cross-sectional designs, showing that unlearning and forgetting contributes to organizational outcomes. The large number of involved organizations is impressive and demonstrates the economic impact of unlearning and forgetting of human or relational capital (Cegarra-Navarro et al., 2011). Other more or less “stand-alone” studies show how managers unlearn and forget to reach the next management level (Matsuo, 2017) or how ethical leadership facilitates the unlearning of destructive behavior (Fernandez and Sune, 2009). Every empirical study summarized in the introduction makes a valuable and unique contribution to the field. Nevertheless, the big picture is still hard to grasp, as the samples selected, methods used, levels selected for analysis and designs are quite diverse. 5.1 Implications for theory Some theoretical implications drawn from this review of empirical results are as follows: we learned from the existing body of research that unlearning and forgetting matter. However, of course, this would also hold true for change or organizational development in general. From a theoretical point of view, empirical research could be more precise in differentiating between change, development and unlearning and forgetting. In many studies, the items used in several questionnaires seem to address change rather than unlearning or forgetting. This can also be observed for the distinction between unlearning and learning. In several studies, it seems to be implicitly assumed that if learning has taken place, unlearning must have been the cause. While this might indeed be the case, it has not yet been addressed and investigated. Further research could also clarify when unlearning and forgetting is necessary and essential and when learning and change is sufficient to fit the purpose of organizational adaptation. Is forgetting and unlearning essential for more radical change such as double-loop learning and episodic change, while learning and development is more relevant to continuous change? Under which conditions is unlearning and forgetting the only way in which an organization can adapt? Some studies demonstrate unlearning and forgetting in the development of managers or show that ethical leadership supports unlearning of destructive behavior. But what is the general role of leadership in unlearning? For instance, is transformational leadership as relevant for unlearning as it is important for change? Do managers need to unlearn first before their subordinates can unlearn? Can leadership actively support employees’ and workers’ unlearning in an organization? Further “blind spots” in research are the roles of structure and technology within organizations, which may either hinder or support unlearning and forgetting. Do organizational structures differ in their ability to support unlearning? Are agile and young organizations faster at unlearning and organizations with a long tradition and many hierarchies slower? What is the role of technology? To what extent might existing technology hinder unlearning because the routines that need to be unlearned are interwoven with technology that has not yet been replaced? Does the implementation of new technology accelerate unlearning and forgetting? A more systematic approach to the development of research questions, e.g. derived from a coherent theoretical framework that relates to the existing evidence, can be helpful to realize this endeavor. Finally, a practical outcome of further research has to be addressed if unlearning and forgetting can be managed. Are there evaluation studies of intervention techniques that accelerate unlearning and forgetting? Can forgetting and unlearning be managed in terms of their speed? 5.2 Implications for practice One practical implication drawn from the summary of existing studies is that there is a need for studies, which mutually refer to each other. When preparing the summary of results, we observed that while authors referred to many theoretical papers in their introductions, they did not refer to the existing empirical body of research. The reason for a particular research question was mainly driven by a theoretical and conceptual paper, rather than by an advancement of empirical results. It can be assumed that research will continue to be slow to advance if every study “reinvents the wheel” instead of building on existing research, e.g. by re-using questionnaires, by conducting replication studies etc. For example, a worthwhile endeavor could be to develop and validate a questionnaire that is used by several researchers in many branches and on different levels of analysis. A standardized survey or questionnaire instrument that is frequently used and becomes standard in the field of organizational unlearning and forgetting could help to greatly increase the number of studies and the empirical results. Further practical implications concern the aspect of what is measured and how it is measured. For example, the use of questionnaire data in a cross-sectional design is only one research method taken from the variety of methods introduced. The empirical field of organizational unlearning and forgetting is still mostly unexplored. The theoretical questions raised above give first ideas for additional research questions, which are worthy of investigation, e.g. the role of structure or technology. However, these questions might be better investigated by using field experiments, longitudinal designs or ESM to observe processes of organizational unlearning and forgetting. Moreover, laboratory experiments can also be useful, for example, to address the role of technology and how technologyembedded routines foster or hinder unlearning. As the suitability of each method depends on the specific theoretical question, different methods (other than cross-sectional questionnaire studies) need to be applied to address different theoretical questions. One practical solution to achieve a more comprehensive understanding of organizational unlearning and forgetting lies in mixed-methods approaches. With respect to limitations, as pointed out by Turner et al. (2017), all methods include strengths and weaknesses regarding internal and external validity, the precision of control and measurement, the authenticity of context and the generalizability of findings. Investigating unlearning and forgetting 529 TLO 26,5 530 In this respect, the present paper tries to encourage more empirical research to enable us to learn together and from each other, with the aim of deepening the insights into organizational unlearning and forgetting, and bringing organizational forgetting research to the next level. References Agrawal, A. and Muthulingam, S. (2015), “Does organizational forgetting affect vendor quality performance? An empirical investigation”, Manufacturing and Service Operations Management, Vol. 17 No. 3, pp. 350-367. Aguinis, H. and Edwards, J.R. (2014), “Methodological wishes for the next decade and how to make wishes come true”, Journal of Management Studies, Vol. 51 No. 1, pp. 143-174. Aguinis, H., Ramani, R.S. and Villamor, I. (2019), “The first 20 years of organizational research methods: trajectory, impact, and predictions for the future”, Organizational Research Methods, pp. 1-27, doi: 10.1177/1094428118786564. Aguinis, H., Pierce, C.A., Bosco, F.A. and Muslin, I.S. (2009), “First decade of organizational research methods: trends in design, measurement, and data-analysis topics”, Organizational Research Methods, Vol. 12 No. 1, pp. 69-112. Akgün, A.E., Lynn, G.S. and Byrne, J.C. (2006), “Antecedents and consequences of unlearning in new product development teams”, Journal of Product Innovation Management, Vol. 23 No. 1, pp. 73-88. Argote, L. (2013), “Organizational forgetting”, in Argote, L. (Ed.), Organizational Learning: Creating, Retaining and Transferring Knowledge, Springer, New York, NY, pp. 57-84. Austin, J.T., Scherbaum, C.A. and Mahlman, R.A. (2002), “History of research methods in industrial and organizational psychology: measurement, design, analysis”, in Rogelberg, S.G. (Ed.), Blackwell Handbooks of Research Methods in Psychology: Handbook of Research Methods in Industrial and Organizational Psychology, Blackwell Publishing, Malden, pp. 3-33. Becker, K. (2010), “Facilitating unlearning during implementation of new technology”, Journal of Organizational Change Management, Vol. 23 No. 3, pp. 251-268. Becker, K.L., Hyland, P. and Acutt, B. (2006), “Considering unlearning in HRD practices: an Australian study”, Journal of European Industrial Training, Vol. 8 No. 30, pp. 608-621. Bruderer, E. and Singh, J.V. (1996), “Organizational evolution, learning, and selection: a geneticalgorithm-based model”, Academy of Management Journal, Vol. 39 No. 5, pp. 1322-1349. Cegarra-Navarro, J.G. and Moya, B.R. (2005), “Business performance management and unlearning process”, Knowledge and Process Management, Vol. 12 No. 3, pp. 161-170. Cegarra-Navarro, J.G. and Dewhurst, F.W. (2006), “Linking shared organisational context and relational capital through unlearning: an initial empirical investigation in SMEs”, The Learning Organization, Vol. 13 No. 1, pp. 49-62. Cegarra-Navarro, J.G., Sánchez-Vidal, M.E. and Cegarra-Leiva, D. (2011), “Balancing exploration and exploitation of knowledge through an unlearning context: an empirical investigation in SMEs”, Management Decision, Vol. 49 No. 7, pp. 1099-1119. Cegarra-Navarro, J.G., Sánchez-Vidal, M.E. and Cegarra-Leiva, D. (2016), “Linking unlearning with work-life balance: an initial empirical investigation into SMEs”, Journal of Small Business Management, Vol. 54 No. 1, pp. 373-391. Cheung, H.K., Hebl, M., King, E.B., Markell, H., Moreno, C. and Nittrouer, C. (2017), “Back to the future: methodologies that capture real people in the real world”, Social Psychological and Personality Science, Vol. 8 No. 5, pp. 564-572. Cooper, H., Camic, P.M., Long, D., Panter, A., Rindskof, D. and Sher, K. (2012), The APA Handbook of Research Methods in Psychology, American Psychology Association, Washington, DC, Vols 1/3. Csikszentmihalyi, M. and Larson, R. (2014), “Validity and reliability of the experience-sampling method”, in Csikszentmihalyi, M. (Ed.), Flow and the Foundations of Positive Psychology, Springer, Dordrecht, pp. 35-54. Easterby-Smith, M. and Lyles, M.A. (2003), “Re-reading organizational learning: selective memory, forgetting and adaptation”, Academy of Management Perspectives, Vol. 17, pp. 51-55. Easterby-Smith, M. and Lyles, M.A. (2011), “In praise of organizational forgetting”, Journal of Management Inquiry, Vol. 20 No. 3, pp. 311-316. Ellwart, T. and Kluge, A. (2019), “Psychological perspectives on intentional forgetting: an overview of concepts and literature”, KI – Künstliche Intelligenz, Vol. 33 No. 1, pp. 79-84. Fernandez, V. and Sune, A. (2009), “Organizational forgetting and its causes: an empirical research”, Journal of Organizational Change Management, Vol. 22 No. 6, pp. 620-634. Fiol, C.M. and O’Connor, E. (2017a), “Unlearning established organizational routines – Part I”, The Learning Organization, Vol. 24 No. 1, pp. 13-29. Fiol, C.M. and O’Connor, E.J. (2017b), “Unlearning established organizational routines – Part II”, The Learning Organization, Vol. 24 No. 2, pp. 82-92. Fisher, C.D. and To, M.L. (2012), “Using experience sampling methodology in organizational behavior”, Journal of Organizational Behavior, Vol. 33 No. 7, pp. 865-877. Grisold, T. and Kaiser, A. (2017), “Leaving behind what we are not: applying a systems thinking perspective to present unlearning as an enabler for finding the best version of the self”, Journal of Organisational Transformation and Social Change, Vol. 14 No. 1, pp. 39-55. Grisold, T., Kaiser, A. and Hafner, J. (2017), “Unlearning before creating new knowledge: a cognitive process”, in Proceedings of the 50th Hawaii International Conference on System Sciences, pp. 4610-4623. Gutiérrez, J.O., Cegarra-Navarro, J.G., Carrion, G.A.C. and Rodríguez, A.L.L. (2015), “Linking unlearning with quality of health services through knowledge corridors”, Journal of Business Research, Vol. 68 No. 4, pp. 815-822. Hedberg, B. (1981), “How organizations learn and unlearn”, in Nystrom, P. and Starbuck, W. (Eds), Handbook of Organizational Design: Adapting Organizations to Their Environment, Oxford University Press, London, pp. 3-27. Hislop, D., Bosley, S., Coombs, C.R. and Holland, J. (2014), “The process of individual unlearning: a neglected topic in an under-researched field”, Management Learning, Vol. 45 No. 5, pp. 540-560. Howells, J. and Scholderer, J. (2016), “Forget unlearning? How an empirically unwarranted concept from psychology was imported to flourish in management and organisation studies”, Management Learning, Vol. 47 No. 4, pp. 443-463. Huang, D., Chen, S., Zhang, G. and Ye, J. (2018), “Organizational forgetting, absorptive capacity, and innovation performance: a moderated mediation analysis”, Management Decision, Vol. 56 No. 1, pp. 87-104. Huber, G.P. (1991), “Organizational learning: the contributing processes and the literatures”, Organization Science, Vol. 2 No. 1, pp. 88-115. Jain, A. and Kogut, B. (2014), “Memory and organizational evolvability in a neutral landscape”, Organization Science, Vol. 25 No. 2, pp. 479-493. Kluge, A. and Gronau, N. (2018), “Intentional forgetting in organizations: the importance of eliminating retrieval cues for implementing new routines”, Frontiers in Psychology: Organizational Psychology, Vol. 9 No. 51. Kluge, A. and Schilling, J. (2004), “Lernende organisation [Learning organization]”, in Schuler, H. (Ed.), Organisationspsychologie – Gruppe Und Organisation, Enzyklopädie Der Psychologie, Hogrefe, Göttingen, pp. 845-909. Kluge, A., Schüffler, A., Thim, C., Vladova, G. and Gronau, N. (2018), “Putting intentional organisational forgetting to an empirical test: using experimental designs to measure forgetting Investigating unlearning and forgetting 531 TLO 26,5 532 of organisational routines”, in Proceedings of the IFKAD Conference at the Technical University of Delft, Belgium, pp. 254-267. Leal-Rodríguez, A.L., Eldridge, S., Roldán, J.L., Leal-Millán, A.G. and Ortega-Gutiérrez, J. (2015), “Organizational unlearning, innovation outcomes, and performance: the moderating effect of firm size”, Journal of Business Research, Vol. 68 No. 4, pp. 803-809. Martelo-Landroguez, S., Albort-Morant, G., Leal-Rodríguez, A.L. and Ribeiro-Soriano, B. (2018), “The effect of absorptive capacity on green customer capital under an organizational unlearning context”, Sustainability, Vol. 10 No. 1, pp. 265-274. Martin de Holan, P. (2011), “Agency in voluntary organizational forgetting”, Journal of Management Inquiry, Vol. 20 No. 3, pp. 317-322. Martin de Holan, P. and Phillips, N. (2004), “Remembrance of things past? The dynamics of organizational forgetting”, Management Science, Vol. 50 No. 11, pp. 1603-1613. Martin de Holan, P., Phillips, N. and Lawrence, T.B. (2004), “Managing organizational forgetting”, MIT Sloan Management Review, Vol. 45 No. 2, pp. 45-51. Matsuo, M. (2017), “The unlearning of managerial skills: a qualitative study of executive officers”, European Management Review, doi: https://doi.org/10.1111/emre.12122. Mehrizi, M.H.R. and Lashkarbolouki, M. (2016), “Unlearning troubled business models: from realization to marginalization”, Long Range Planning, Vol. 49 No. 3, pp. 298-323. Molina-Azorin, J.F., Bergh, D.D., Corley, K.G. and Ketchen, D.J. Jr (2017), “Mixed methods in the organizational sciences: taking stock and moving forward”, Organizational Research Methods, Vol. 20 No. 2, pp. 179-192. Morais-Storz, M. and Nguyen, N. (2017), “The role of unlearning in metamorphosis and strategic resilience”, The Learning Organization, Vol. 24 No. 2, pp. 93-106. Nguyen, N. Grisold, T. and Klammer, A. (2018), “Organizational unlearning: opportunities and interdisciplinary perspectives”, The Learning Organization, available at: www. emeraldgrouppublishing.com/authors/writing/calls.htm?id=7623 Nystrom, P.C. and Starbuck, W.H. (1984), “To avoid organizational crises, unlearn”, Organizational Dynamics, Vol. 12 No. 4, pp. 53-65. Pistrang, N. and Barker, C. (2012), “Varieties of qualitative research: a pragmatic approach to selecting methods”, in Cooper, H. (Ed.), APA Handbook of Research Methods in Psychology, Research Designs: Quantitative, Qualitative, Neurophysiological and Biological, American Psychological Association, Washington, DC, pp. 5-19. Podsakoff, P.M. and Dalton, D.R. (1987), “Research methodology in organizational studies”, Journal of Management, Vol. 13 No. 2, pp. 419-441. Reese, S. (2017), “Putting organizational unlearning into practice: a few steps for the practitioner”, The Learning Organization, Vol. 24 No. 1, pp. 67-69. Runkel, P.J. and McGrath, J.E. (1972), Research on Human Behavior: A Systematic Guide to Methods, Holt, Rinehart and Winston, New York, NY. Scandura, T.A. and Williams, E.A. (2000), “Research methodology in management: current practices, trends, and implications for future research”, Academy of Management Journal, Vol. 43 No. 6, pp. 1248-1264. Schüffler, A., Thim, C., Haase, J., Gronau, N. and Kluge, A. (2019), “Information processing in the work environment 4.0 and the beneficial impact of intentional forgetting for change management”, Zeitschrift Für Arbeits- Und Organisationspsychologie. [German Journal of Work and Organizational Psychology]. Starbuck, W.H. (2017), “Organizational learning and unlearning”, The Learning Organization, Vol. 24 No. 1, pp. 30-38. Stone-Romero, E.F. (2011), “Research strategies in industrial and organizational psychology: nonexperimental, quasi-experimental and randomized experimental research in special purpose and nonspecial purpose settings”, in Zedeck, S. (Ed.), APA Handbook of Industrial and Organizational Psychology: Building and Developing the Organization, American Psychological Association, Washington, DC, pp. 37-72. Stone-Romero, E.F., Weaver, A.E. and Glenar, J.L. (1995), “Trends in research design and data analytic strategies in organizational research”, Journal of Management, Vol. 21 No. 1, pp. 141-157. Tranfield, D., Denyer, D. and Smart, P. (2003), “Towards a methodology for developing evidenceinformed management knowledge by means of systematic review”, British Journal of Management, Vol. 14 No. 3, pp. 207-222. Tsang, E.W.K. (2017), “How to concept of unlearning contributes to studies of learning organizations: a personal reflection”, The Learning Organization, Vol. 24 No. 1, pp. 39-48. Tsang, E.W.K. and Zahra, S.A. (2008), “Organizational unlearning”, Human Relations, Vol. 61 No. 10, pp. 1435-1462. Turner, S.F., Cardinal, L.B. and Burton, R.M. (2017), “Research design for mixed methods: a triangulation-based framework and roadmap”, Organizational Research Methods, Vol. 20 No. 2, pp. 243-267. Usman, M., Hameed, A.A. and Manzoor, S. (2018), “Exploring the links between ethical leadership and organizational unlearning: a case study of a European multinational company”, Business and Economic Review, Vol. 10 No. 2, pp. 28-54. Uy, M.A., Foo, M.D. and Aguinis, H. (2010), “Using experience sampling methodology to advance entrepreneurship theory and research”, Organizational Research Methods, Vol. 13 No. 1, pp. 31-54. Visser, M. (2017), “Learning and unlearning: a conceptual note”, The Learning Organization, Vol. 24 No. 1, pp. 49-57. Wong, P.S., Whelan, B. and Holdsworth, S. (2018), “Are contractors ready for greater use of prefabrication in projects? An empirical analysis on the role of unlearning and counterknowledge”, International Journal of Construction Management, pp. 1-16, doi: https://doi.org/ 10.1080/15623599.2018.1539160 Yang, K.P., Chou, C. and Chiu, Y.J. (2014), “How unlearning affects radical innovation: the dynamics of social capital and slack resources”, Technological Forecasting and Social Change, Vol. 87, pp. 152-163 Zedeck, S. (2014), APA Dictionary of Statistics and Research Methods, American Psychological Association, Washington, DC. Corresponding author Annette Kluge can be contacted at: annette.kluge@rub.de For instructions on how to order reprints of this article, please visit our website: www.emeraldgrouppublishing.com/licensing/reprints.htm Or contact us for further details: permissions@emeraldinsight.com Investigating unlearning and forgetting 533