Assessing the technological capabilities of firms: developing a policy tool Howard Rush1, John Bessant2 and Mike Hobday1 1 Centre for Research in Innovation Management, University of Brighton, The Freeman Centre, Falmer, Brighton, BNI 9QE, UK. H.J.Rush@bton.ac.uk, M.G.Hobday@brighton.ac.uk 2 Advanced Institute of Management Research and the Tanaka Business School, Imperial College, South Kensington Campus, London SW72AZ, UK. j.bessant@imperial.ac.uk The development of technological capabilities results from an extended learning process and external policy agents can play an important role in its development. This paper outlines trends in governmental and non-governmental policy initiatives and the use of concepts such as capability and absorptive capacity, which are positioned within generic-staged models of capability maturity. This paper describes the development of a technology capability assessment/audit tool that has been designed to help locate firms within four archetypes based upon their level of maturity on nine key dimensions of the management of technology. The tool is intended to help bridge the gap between our theoretical understanding of the principles of technology management and policy practice – allowing policy makers to design mechanisms that focus resources in areas of greatest need through the appropriate selection of policy mechanisms and the targeted design of policy. The use of this tool in field experiments is described along with the implications for policy making. I nnovation matters – unless an organisation changes its offerings (product/service innovation) and the ways it creates and delivers those offerings (process innovation), it risks compromising its survival and growth. In the long term, however, it is not specific innovations but rather the capability to generate a stream of product and process changes that matters. This puts emphasis on the way in which innovation is managed and deepened. Such capability is not a natural endowment. It results from an extended learning process gradually accumulating processes, procedures, routines and structures, which, when embeded, is often referred to in practice as ‘the way we do things around here’. Although important, the development of the capabilities necessary to innovate does not always take place and so becomes a legitimate concern for external policy agents. For example, a trade association may worry about the capabilities of weaker firms in its sector in terms of their international competitiveness, or a major customer in a supply chain might have concerns about its weaker members being able to deliver on time and to required quality. A local government agency may wish to see stronger firms in its region in order to preserve employment or to stimulate economic growth. Dealing with such weaknesses traditionally involves several blunt policy options, including various forms of investment support that attempt to fill the gap caused by insufficient capability within the target organisations. However, an increasingly popular approach has been to intervene with the specific objective of developing or enhancing innovation capability. R&D Management 37, 3, 2007. r 2007 The Authors. Journal compilation r 2007 Blackwell Publishing Ltd, 9600 Garsington Road, Oxford, OX4 2DQ, UK and 350 Main St, Malden, MA, 02148, USA 221 Howard Rush, John Bessant and Mike Hobday The term innovation covers an extremely broad range of activities including, among other things, organisational, financial, marketing and technological innovation. This paper, and the assessment tool described, focuses on the latter – those capabilities directly associated with the development, acquisition and use of technology. In this paper, we outline some of the trends in the evolution of capability-related policy. As encouraging as some of these policy developments have been, particularly in their recognition that firms are not all the same, the change in policy emphasis also raises a number of important questions and issues that we address in turn. These include: How is innovation capability defined and what aspects do we want to improve? How is innovation capability distributed across the population of firms – so that policy agents can target specific development levels rather than assume that one policy size fits all? How can we measure these levels and can tools be developed to assist policy agents in their effort? The paper describes a technological capabilities assessment tool intended as an aid to policy makers in the identification of the strengths and weaknesses of firms within a sector or a region in order to be able to design policies that can be better targeted and aligned to the policy objectives. The discussions that follow on the questions raised above provide the underpinnings of the assessment tool, which has now been used in different contexts in Korea, Thailand and Ireland.1 1. Evolution of capability-related policy Recent policies designed to contribute to the building of technological capabilities have their antecedents as far back as the late 1970s. Vickery and Blau (1989) and Arnold and Thuriaux (1998) describe a wide range of programmes that proliferated through OECD countries during the 1980s specifically designed to promote the application and diffusion of technologies such as micro-electronics, flexible manufacturing systems or computer-aided design. These schemes, which usually involved financial inducement or some form of rudimentary intermediary support, were often judged to have been successful (if somewhat limited) in meeting their stated aims and objectives of increasing the takeup of new technologies (Northcott et al., 1985, 1986). However, because 222 R&D Management 37, 3, 2007 such policy solutions often involved a standardised and formulaic approach, their impact on the development of capabilities was patchy and only indirect at best – usually a consequence of ‘learning-by-doing’. Over time and in response to a more thorough exploration of the processes underlying the management of technology, there has been a discernible shift in the type of schemes on offer that recognise that different firms face different types of problems and require a different range of solutions. Policies have increasingly been aimed at specially improving technological capabilities. In some cases, what had previously been the domain of centralised national governmental departments or agencies has now become the preserve of regional authorities. At this level, there is likely to be a better understanding of the needs of the firms in the local areas, making it possible to design programmes with a tighter fit to local requirements. Some of these policies involve an evolution in the roles played by intermediary structures (Carlsson and Jacobsson, 1993). Innovation consultants, for example, are employed to act as counsellors, helping firms identify and articulate their needs, or as a broker, enabling them to access relevant sources of support (Skaug, 1992; Bessant and Rush, 1995; Boden and Miles, 1998). Some policies have also expanded the roles of innovation consultants to act as facilitators in learning networks, which are proving to be an increasingly popular mechanism for local innovation policies (Bessant et al., 2003). Other structural mechanisms include the provision of specialist information centres, or specific financial support to encourage research and development (R&D) or technology transfer activities (Autio and Wicksteed, 1998; Trend Chart, 2004, Howell, 2006). Increasingly, such interventions have emerged from non-governmental policy agents. For example the Industry Forum programme in the United Kingdom has a capability development initiative targeted at weaker players in the automotive components supply chain and was established by key buyer firms through their trade association, the Society of Motor Manufacturers and Traders (Bateman and David, 2002). A number of major buyer firms have been attempting to develop better capability across their supply chains through various forms of supply chain learning initiative, following the well-established example of Toyota. (AFFA, 2000; Dyer and Nobeoka 2000; Bessant, Kaplinsky and Morris, 2003) At the regional level, a number of cluster initiatives r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Assessing the technological capabilities of firms Table 1. Different modalities for supporting development of innovation capability (a selection) Delivery option 1. Outreach/‘missionary’ services where firms are approached by field agents or counsellors to help them recognise and identify needs for change 2. Benchmarking and other measurement/comparison processes that enable targeting of manufacturing development 3. Strategic development process via a structured methodology that enables firms to create a framework within which change will be located 4. Support for technology search — where the problem is not clearly articulated 5. ‘Technology signposting’ — facilitating access where problem is clearly articulated 6. Facilitating access to funding for specific projects 7. Access to demonstration projects 8. Support for technology transfer — short-term access/consultancy 9. Support for long-term technology transfer — e.g. Universities or Research Institutes to industry 10. Access to specialist equipment on occasional basis — e.g. specialist test services 11. Facilitating experience-sharing and learning 12. Assistance with training and development 13. Major technical project/contract research 14. Network access – links to expertise via database, directory, listings, etc. Updated from Dodgson and Bessant (1996). have also made the development of technological capability a key feature (Martinussen, 1995; Keeble and Williamson, 2000, Morosini, 2004). Table 1, while not intended to be comprehensive, provides a breakdown of 14 of the delivery options that have, often in combination, been incorporated into programme designed to assist firms in developing their technological capabilities. Outreach services (1) using ‘innovation counsellors’ have, for example, been used with the UK Manufacturing Advisory Service and in the US Manufacturing Extension Program as well as the Canadian IRAP scheme. Benchmarking (2) is a regular feature of both government programmes such as those offered by the German Fraunhofer IPK and non-governmental programmes such as the confederation of British Industry’s PROBE initiative. Support for technological search (4) was the key component of the UK Supernet programme and the Finish Tekes technology clinics, whereas access to demonstration projects was used by Inside UK Enterprise Technology Enterprise. Technology transfer (9) from universities to industry has been a longstanding focus of the Irish Techstart programme and the UK Knowledge Transfer Partnership (formerly the Teaching Company Scheme). 1.1. Capability, absorptive capacity and the use of maturity models Instrumental to the development of the policy options outlined above has been the deepening understanding of concepts such as capability and absorptive capacity. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Capability implies an ability to do something and we are particularly interested in this as applied to searching out and using technology for strategic advantage. Perhaps the most significant concept in relation to this is that of absorptive capacity, which, according to Cohen and Levinthal (1990, p. 128), is ‘an ability to recognize the value of new, external knowledge, assimilate it, and apply it to commercial ends’. It is, therefore, to be found in the underlying knowledge and experience base of the firm that enables capabilities (technological or otherwise) to be generated. A growing number of studies have operationalised and elaborated upon this concept. Zahra and George (2002), for example, argue that absorptive capacity consists of four distinct capabilities: Acquisition – the search for new knowledge; Assimilation – understanding new knowledge; Transformation – seeing how new knowledge can be used in the context of the firm’s issues and existing knowledge; and Application – implementation of actions enabled by the new knowledge. Martin et al. (2003) develop these further, suggesting that: Acquisition is the organisation’s dynamic capacity to identify and acquire external knowledge (speed, intensity and direction of knowledge acquisition); Assimilation being the organisation’s routines and processes that allow it to understand and process information from external sources; Transformation being the capability to develop and fuse new with existing knowledge; and R&D Management 37, 3, 2007 223 Howard Rush, John Bessant and Mike Hobday Exploitation (which we understand to be similar to Zahra and George’s use of the term ‘application’) as the capacity and routines of an organisation to use its acquired and transformed knowledge to refine, build on and leverage existing learning competencies. Clearly, these concepts can be applied to the domain of technological innovation where we suggest that it refers to the ability to find and use technology to secure and sustain competitive advantage. However, while the ability to acquire, assimilate, transform and apply (or exploit) new knowledge is necessary for all successful organisations, firms differ in their awareness of the need to change and in their abilities to effect such changes. Increasingly, attention has, therefore, focused on the concept of maturity models, which can be used to benchmark an organisation’s competence in some particular activity against a body of knowledge. Such models tend to show generic levels of capability development from a low base up to one of excellence. For example Hillson (2003) proposes a model of project management maturity, relating to an organisation’s awareness of the effectiveness of its project management practices and describes the Project Management Maturity Model (ProMMM). The model describes four levels of increasing project management capability (naı̈ve, novice, normalised and natural), with each of these levels calibrated in terms of four dimensions (culture, process, experience and application). Carnegie Mellon’s Capability Maturity Model (CMM) for software proposes levels that are based on the existence of best-practice software development processes to ensure programme reliability, and the degree to which a firm sticks to these processes in the face of environmental disruption. It identifies the practices that are basic to implementing effective processes as well as advanced practices. It also assigns to those practices associated maturity levels ranging from unrepeatable to a mature, well-managed process. Typically, a path is recommended through the various practices for achieving higher levels of maturity and, therefore, improving an organisation’s processes. The CMM consists of five stages: initial, repeatable, defined, managed and optimising, with objectives similar to the ProMMM, but contextualised in terms of software processes (Curtis and Paulk, 1993). Finally, total quality management (TQM) practices have been categorised at levels leading up to a ‘black belt’. Before being awarded such a 224 R&D Management 37, 3, 2007 status, individuals must demonstrate problemsolving, project management and team leadership skills. The systematic training approach followed typically involves five problem-solving steps of: define, measure, analyse, improve and control (Ingle and Roe, 2001). 1.2. How is technological capability distributed across a population? Attempts have also been made to develop such approaches for the innovation field. For example Bell’s (2003) competency levels model for technological innovation in which organisations pass from the point of ‘acquiring and assimilating imported technologies’, through phases of ‘technology deepening and upgrading’ and ‘closing in on the international technological frontier’ to reach a stage where the organisation is ‘generating core advances at international frontiers’. This describes a process of moving from low or zero capability, developing minimal capability up to a level of competence, ultimately to become (high) performers. In a similar fashion, Arnold and Thuriaux (1998) describe four degrees of a firm’s levels of knowledge relating to technological capability. These degrees of ‘mastery’ are conceived in terms of boxes that progress from opaque to transparent and closed to open in a series of boxes, starting with a closed ‘black box’ through stages of ‘grey’ and ‘white’ box to an ‘unboxed’ state at which point a firm is generally able to develop significantly new variants or innovations. Whether the learning is generated internally or brought in from external sources, the aim is to move towards an ‘unboxed’ state. Arnold and Thuriaux’s scheme can be viewed as a base level, (black box) with only an awareness of the problem (not of the solution) and so requiring a search for new knowledge, and three succeeding levels of understanding that roughly correspond to the assimilation and transformation levels of Zahra and George (2002). There are similarities in this view to the extended model of absorptive capacity put forward by Zahra and George (2002) that saw two distinct activities – assimilation and transformation – being involved. Drawing on these ‘competency’- or ‘maturity’level approaches, we suggest that the development of technological capability can be seen as a set of ‘punctuated equilibrium’ states. As firms move into more complex environments, they need a richer set of capabilities to deal effectively with r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Assessing the technological capabilities of firms the threats and opportunities that confront them.2 We discuss this model in terms of four archetypes that characterise these states. Type 1 firms can be labelled as unaware or passive and these are characterised as being ‘unconscious’ or unaware about the need for technological change in what may be a hostile environment and where technological know-how and ability may be vital to survival. These firms do not know where or what they might improve, or how to go about the process of technological upgrading. As such, they are highly vulnerable to competitive forces. For example, if low-cost competitors enter the market, or if customers require faster delivery or higher quality – they are not able to pick up relevant signals or respond quickly. Even if they do, they may waste scarce resources by targeting the wrong kind of improvement. Such firms are characterised by a fire fighting, crisis style of management with little if any longterm strategic focus (Bessant, 1999). Often family owned and with a strong local orientation, their biggest limitation is their relative isolation. Their capabilities are limited to reactive innovation, often as sub-contractors supplying goods and services to other firms. Emphasis in innovation is almost entirely on process change with a strong focus on cost as the key competitive driver. Key problems in this group can be described as a lack of: awareness of potential problems or of resources (including human resources) that might help deal with them; motivation to change; managerial skills to prioritise and planning for change; implementation skills; and awareness or access to national innovation system (especially the ability to identify relevant partners or service providers). Not surprisingly, unaware or passive firms are ill prepared in all major areas of technology acquisition, use, development, strategy and so on. Such firms usually have a predominantly ‘local’ orientation and, technological change, when it does happen, is often dominated by suppliers and confined to ‘off–the-shelf’ solutions. In terms of organisational development, a thorough basic improvement programme is urgently needed. Assistance is required in enabling these firms to recognise the need for change developing a strategic framework, identifying appropriate changes, and in acquiring and implementing r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd necessary technologies. They also require assistance in sustaining this process of change over the long term.3 Type 2 firms might be defined as reactive in that they recognise the challenge of change and the need for continuous improvements in their technological capabilities but are unclear about how to go about the process in the most effective fashion. As their internal resources are often limited and they lack the key skills and experience, they tend to react to technological threats and possibilities but are often unable to shape and exploit events to their advantage. Their external networks are usually poorly developed, with most of their technological know-how coming from their suppliers and following other firms in their sector. Typically, this group treats symptoms rather than causes of problems; for example, dealing with bottleneck operations by replacing machinery only to find that the problem becomes worse because the root cause is, in fact, production scheduling. Such firms are, therefore, often characterised by inappropriate innovation as much as by a lack of it. Other key problems include the lack of: detailed awareness of innovation challenges – for example, they would not be well aware of the nature of the technological frontier, only that they lie some way behind it (included in this is a lack of awareness to anticipate change); awareness of broader market and competitive drivers – their innovation agenda is often set through signals filtered via key customers or by imitative behaviour related to similar firms; awareness of the full range of technological options to upgrade capability – usually manifest in an emphasis on investment in new technology packaged into capital equipment rather than an interest in improved or different use of existing resources within the firm; awareness of distinctive competency or the need to identify and build this; internal resources or the awareness of the full range of resources available within the national innovation system – often manifest in a reliance on a few regular sources of technology and limited search behaviour; capability in articulating core problems and a tendency to treat symptoms rather than deal with root cause issues; strategic orientation to technological change (instead of the tendency towards piecemeal innovation); R&D Management 37, 3, 2007 225 Howard Rush, John Bessant and Mike Hobday a framework for prioritising innovation or for ensuring that changes support the business strategy; and implementation capability. The needs of this group centre first on the development of a strategic framework for technological change, so that key priority areas can be addressed. Allied to this are needs in searching wider for solutions, in exploring new concepts (e.g. changing production layout rather than simply acquiring new machinery) and in acquiring and implementing new products and process capabilities. In the longer term, such firms could be expected to develop an internal capability for strategic upgrading and require less and less support.4 Type 3 firms have a well-developed sense of the need for technological change. We refer to these as strategic in that they are capable of implementing new projects and in adopting a strategic approach to the process of continuous innovation. They have a clear idea of what has to be done, when and by whom, and have the internal capabilities in both technical and managerial areas and can implement changes with skill and speed (what might be described as having an ‘ability to think within the box’). These firms benefit from a deliberately developed strategic framework in terms of search acquisition, implementation and improvement of technology, including the ability to network and collaborate effectively towards these ends. Nevertheless, strategic firms still face a number of problems, including the following: they tend to lack the capabilities to re-define markets through new technology (e.g. by developing new product categories either alone or in partnership with others, which can result in new market niches and segments – such as IBM managed in e-commerce, Nokia did in mobile communications or Sony achieved with the Walkman); they lack the capabilities to create new market opportunities; they exhibit the tendency to compete within the boundaries of an existing industry and may become ‘trapped’ in a mature or slow-growth sector; and they have a limited knowledge of where and how to acquire new technologies beyond the boundaries of their traditional business. Overall, these companies have strong in-house capabilities and think strategically about technol226 R&D Management 37, 3, 2007 ogy in the medium and long term. In some areas, these firms may be behind the international technological frontier but they have important foundations upon which to build. The needs of this group are essentially around providing complementary support to internal capabilities and challenging existing business models. Improving access to specialist technical and marketing expertise, enabling access to new networks of technology providers (e.g. overseas sources) can assist these firms to think ‘outside’ of the industrial box they find themselves in, should the need arise. Such firms may also benefit from occasional, project-based support from consultancy companies or from specialist research and technology organisations, locally or internationally. These firms may benefit from improved access to graduates and from linking up with universities that offer new ideas, access to advanced technology and new skills.5 Firms in our final category (type 4) have welldeveloped sets of technological capabilities and are able to help define the international frontier. We refer to these firms as creative in that they are able to adopt a proactive approach to exploiting technology for competitive advantage. They are at ease with modern strategic frameworks for innovation and take it upon themselves to rewrite the rules of the competitive game with respect to technology, markets and organisation. Strong internal resources are coupled with a high degree of absorptive capacity that can enable diversification into other sectors, where their own skills and capabilities bring new advantages and re-define the ways in which firms traditionally compete, or wish to compete. Their technology and market networks are extensive so that they are kept informed about new technological opportunities and remain in touch with suppliers of equipment and ideas. There are unlikely to be more than a few firms in this category at any one time and they are generally seen as ‘risk takers’. The needs of this group are mainly around complementing existing internal capabilities with outside sources, assessing risks and uncertainties and sustaining their position as ‘rule breakers’. They tend to be open companies that collaborate and learn from partners in the external environment and invest in developing new technologies and resources. From time to time, projects emerge that threaten to disrupt their existing businesses and they are often in a strong position to convert such threats into new market opportunities. Such firms may need to develop new contacts with specialist r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Assessing the technological capabilities of firms groups in order to resolve complex technical problems and generate new opportunities. While not without their own concerns (e.g. risk associated with heavy investments outlay, intense international competition, etc.), they are often in a position to make useful contributions to government policy as they try to position and develop their national systems of innovation for the future.6 1.3. How can we measure technological capability? Identifying archetypes that characterise each of four ‘punctuated equilibrium’ states in the development of technological capabilities, however, remains an academic exercise of only limited value to policy actors. A means of accurately locating firms within the framework is still required in order that their strengths and weaknesses can be identified and appropriate policies and organisational development strategies are applied. Drawing on literature on the management of an innovation process, we have attempted to link knowledge about key abilities in technological innovation to states of development of technological capability that enable a firm to choose and use technology to create strategic competitive advantage. Studies that particularly influenced our thinking in developing the tool included developed country studies on technology development such as Ansoff and Stewart (1967), Teece and Pisano (1994), Utterback and Abernathy (1975), Kay (1993) and Hamel and Prahalad (1994), Tidd et al. (1997). Important studies focusing on developing countries included Kim (1997); Choi (1994), Fransman and King (1984), Gerstenfeld and Wortzel (1997), Dahlman et al. (1985). Organisational learning studies also underpin the importance of capability development in firms (e.g. Argyris and Schon, 1978; Stata, 1989; Leonard-Barton, 1992; Garvin, 1993). The tool also builds directly upon previous World Banksponsored research (Arnold et al., 2000). This literature has provided the foundation for our attempt to link knowledge about key abilities in technological innovation to states of development of technological capability that enable a firm to choose and use technology to create strategic competitive advantage. We have identified nine principal components as being fundamental to the model. These are: r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Initial awareness of the need to change and willingness to begin looking inside and outside the firm for possible triggers for change. This refers to the ability of senior management to recognise the role of technology in competitiveness and the dangers of ‘standing still’ in today’s highly competitive environment. Searching out triggers for change – picking up demand signals from the market or within the firm about the changes needed or picking up signals about potential opportunities raised by new technological developments. This is the ability to scan or monitor external technology events and trends, which might affect the company or provide opportunities for growth or competitiveness. Building of core competencies – recognition of requirements for technology through a systematic and regular audit of its current competencies and a comparison of those that it needs to develop or acquire in order to become or remain competitive. This category refers to the success of a company in defining its individual technological strengths and build up a unique advantage in specific areas. A company with a strong technological competence with an understanding of how its distinctive technological strengths differ from its competitors and how to further develop its skills and knowledge to remain competitive. It will have well-developed methods for protecting and exploiting its intellectual property. Development from these of a technology strategy – some clear idea of where to change and why. Formulating a technology strategy is a key part of the overall business strategy of any leading firm. This is the process by which visions, objectives and priorities are set and communicated. Even the best-resourced organisation cannot do everything with respect to technology, and so part of the strategic challenge lies in choosing which technology activities to conduct in house and which to outsource to partners. The exploration and assessment of the range of technological options available – making comparisons between all the options available that can be achieved through some form of benchmarking, feasibility studies, etc. – and selection of the most appropriate option based upon the comparison. Leading companies are able to gather information on the range of technological options available, chose quickly among competing solutions and identify the most appropriate source that ‘fits’ with their needs. R&D Management 37, 3, 2007 227 Howard Rush, John Bessant and Mike Hobday A leading firm is able to make a comparison between or ‘benchmark’ the various options available and can reliably select the most appropriate option based upon this comparison. Acquisition of the technology. Once a new technology option is decided upon, a firm needs to deploy the resources to exploit it either by creating technology via in-house R&D, or acquiring it through a joint venture or technology licensing, etc. In some cases, this may be a simple matter of buying off-the-shelf, or it may involve exploiting the results of research already carried out. In other cases, it might require extensive search and research to acquire the technology; some of these avenues are likely to involve extensive negotiation around price, specification, transfer of knowledge, property rights, etc. Implementation, absorption and operation of the technology within the firm. Having acquired or developed technology, a firm needs to implement the technology within the organisation, which may involve various stages of further development to final implementation or launch, as in the case of a new product or service or a new manufacturing process or method within the organisation (and learning how best to use it). This may involve extensive project planning and management activities and may require configuration or both technology and organisation to obtain a good and workable fit. Learning forms an important part of the building of technological competencies and involves reflecting upon and reviewing technology projects and processes within the firm, in order to learn from both successes and failures. In leading firms, this learning process can become conscious and formal leading to continuous improvement in effectiveness, efficiency and strategy formulation. In order to learn how to manage the other technological competencies/ components better, a firm needs to develop internal methods to capture systematically relevant knowledge from its own (and other firms’) experience and act on this knowledge in order to sustain technological development in the long term. Exploiting external linkages and incentives. In each of the eight previous components, firms can and, in some cases, should make use of external suppliers of technology and related services (e.g. consultancy companies, government research institutes, universities). In addi- 228 R&D Management 37, 3, 2007 tion to giving an indication as to the level of technological sophistication and openness of the firm, this dimension can also give an indication of how well developed are the external support systems for technology development. These can map on to a simple model of technological change over time that involves several stages based upon the four archetypes described in the previous section. Although, as presented, such a model may appear to be a linear process, we recognise that there are numerous interactions and feedback loops between different components. Using the nine-component framework described above, we can generate a series of questions to ask firms to help assess their technological capability. Such questions, with the aid of the corresponding guidance notes that accompany them, can allow a trained manager, academic researcher or policy maker to identify those behaviour and routines that contribute to or are necessary for the development of a firm’s technological capabilities. For example, a firm that makes no effort to scan its environment for signals about threats and opportunities is likely to be much weaker than one that has in place sophisticated mechanisms for spotting and evaluating signals about relevant changes – such as the emergence of new technologies. We can apply this approach to the whole of the nine components of the model incorporated into the tool and assign a score against the firm in each of the dimensions of technological capability. The audit tool was originally developed to carry out in-depth case studies, postal questionnaires and rapid face-to-face interview audits. The version of the tool presented in Table 2 is a highly simplified, summary version of the full tool presented in Bessant et al. (2000). It can be used for an initial ‘filtering’ of firms and does provide a good indication of the range of questions covered by the in-depth tool.7 The questions (1–24) call for a subjective assessment of the nine dimensions of capability by one or more senior managers of the individual firm, according to the scale in the table (which corresponds to the four levels of capability). Questions 1–11 are more focused on how aware of technological issues the firm is, while questions 12–24 concentrate on how well the firm is able to achieve results in practice. For each firm, a graphical representation of the level of r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Table 2. Short version of the innovation capability audit tool Technology activity area Key questions Assessment score Reactive – weak to average capability 2 Strategic – strong capability 3 Creative – very strong capability 4 N/A 229 Assessing the technological capabilities of firms R&D Management 37, 3, 2007 Q1 þ 2 Awareness 1 To what extent does technology play a part in your company’s business strategy? 2 Which technologies are most important to your business? Q3 þ 4 Search 3 How does your firm assess technological opportunities and threats? 4 Who (person or group) is responsible for technology assessment within your company? Q5 þ 6 Building a core competence 5 Please describe any special technological strength your company benefits from? 6 How does your company use technology to create business opportunities for the future? Q7 þ 8 þ 9 Technology strategy 7 How skilled is your management in formulating a technology strategy to meet business goals? 8 What are the main technology priorities of your firm? 9 How does technology fit in with the ‘vision’ of your company? Q10 þ 11 Assessing and selecting technology 10 How does your company go about selecting the technology it needs for its business? 11 How do you know when you have selected the best technology option? Q12 þ 13 Technology acquisition 12 How does your company go about acquiring technology from external sources once it is selected? 13 Which external technology suppliers are most important to your business? Q14 and 15 Implementing and absorbing technology 14 How are your various technology activities (e.g. engineering and R&D) organised within your company? 15 How are projects for introducing new technologies organised? Q16, 17 and 18 Learning 16 Does your company have systems for assessing technology projects? If so, please elaborate. 17 Does your company learn effectively from one technology project to another? If so how? 18 Does your firm carry out post-project reviews? Please describe. Q19–24 Building external linkages Unaware – very weak capability 1 230 19 Do you utilise any Government agencies or schemes when developing technology? If so, which ones? 20 Do you use any private consultancy organisations to assist you in technology assessment? If so, who? 21 Do you use any outside bodies to help you develop technology? If so who? 22 Do you hire any external organisations to help you assess your technology performance? If so who? 23 Do you work with any universities in key technology projects? If so which one/s? 24 Do you collaborate with any government research institutes for future technology projects? If so which one/s? Assessment score Technology activity area Table 2. (Contd.) Key questions Unaware – very weak capability 1 Reactive – weak to average capability 2 Strategic – strong capability 3 Creative – very strong capability 4 N/A Howard Rush, John Bessant and Mike Hobday R&D Management 37, 3, 2007 capabilities against each of the nine dimensions is then generated, usually in the form of a ‘spider diagram’ (as depicted in Figure 1). The short version of the tool not only provides a simple mechanism for rapidly auditing the capability of individual firms but also a way of benchmarking the strengths and weaknesses of individual firms against the ‘best-practice’ model defined by creative-type firms. The aim is not to develop precise quantitative measurements but to rapidly generate a picture of how well the firm performs overall, and key areas of strength and weakness across the nine dimensions (as in Figure 1). An example of one component section of the in-depth tool is also depicted in Table 3. Conceivably, the in-depth version of the tool could also be used ‘in-house’ by managers with sufficient expertise (as might, e.g. be found in a large firm that fell into our strategic or creative categories) or by private management consultants as part of an organisational development programme. However, it was designed to be used by policy agents (ideally innovation counsellors) who might typically require a day of training in the use of the tool. Use of the short version of the tool might typically require a couple of hours from a single or several appropriate managers depending upon the size of the firm. To undertake a full assessment naturally requires more time and effort and the identification of appropriate personnel. However, by employing experienced innovation consultants using an extended interview/focus group process and working with managers who have been prepared for the task in advance, one should be able to complete the task in the equivalent to 1–3 days, depending on the size of the firm. Historical case material is gathered and a ‘story’ covering technological progress and key milestones is obtained from each company. Basic information on the company’s history, number of employees, turnover, product markets and technologies required is also checked and updated during the interview. Explanatory answers to the audit questions are written up to provide a detailed, qualitative assessment for each firm. Although scores are assigned that allow for the positioning of the firm, it is recognised that such scores still represent a subjective process and some of the capabilities being assessed are, to some degree, intangible – which is why the explanatory answers and adherence to the guidelines provided are important for retaining confidence in the tool’s reliability. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Assessing the technological capabilities of firms Awareness 4 3.5 Linkages Search 3 2.5 2 1.5 1 Learning Core competence 0.5 Best practice model Company x profile 0 Implementation Strategy Acquire Assess/selection Figure 1. Profile of results. 1.4. Using the tool: some snapshot cases The first test of the tool was in policy-oriented research in Korea for the World Bank (Hobday et al., 2001, 2004). Before this, most studies of Korean industry focused on policy issues, sector and industrial surveys or macroeconomic analysis. There were relatively few studies of firm-level innovation management. The use of the tool was to provide an assessment of the technological strategies and capabilities of Korean firms. A test sample of 25 firms was selected, with the sample constructed to cover a wide range of sector groupings (e.g. electronics and automobiles) as well as examples of local, foreign-owned and joint venture firms. The aim was not to attempt to construct a representative or random sample, but to choose industries on the basis of export importance, technology intensity and export vs. local market orientation. Given the size and complexity of the Korean industrial base, the sample was inevitably partial. In selecting firms for the testing of the tool, we hoped to gain a fairly wide spread of firmlevel perspectives in order to carry out comparisons and point to general levels of industrial capability. Given the small size of the sample, the findings were indicative rather than conclusive, although they pointed to some fairly consistent results regarding the level of technical capabilities among Korean firms and the challenges facing them. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd In terms of the lessons learned in the use of the tool itself, it was realised that evidence collected could be largely ‘snapshot’, relying on the interviewee’s memory of recent history, rather than a thorough analysis of past events. The importance of interviewee selection was clearly important and, time permitting, multiple interviews. Data collected should also be supplemented (in advance of the interviews if possible) by using secondary sources (e.g. annual reports, patent searches) that might be available from the company (e.g. annual reports) or web searches of business news and stock market analysts’ assessments, etc. (In our opinion, the use of such secondary sources can prove valuable in contextualising the results from the tool and the interviewees’ understanding of the response obtained in the interview but are unlikely to ever replace the need for face-to-face interviews.) Notwithstanding these limitations, the tool was able to locate firms within the competence or maturity framework model, providing both a detailed audit on the nine capability components, as well as an interesting assessment of the range of capabilities within and between sectors. The study was also able to provide interesting insights into whether latecomer firms were in a position to begin to compete on the basis of new product development and in-house research and development. The capabilities audit tool helped to show, among other things, the potential supporting role that leading capital goods producers could provide to R&D Management 37, 3, 2007 231 R&D Management 37, 3, 2007 Section 7: Implementing and absorbing technology Having acquired technology, a firm needs to implement the technology within the organisation, which may involve various stages of further development to final launch, as in the case of a new product or service in the external market place, or a new manufacturing process or method within the organisation. This often involves further innovation as the technology is adapted and reconfigured. As well as in-house knowledge and skills, a leading firm will usually need well-developed project management capabilities to ensure implementation is effective and efficient. Question Underlying themes How to interpret the response How do you manage the process of implementing technology-based projects? Successful firms have skills and experience in project management. How do you manage risk in development projects? Successful firms operate some form of risk management – such as the use of a ‘stage-gate’ framework to guide product development. How do you ensure co-operation and communication between different functions in the firm – R&D, engineering, production, marketing, etc.? Successful firms are able to engage cross-functional expertise (e.g. marketing, production, quality) to create new products/processes. If the firm is inexperienced or incapable of project management there is a high risk that projects will run over time or budget. Without a suitable risk management framework the firm may find itself unable to monitor progress or to stop projects which have run into difficulties or no longer fit the strategy. If the firm is unable to bring different functions together during the project there is a risk of costs and time problems. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Summary score On the basis of the answers to these (and other) questions, where would you position the firm on this scale? Score 1 2 3 4 Typical characteristics The firm has little project management experience or structure. Projects can easily drift out of control. The firm has some project man agement expertise but has no framework for risk management or continuing assessment of project progress. The firm makes use of a risk management framework and has project management skills. The firm has a well-developed framework for risk management and is experienced in project management. It has structures and processes in place to enable cross-functional co-operation and early involvement. Your assessment Howard Rush, John Bessant and Mike Hobday 232 Table 3. Example from full audit tool Assessing the technological capabilities of firms firms seeking to develop need products. Far greater R&D investment was also needed and a re-orientation of the role played by Korean universities and government-funded research institutes. The second major testing of the tool was in a subsequent World Bank study examining capability building in Thailand (Hobday and Rush, 2002). The aim was to show how firm-level information can be used to inform policymaking help to ‘fine tune’ technology programmes according to the level of technological development of different kinds of firms. It was recognised that, generally, the information available to policy makers via R&D statistics, patent data, innovation surveys, etc., is at too high a level of aggregation to distinguish systematically between different groups of firms. We used this opportunity to test the tool as a means of providing information on technological levels, dynamics and problems facing local firms. On this occasion, it was decided to select a single sector of high importance to the country; in this case, electronics, because of its position as the largest export sector in Thailand and as an example of a high-technology industry with substantial upgrading potential. Electronics was also deemed to be of interest because of the mix of a variety of different foreign-owned and national firms thought to be at different stages of technological development. The audit tool identified a wide variety of firms (both foreign and local) with critical needs ranging from basic training for production skills, to engineering support for SMEs, through to new product development, R&D and advanced strategic management. From this list, it is clear that no single policy mechanism could effectively meet this plurality of needs. We were able to delineate strengths and weaknesses for each of the four categories of firms, which would be of help to policy makers in determining priorities and where to concentrate-limited resources. By concentrating on a single sector, the audit tool was also useful in identifying a number of major gaps in the supply chain (some simpler components and indirect materials as well as in important technology support areas such as tool and die making), a variety of infrastructural problems (in the electricity and water supply) and particular skills shortages that might be addressed by appropriate policies – for example via government-sponsored training and human resources development in technical colleges and universities. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd The evidence provided by the tool also indicated that it would be worth considering targeting policies towards those groups of firms that could be described as ‘policy receptive’ and that there might be an advantage in developing technology policies that address the needs of those firms with the potential to upgrade technologically, rather than on the basis of ownership. Upon reflection of the use of the tool in a single sector, we note that there is the potential for a degree of sectoral ‘finetuning’ of the tool in order to delve more deeply into the specific technological capabilities relevant to any particular sector. What it does allow, as seen within the study in Thailand, was the ability to identify, for the policymaker, differences between different market segments, owners, niche market vs. mass producers, etc., and provide the sort of information that might be useful in the targeting of policy. A third use of the tool is currently underway in Ireland with a group of firms across a wide range of sectors. The firms included in the sample are part of a government-sponsored programme of assistance. The audit tool/interview approach was chosen because it was felt that it would provide a unique insight into a selection of Irish companies, particularly in their approaches to R&D and the impact of public funding support on that R&D. The short version of the tool was used on a sample of 15 companies from the Irish Medical Device Association and the full tool with 28 companies who had received RTI funding through an Enterprise Ireland programme. One of the methodological findings of this study has been the possibility that the short version tends to rate the firm as having higher levels of perceived capabilities than the full tool completed with the aid of a knowledgeable consultant. (Hanrahan, 2006). 2. Conclusions: implications for policy Over the past decade, the design of technology and innovation policies around the world seems to indicate that there is now a general acceptance that, because firms are not all alike, one size (i.e. type of policy mechanism) does not fit all. That policies need to be increasingly fine grained is clear – the challenge is how? The deepening of our appreciation of the concept of innovation capability of firms at different levels or stages in their developments, along with the differentiation and codification of the different components of capabilities that underlie technology and innovation R&D Management 37, 3, 2007 233 Howard Rush, John Bessant and Mike Hobday management, has improved our recognition of when and why firms might use external help. There still remains, however, a substantial gap between our understanding and practice. Policy makers and policy researchers still require better tools. The Technological Capabilities Audit described in this paper is just one tool in the toolkit that is required. Such tools and their use in field experiments can help to generate knowledge about how to build relationships and how to sustain capability development. Although some policy makers might conceivably use the Technology Capability Audit tool in their decision making processes to determine which firms receive financial support, it was not designed with this intention. Such financial decisions are dependent upon the specific objectives and design of each policy, which will differ from country to country and case to case. Our aim was to provide a means of assisting policy makers in tailoring support according to the level of capability of the firm. While such tools are developed as a means of positioning, they also contain an educational aspect, including methods of understanding and ‘unpacking’ difficult concepts such as capability and innovation. With increasing decentralisation of policy responsibility comes the need to give a wide range of policy agents exposure to the principles of innovation management capability and such tools can help in their comprehension of these principles. Increasing use of tools will help in moving from theory and research to practice – the identification of strengths and weaknesses and location on a capabilities ladder is a necessary step. Ultimately, the real challenge and need for creativity will, however, be in the design of support policies and mechanisms that effectively assist firms. References AFFA (2000) Supply chain learning: chain reversal and shared learning for global competitiveness, Department of Agriculture, Fisheries and Forestry (AFFA), Canberra, Australia. Allen, P. (2001) A complex systems approach to learning, adaptive networks. International Journal of Innovation Management, 5, 149–180. Ansoff, H.I. and Stewart, J.M. (1967) Strategies for a technology-based business. Harvard Business Review, 45, 6, 71–83. Argyris, C. and Schon, D.A. (1978) Organizational Learning: a Theory of Action Perspective. Reading, MA: Addison-Wesley. 234 R&D Management 37, 3, 2007 Arnold, E., Bell, M., Bessant, J. and Brimble, P. (2000) Enhancing Policy and Institutional Support for Industrial Technology Development in Thailand. Vol 1: The Overall Policy Framework and the Development of the Industrial Innovation System. Washington, D.C.: World Bank/National Science and Technology Development Agency (NSTDA) of Thailand. Arnold, E. and Thuriaux, B. (1998) Developing firms’ technological capabilities. Working paper, Technopolis, Brighton. Autio, E. and Wicksteed, W. (1998) Technology clinic initiative: evaluation report. TEKES Technology Development Centre, Helsinki. Bateman, N. and David, A. (2002) Process improvement programmes: a model for assessing sustainability. International Journal of Operations and Production Management, 22, 5, 515–526. Bell, M. (2003) Knowledge resources, innovation capabilities and sustained competitiveness in Thailand, final report to NSTDA. SPRU, Brighton. Bessant, J. (1999) Developing learning networks. Technology Management, University of Cambridge, Cambridge. Bessant, J., Kaplinsky, R. and Lamming, R. (2003) Putting supply chain learning into practice. International Journal of Operations and Production Management, 23, 2, 167–184. Bessant, J. and Rush, H. (1995) Building bridges for innovation: the role of consultants in technology transfer. Research Policy, 24, 97–114. Bessant, J., Rush, H. and Hobday, M. (2000) Assessing technological capabilities: an audit tool. Report to World Bank; project on Korea and the knowledgebased economy. Bessant, J., Kaplinsky, R. and Morris, M. (2003) Developing capability through learning networks. International Journal of Technology Management and Sustainable Development, 2–1. Boden, M. and Miles, I. (eds), (1998) Services and the Knowledge-Based Economy. London: Continuum. Carlsson, B. and Jacobsson, S. (1993) Technology and the wealth of nations. In Foray, D. and Freeman, C. (eds) Technological Systems and Economic Performance: the Diffusion of Factory Automation in Sweden. London: Pinter Publishers, pp. 77–94. Choi, Y. (1994) Dynamic techno-management capability: the case of Samsung semiconductor sector in Korea. PhD Thesis, Department of Economics and Planning, Roskilde University. Cohen, W.M. and Levinthal, D.A. (1990) Absorptive capacity: a new perspective on learning and innovation. Administrative Science Quarterly, 35, 1, 128–152. Curtis, B. and Paulk, M. (1993) Creating a software process improvement programme. Information and Software Technology, 35, 6/7, 381. Dahlman, C.J., Ross-Larson, B. and Westphal, L.E. (1985) Managing Technological Development: Lessons from the Newly Industrializing Countries. Washington, D.C.: The World Bank. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Assessing the technological capabilities of firms Dodgson, M. and Bessant, J. (1996) Effective Innovation Policy. London: International Thomson Business Press. Dyer, J. and Nobeoka, K. (2000) Creating and managing a high-performance knowledge-sharing network: the Toyota case. Strategic Management Journal, 21, 3, 345–367. Fransman, M. and King, K. (eds) (1984) Technological Capability in the Third World. London: Macmillan. Garvin, D.A. (1993) Building a Learning Organization. Harvard Business Review, 72, 4, 78–92. Gerstenfeld, A. and Wortzel, L.H. (1977) Strategies for Innovation in Developing Countries. Sloan Management Review, 19, 1, 57–68. Hanrahan, E. (2006), Personnal communication and draft chapters of a PhD, Enterprise Ireland and the University of Brighton. Hamel, G. and Prahalad, C.K. (1994) Competing for the Future. Boston, MA: Harvard Business School Press. Hillson, D. (2003) Assessing project management capability. Journal of Facilities Management, 2, 3, 298. Hobday, M., Rush, H. and Bessant, J. (2004) Approaching the innovation frontier in Korea: the transition phase to leadership, Research Policy, 33, 1433–1457. Hobday, M., Rush, H. and Bessant, J. (2001) Firmlevel innovation in the Korean economy. Report to the World Bank. Hobday, M. and Rush, H. (2002) Innovation readiness in the electronics industry in Thailand: policies for technological upgrading. Report to the World Bank. Howells, J. (2006) Intermediation and the role of intermediaries in innovation. Research Policy, 35, 715–728. Ingle, S. and Roe, W. (2001) Six sigma black belt implementation. The TQM Magazine, 13–4, 273– 280. Kay, J. (1993) Foundations of Corporate Success: How Business Strategies Add Value. Oxford: Oxford University Press. Keeble, D. and Williamson, F. (eds), (2000) High Technology Clusters, Networking and Collective Learning in Europe. Aldershot: Ashgate. Kim, L. (1997) Imitation to Innovation: the Dynamics of Korea’s Technological Learning. Boston, MA: Harvard Business School Press. Leonard-Barton, D. (1992) Core capabilities and core rigidities: a paradox in managing new product development. Strategic Management Journal, 13, 111–125. Martin, G., Massy, J. and Clarke, T. (2003) When absorptive capacity meets institutions and (e)learners: adopting, diffusing and exploiting e-learning in organizations. International Journal of Training and Development, 7, 4, 228–244. Martinussen, J. (1995) Elements of Success in Cluster Policies: from a Practitioner’s Point of View. Worcester: Business Net Ltd. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd Morosini, P. (2004) Industrial clusters, knowledge integration and performance. World Development, 32, 2, 305–306. Northcott, J., Bennett, S., Bessant, J., Kelly, S., Lamming, R., Mills, L., Rush, H. and Walker, S. (1985) Promoting Innovation: Microelectronics Applications Programme, London: Policy Studies Institute. Northcott, J., Bennett, S., Bessant, J., Lamming, R., Marti, J., Mills, L., Rush, H. and Walling, A. (1986) Promoting Innovation 2: Microelectronics Consultancy Support, London: Policy Studies Institute. Rush, H., Bessant, J. and Hobday, M. (2006) Assessing Technological Capability: design and uses of an audit tool, Lourdin, J. (ed.). Prague: Filosfia. Skaug, E. (1992) Brief description of the BUNT Programme. Technology Transfer and Implementation. London: Science and Engineering Research Council. Stata, R. (1989) Organizational learning – the key to management innovation. Sloan Management Review, Spring, 63–74. Teece, D.J. and Pisano, G. (1994) The dynamic capabilities of firms: an introduction. Industrial and Corporate Change, 3, 537–556. Tidd, J., Bessant, J. and Pavitt, K. (1997) Managing Innovation: Integrating Technological, Market and Organizational Change. Chichester: John Wiley and Son. Trend Chart (2004) Innovation Policy in Europe 2004, DG Enterprise and Industry, European Commission, Brussels. Utterback, J.M. and Abernathy, W.J. (1975) A dynamic model of process and product innovation. OMEGA, The International Journal of Management Science, 3, 6, 639–656. Vickery, G. and Blau, E. (1989) Government policies and the diffusion of microelectronics. Paris, Organisation for Economic Co-operation and Development. Zahra, S.A. and George, G. (2002) Absorptive capacity: a review, reconceptualisation, and extension. Academy of Management Review, 27, 2, 185–203. Notes 1. The development of the tool was sponsored by the World Bank as part of the World Bank Project on Korea and the Knowledge-Based Economy reported on later in this paper. A work paper, on which this paper is based, Rush et al. (2006), is to be published in the workshop proceedings of Innovation Cultures: Challenges and Learning Strategy, Prague, 2005. 2. By ‘complexity,’ we refer to the number of different elements in the environment and their potential interaction. Thus, highly complex environments are inherently uncertain and unpredictable and require sophisticated capabilities to deal with them (see Allen, 2001; McKelvey, 2004). R&D Management 37, 3, 2007 235 Howard Rush, John Bessant and Mike Hobday 3. Unaware or passive firms typically benefit from those policies which include outreach and missionary services, benchmarking, strategic development and support for technology search, demonstration projects and other forms of facilitated experience sharing. 4. Policy mechanisms that are being used with reactive firms include benchmarking, strategic development processes, support for technology search, access to funding for specific projects, demonstration projects and support for technology transfer, experience sharing and assistance with training and development. 236 R&D Management 37, 3, 2007 5. Strategic firms benefit from those policies which include technology signposting, access to funding for specific projects and support for technology transfer from, for example, short-term consultancy. 6. Creative firms can benefit from technology signposting, access to funding for specific projects, support for technology transfer (consultancy), major technical project contracts and network access. 7. The questions used in the ‘short-tool’ version do not match directly onto the full tool as, by necessity, they are limited and must cover wider issues and the tool is mainly intended as a ‘filtering’ tool for policy makers. r 2007 The Authors Journal compilation r 2007 Blackwell Publishing Ltd