THE ROLE OF AGGREGATION IN THE MEASUREMENT OF IT-RELATED
ORGANIZATIONAL INNOVATION
Forthcoming in: Management Information Systems Quarterly , 2001, Volume 24, Issue 4.

THE ROLE OF AGGREGATION IN THE MEASUREMENT OF IT-RELATED
ORGANIZATIONAL INNOVATION
ROBERT G. FICHMAN is an Assistant Professor of MIS at the Carroll School of
Management, Boston College. He received B.S. and M.S. degrees in Industrial and Operations
Engineering from the University of Michigan, and a Ph.D. in Information Technologies from the
MIT Sloan School of Management. His current research interests include IT adoption and diffusion, management of technology and innovation, software development tools and methods, and real options. He has published in IEEE Computer, Information Systems Research,
Management Science , Sloan Management Review and elsewhere.
The author gratefully acknowledges the financial support provided for this research by the MIT
Center for Information Systems Research, and many helpful comments from the senior editor, associate editor and four anonymous referees.

THE ROLE OF AGGREGATION IN THE MEASUREMENT OF IT-RELATED
ORGANIZATIONAL INNOVATION
Abstract
The extent of organizational innovation with IT, an important construct in the IT innovation literature, has been measured in many different ways. Some measures are more narrowly focused while others aggregate innovative behaviors across a set of innovations or across stages in the assimilation lifecycle within organizations. There appear to be some significant tradeoffs involving aggregation. More aggregated measures can be more robust and generalizable and can promote stronger predictive validity, while less aggregated measures allow more context-specific investigations and can preserve clearer theoretical interpretations. This article begins with a conceptual analysis that identifies the circumstances when these tradeoffs are most likely to favor aggregated measures. It is found that aggregation should be favorable when: (1) the researcher's interest is in general innovation or a model that generalizes to a class of innovations, (2) antecedents have effects in the same direction in all assimilation stages, (3) characteristics of organizations can be treated as constant across the innovations in the study, (4) characteristics of innovations can not be treated as constant across organizations in the study, (5) the set of innovations being aggregated includes substitutes or moderate complements, and (6) sources of noise in the measurement of innovation may be present. The article then presents an empirical study using data on the adoption of software process technologies by 608 US based corporations.
This study—which had circumstances quite favorable to aggregation—found that aggregating across three innovations within a technology class more than doubled the variance explained compared to single innovation models. Aggregating across assimilation stages had a slight positive effect on predictive validity. Taken together, these results provide initial confirmation of the conclusions from the conceptual analysis regarding the circumstances favoring aggregation.
Keywords : Assimilation, Innovation Adoption, Innovation Diffusion, Implementation, Infusion,
Routinization, Measurement
ISRL categories : DD05, DD0501, DD0502, EL05, FD,

The ability to innovate has always been an important contributor to organizational success. Broadly speaking, research on IT innovation has been concerned with such questions as: What distinguishes organizations that are most predisposed to innovate? How can organizations create a more innovation-friendly climate, and execute more effectively throughout the innovation process? What constitutes readiness to adopt a particular technology, and how can organizations assess and improve this readiness? In general, it has been found that innovation is more likely among organizations that have the necessary resources to innovate (e.g., due to organizational slack or technical expertise), a strong motivation to innovate (e.g., due to high perceived benefits or needs), and a general organizational climate conducive to innovation (e.g., due to positive managerial attitudes toward change).
A common element in this stream of research is the dependent construct, namely, the degree of ITrelated organizational innovation. Many different measures have been used to capture this construct, including earliness of adoption, frequency of adoption (e.g., the number of adoptions across a set of innovations), and various dimensions of extent of implementation (e.g., internal diffusion, infusion, and routinization). Some of this diversity reflects understandable differences in researcher objectives and availability of data. However, this diversity also reflects tradeoffs made by the researchers, especially tradeoffs about the degree to which measures should aggregate across innovative behaviors.
Aggregation can take two basic forms: (1) aggregating innovative behaviors across innovations
(such as when number of adoptions is used), and (2) aggregating across the assimilation lifecycle within organizations (such as when behaviors that occur in both early and late stages of assimilation are reflected in the measure). As this paper will argue in some detail, the main tradeoffs favoring

more aggregated measures are that they tend to be more robust to non-systematic effects, more generalizable, and stronger in terms of predictive validity. Conversely, less aggregated measures permit an intensive focus on understanding particular innovative behaviors, allow the use of innovation and/or stage-specific hypotheses, and can promote clearer theoretical interpretations.
Measures that aggregate across innovations used to dominate in the literature, and some of the earliest studies by IT researchers employed such measures (e.g., (Zmud, 1982)). However, the use of aggregated measures has waned in favor of single-innovation measures, even in studies that seek to generalize beyond a specific technology and that could, in principle, use aggregated measures
(see, for example, (Bretschneider and Wittmer, 1993; Cooper and Zmud, 1990; Grover, et al., 1997;
Rai, 1995)). While it is difficult to know with certainty, it seems likely that some sharp conceptual criticisms of aggregated measures first articulated by Downs and Mohr (1976), and subsequently embraced by other influential scholars (Rogers, 1983; Tornatzky and Klein, 1982) may have had a discouraging effect on the use of aggregated measures, despite subsequent work that suggests such measures can be useful (Damanpour, 1991). Or, it could be that a shift in attention among innovation researchers to more context-specific research questions (Rogers, 1995, pg. 390) accounts for the move toward narrower innovation measures, even though—as the current study will show— some kinds of aggregated measures can still be used to study contextual factors.
In any event, it appears the time has come for a fresh look at the role of aggregation in the measurement of IT-related innovation .
To begin with, there has yet to be a systematic analysis of the apparent tradeoffs presented by more aggregated measures. In addition, the nature of IT innovation has been evolving in a way that argues for a greater emphasis on aggregated measures going forward. Business models and their constituent processes are increasingly IT-enabled, which means
IT is touching more parts of the business and in more fundamental ways. In addition, IT itself has become increasingly highly integrated, with umbrella concepts like ERP, datawarehousing and
- 2 -

electronic commerce referring to a cluster of related technologies. This shifts the focus from an organization's ability to innovate with respect to a narrowly defined, single IT innovation to its capability to innovate with respect to an array of possibly interrelated innovations. Furthermore, there has been a growing recognition of the implementation challenges presented by complex IT innovations (Fichman and Kemerer, 1999; Swanson and Ramiller, 1997) and this argues for attention to measures that capture an organization's propensity to innovate across all stages of the assimilation lifecycle, rather than just a part of the process.
The goal of this research is to shed light on the issue of aggregation in the measurement of ITrelated innovation, and in particular, to develop prescriptions for when the tradeoffs are most likely to favor aggregation. Toward this end, the paper begins with a description of popular innovation measures. Next it provides an analysis of the circumstances most favorable to the use of aggregated measures. This is followed with an empirical study that analyzes the predictive validity of several measures under circumstances that appear favorable to aggregation. The purpose of this analysis is to provide an initial confirmation of prescriptions developed earlier, the expectation being that when tradeoffs favor aggregated measures, this should result in greater predictive validity. Finally, based on this analysis, some conclusions are offered on the use of aggregated measures in IT innovation research.
Organizational innovation has been defined as "the adoption of an idea or behavior that is new to the organization adopting it" (Daft, 1978 p. 197). This broad definition permits many possibilities as far as what it means for an organization to innovate. In the IT diffusion literature, researchers have usually conceptualized innovation as pertaining to the organizational initiation, adoption and/or implementation of one or more emerging technologies (Fichman, 2000; Prescott and Conger, 1995).
Organizations have been viewed as more innovative when they exhibit these sorts of behaviors earlier, more frequently, and/or more intensively. Table 1 summarizes some of the more common
- 3 -

measures of IT innovation. Appendix A provides several representative examples of operationalizations for each measure, with a particular emphasis on IT studies. Useful additional discussions of measurement issues can be found in (Downs and Mohr, 1976; Massetti and Zmud,
1996; Saga and Zmud, 1993; Tornatzky and Klein, 1982; Zmud and Apple, 1992).
T ABLE 1: M EASURES OF O RGANIZATIONAL I NNOVATION
Measure
Earliness of Adoption
Internal Diffusion
Infusion
Routinization
Assimilation
Definition
Relative earliness of adoption within a population of potential adopters.
The extent of use of an innovation across people, projects, tasks, or organizational units.
The extent to which an innovation's features are used in a complete and sophisticated way.
The extent to which an innovation has become a stable and regular part of organizational procedures and behavior.
The extent to which an organization has progressed through the assimilation lifecycle for a particular innovation (stretching from initial awareness to full institutionalization).
Aggregated Initiation
Aggregated Adoption
The frequency or incidence of innovation initiation.
The frequency or incidence of innovation adoption.
Aggregated Implementation The degree of implementation of innovations that have been adopted.
Aggregation Across Innovations and Assimilation Stages
The measures in Table 1 differ along two important dimensions: (1) the degree to which they commingle behaviors across innovations, and (2) the degree to which they commingle behaviors across the assimilation lifecycle. They also differ in other ways—some measures are comparatively rich (e.g., infusion); some are better suited for technologies that are adopted person-by-person, group-by-group or project-by-project (e.g., internal diffusion); some are better when adoption itself is the event of greatest interest (e.g., earliness of adoption). However, I highlight the two dimensions identified above because they go to the heart of key tradeoffs surrounding measurement.
Regarding the first dimension, measures such as aggregated initiation, aggregated adoption, and aggregated implementation commingle behaviors across innovations. Organizations exhibiting innovative behaviors with respect to several innovations are scored more highly. The extent of the
- 4 -

commingling is determined by the number of innovations in the set. In IT research this has ranged from as few as three (Zmud, 1982) to as many as fifteen (Grover and Goslar, 1993).
Regarding the second dimension, some measures focus on a more narrow slice of the assimilation process in organizations while others commingle behaviors spanning more of the process. Initiation, for example, only captures whether an organization has ever initiated the evaluation of an innovation. Infusion captures the degree of implementation among adopters, but is not concerned with differences in innovative behaviors up to the point of adoption. Earliness of adoption is determined by three innovative behaviors: earliness of initiation, speed of evaluation, and a positive go/no-go decision. Assimilation stage takes commingling to its greatest extreme. This measure implicitly combines the earliness of initiation, speed at which subsequent stages are traversed, and the absence of rejection, implementation failure, or early discontinuance—all within a single innovation score.
Whether intended or not, many studies have in effect aggregated across both innovations and stages to some degree. For example, the implementation scale first employed by Zmud (1982), and later used by others (Grover and Goslar, 1993; Nilakanta and Scamell, 1990) assigns a score as the average extent of implementation for innovations that have been adopted. However, since there has been no control for earliness of adoption in these studies, this means that organizations that initiated earlier, or traversed the initiation and adoption stages more quickly will have had more time to reach later stages of implementation.
1 Therefore, it is likely that this measure involves some degree of commingling of behaviors across stages.
In sum, a wide variety of measures—with varying degrees of aggregation across innovations and stages—have been used in IT innovation research. I now turn to an analysis of key issues related to
1 However, Rai (1995) did control for time when studying the determinants of CASE tool implementation.
- 5 -

aggregation, with the goal of developing some prescriptions for when aggregation is most likely to be beneficial.
The question facing innovation researchers today is not whether aggregation is permissible, but rather, under what circumstances should the potential benefits of aggregation (e.g., greater robustness and generalizability) outweigh the potential costs (e.g., loss of context-specificity and diminished clarity of theoretical interpretation). In this section I will argue that the relative merits of aggregation turn on six issues: (1) the primary objective of the research, (2) the validity of generalization across assimilation stages, (3) the effects of organizational characteristics, (4) the effects of innovation characteristics, (5) the effects of innovation substitutes and complements, and
(6) the effects of reporting errors and idiosyncratic adoption.
(1) Primary Research Objective
Most studies of organizational innovation with IT have been driven by one (or more) of three research objectives. These include: (1) identifying the determinants of innovation with respect to some particular technology , (2) identifying the determinants of generally "innovative" organizations , and (3) determining the role of one or more theoretical factors in innovation, but not with an overriding interest in the technology or innovative organizations per se . I refer to these styles of research as technology-focused , innovativeness-focused , and factor-focused , respectively.
Technology-focused studies seek to develop a model that explains innovation with respect to a particular technology or class of technologies with similar characteristics (see, for example, (Grover and Goslar, 1993; Howard and Rai, 1993). These studies tend to identify explanatory factors that should be most salient given the nature of the innovation under study. The goal here is to maximize explanatory power for one innovation (or innovation class) that is viewed to be especially important, with the goal of deriving managerial implications for how to successfully adopt and diffuse that
- 6 -

particular technology or technology class. In these studies, generalization beyond the innovation at hand has usually been a secondary concern at most. As would be expected, most such studies use single-innovation measures, although when the focus is on a class, aggregated measures have been used (Grover and Goslar, 1993).
Innovativeness-focused studies, by contrast, are concerned with identifying the properties of organizations that innovate over time in a variety of settings. As might be expected, these studies typically aggregate across technologies. However, they have avoided measures that also aggregate across assimilation stages. In fact, such studies have often sought to minimize aggregation across stages in order to examine whether certain variables have differently-directioned effects on different stages (Damanpour, 1991). Interestingly, it appears that in the last decade only two studies by IT
researchers have employed a more general notion of organizational innovativeness with IT as the
outcome variable, and both used unconventional measures to capture this concept (see (Armstrong and Sambamurthy, 1999; Lind and Zmud, 1991)).
2
Factor-focused studies are concerned with understanding the role of one or more particular theoretical factors in determining innovation (e.g., (Cooper and Zmud, 1990; Fichman and Kemerer,
1997; Grover, et al., 1997; Nilakanta and Scamell, 1990; Rai, 1995; Zmud, 1982). These studies range from ones with a focus on a particular factor (Bretschneider and Wittmer, 1993) to those testing a more general model of innovation (Grover, et al., 1997). This type of study has been the most common, and IT researchers have employed both unaggregated and highly aggregated measures of innovation. Most of these studies are concerned with generalization to at least the level of a class of related technologies, even when single-innovation measures have been used (e.g.,
(Cooper and Zmud, 1990)).
- 7 -

Innovation classes can be defined narrowly or broadly, and the same innovation can belong to more than one class. Object-oriented programming, for example, could be treated as an example of a programming innovation, a software process innovation, an IT innovation, a complex organizational technology, a radical innovation—all depending on what the researcher is trying to accomplish with the study. The key point is to identify the level of abstraction that best fits the theoretical model to be tested, and to use the distinguishing characteristics of innovations taken at that level of abstraction to help identify which factors will be most salient in the context of the intended study.
When a theoretical model maps cleanly to the level of some innovation class, this encourages the use of measures that aggregate across innovations in that class to promote generalizability. Even so, there can be compelling reasons to prefer a single innovation measure even when the intended level of generalization would permit aggregation. For example, a researcher may be interested in factors that require innovation-specific operationalizations, in which case it would be more appropriate to
develop separate models for each innovation.
3
Other things equal, it seems clear that innovativeness-focused studies should employ aggregated measures unless there is some compelling reason not to. It is equally clear that technology-focused studies should avoid measures that aggregate across technologies unless the focus is on a technology class. It is for the third category, factor-focused studies, that aggregation has the most complex set of tradeoffs. Here the decision whether or not to aggregate will turn on the remaining five issues described below.
2 Lind and Zmud (1991) defined innovativeness as the frequency of ideas and requests for new IT applications by users and expert ratings of a department's IT innovativeness . Sambamurthy (1999) used executive reports of relative success in applying IT to support strategy, marketing, and logistics as a measure of the firm's extent of innovation with IT.
3 Alternatively, when the adoption decision is the unit of analysis, Downs and Mohr (1976) recommend using the adoption-decision design. See (Meyer and Goes, 1988) for an example.
- 8 -

(2) Generalization Across Assimilation Stages
When considering the role of aggregation in measuring innovation, a key concern is the extent to which the underlying theoretical model can be generalized across the assimilation lifecycle within organizations. For example, theories driven by organizational learning generalize across stages because significant knowledge barriers exist in all stages of assimilation (Fichman and Kemerer,
1997). Meyer and Goes (1988) have also developed a model with several predictors that span assimilation stages. These kinds of models focus on variables that are expected to have the same direction of influence regardless of assimilation stage. In the two studies mentioned above, assimilation stage reached by a certain date was used as the outcome variable, which, as explained earlier, implicitly aggregates innovative behaviors across all the stages each organization has traversed as of that date. This is not to say these studies assume all stages are exactly the same, or that there is no value in more narrow studies focused on particular stages. Rather, it is assumed that variables do exist (e.g., resources, fit, expertise, competitive environment) that promote (or hinder) progress throughout the assimilation process and will show up as significant even though there may be other factors that have less consistency of influence.
Conversely, some theories argue that key predictors of innovation tend to have much different  if not opposite  impacts on different stages in the assimilation lifecycle. In their critique of aggregated measures, Downs and Mohr (1976) discuss the "organic" versus "mechanistic" dichotomy
(Aiken and Hage, 1971). They explain that low centralization and formalization (as found in "organic" organizations) should lead to a greater willingness to embrace new ideas, and hence should encourage the initiation of innovation. However, they argue these same characteristics should hinder adoption and implementation by inhibiting the development of organizational consensus surrounding the innovation. Conversely, high centralization and formalization (as found in "mechanistic" organizations) should tend to hinder initiation, but promote adoption and implementation.
Others innovation researchers have made similarly structured arguments in support of the idea of
- 9 -

differently-directioned effects (Grover and Goslar, 1993; Tornatzky and Klein, 1982; Zaltman, et al., 1973; Zmud, 1982). When important determinants of innovative behaviors do have differentlydirectioned effects, then aggregating across stages will be problematic, because the facilitating influence of variables in one stage will be offset by the inhibiting effect in other stages, resulting in a loss of explanatory power and instability of results across studies (Downs and Mohr, 1976).
However, growing evidence suggests that differently-directioned effects may not figure as prominently in the study of innovation as researchers originally hypothesized. Damanpour (1991), in a meta-analysis of 23 studies found that while the strength of effects varied depending on whether aggregated initiation, aggregated adoption or aggregated implementation was used to operationalize innovation, these effects were almost always in the same direction. In studies of IT innovation,
Zmud found some support for the differently-directioned effects hypothesis, with formalization having a negative (though insignificant) correlation with initiation and positive correlation with adoption and implementation (1982). However, Grover and Goslar (1993) found that uncertainty and centralization had significant effects in the same direction in all stages.
In sum, the extent to which a theoretical model generalizes across assimilation stages will depend on the study context and included variables. Where such generalization appears warranted, then the tradeoffs related to robustness, generalizability and clarity of theoretical interpretation will tip toward aggregation across stages. When such generalization contradicts plausible hypotheses, aggregation across stages should be avoided.
(3) Characteristics of Organizations
Characteristics of organizations such as size, structure and expertise are important determinants of innovation in general and also with respect to IT. However, although research models often treat these sorts of characteristics as if they were a uniform property of an organization with a single value, in reality, the measured values for these characteristics can vary from unit to unit within an
- 10 -

organization. And since different innovations may be adopted by different organizational units, the measured values for these organizational characteristics can therefore also vary depending on the innovations included in a study. For example, if an organization has a centralized manufacturing department but a decentralized sales department, then the value for centralization will be high in a study of manufacturing innovations and low in a study of sales innovations. Such characteristics are referred to as secondary organizational characteristics, to distinguish them from primary characteristics, which always have a constant value (e.g., firm size, industry sector) regardless of the innovation being considered (Downs and Mohr, 1976).
The typical approach in studies using aggregated measures is to assign each organization a single overall score for each organizational characteristic in the study. This same approach has been taken in studies of IT innovation. However, when the characteristic is secondary, Downs and Mohr argue this approach can average away potentially explainable variance in the observed relationship between that characteristic and measured innovation. Departments with high centralization will be given the same value for the independent variable "centralization" as those in departments with low centralization. Downs and Mohr argue that this can cause poorly predictive models and an instability of findings across studies. As a result, they suggest that aggregated adoption should be avoided in the study of organizational innovation. Yet, recent evidence casts doubt on these conclusions. Damanpour, in the aforementioned meta analysis, reports that studies using more highly aggregated measures had stronger , rather than w eaker , statistical confirmation of expected theoretical relationships compared to those using less aggregated measures (1991).
Although Damanpour does not attempt to reconcile his results with the assertions of Downs and
Mohr, some candidate explanations exist. First, it could be that organizations tend to be more uniform across units in terms of secondary characteristics than Downs and Mohr had assumed.
Also, it is worth noting that some of the studies in the meta analysis only aggregated innovations
- 11 -

adopted by a single organizational unit, and in such cases, many secondary characteristics can be treated as though there were primary because they should have a constant score for all innovations in that study. Studies of IT innovation have often aggregated innovations adopted just within the IT department (Nilakanta and Scamell, 1990; Zmud, 1982; Zmud, 1984). In fact, one advantage of IT innovation research is the rapid pace of innovation in the tools and techniques used to develop and administer IT systems, and the resulting wide variety of innovations adopted for use within the IT unit itself. In addition, the benefits of aggregation can compensate for the possible adverse effects caused by either mis-estimating or omitting secondary characteristics. (As explained below, these benefits include the moderating of "noise" potentially introduced by omitted innovation characteristics, reporting errors, idiosyncratic adoption, and innovation substitutes).
In conclusion, while the potential effects of secondary characteristics of organizations are a concern for studies that use aggregated measures, these concerns can be avoided by limiting aggregation to innovations adopted by the same organizational unit or by focusing on contexts where secondary organizational characteristics are not likely to vary strongly across innovations. In such cases the tradeoffs related to robustness, generalizability and theoretical interpretation will be resolved in favor of aggregation.
(4) Characteristics of Innovations
Just as some organizational characteristics vary depending on the innovation being considered, some innovation characteristics vary depending on the organization being considered. Such characteristics are called secondary characteristics of innovations (Downs and Mohr, 1976). Compatibility is a good example, since the same innovation can vary dramatically in how compatible it is for different
organizations (Meyer and Goes, 1988; Ramiller, 1994). Complexity, relative advantage, cost, and
- 12 -

many other characteristics of an innovation can also vary across organizations.
4 (Examples of primary characteristics—which do not vary across organizations—include those associated with the industry context surrounding the innovation, such as the size of the installed base.)
Due to differences in research interests and practical limitations on how much data can be captured in a single study, it is common for innovation characteristics to be omitted from a research model.
As with any omitted variables, this can be a source of noise—but aggregation should moderate it.
To see why, let us suppose that a researcher has developed a model where an organization's innovative capacities predict innovation. Then let us suppose there are two organizations, A and B, with different innovative capacities. Organization A, which has a high capacity to innovate, defers the adoption of Technology X because it happens to be especially incompatible with existing needs, skills, work practices or technical infrastructure. This same organization adopted Technologies Y and Z, consistent with theoretical predictions. Conversely, Organization B, which has a profile that suggests low capacity to innovate, nevertheless chooses to be on the leading edge for Technology X because it happens to be highly compatible. This same organization passed on Technologies Y and
Z. In a single-innovation design based on Technology X that does not control for innovation characteristics, Organization A would be given a lower score for innovation than Organization B in spite of the model's prediction to the contrary, and in spite of the fact that Organization A does in fact tend to innovate more often than Organization B.
In sum, in a single innovation design the omission of secondary innovation characteristics may introduce noise that makes it more difficult to discern the effects of included predictor variables. In an aggregated design, by contrast, omitted secondary innovation characteristics should pose less of a problem because their effects will tend to be smoothed out across innovations. In the above
4 This is perhaps especially true of IT innovations due to the high levels of interpretive flexibility often seen in these technologies (Orlikowski, 1996).
- 13 -

example if a researcher were to aggregate across Technologies X, Y and Z, then Organization A, which was expected to be more innovative, would in fact be scored as more innovative than
Organization B. Thus, when secondary innovation characteristics plausibly exist and are not otherwise controlled for, the tradeoffs related to robustness and predictive validity will favor more aggregated measures.
(5) Innovation Substitutes and Complements
It would appear that aggregating innovation substitutes or moderate complements might have special advantages compared to aggregating unrelated innovations, aggregating strong complements, or using single-innovation measures.
When an innovation has one or more substitutes diffusing at the same time, aggregation across these substitutes may dampen a subtle source of noise in the measurement of innovation. To see why, suppose a pair of emerging innovations were perfect substitutes—that is, an organization might adopt one or the other, but would never adopt both. If a researcher were to capture adoption for just one in the pair, this would wrongly assign a score of low innovation to every organization that had instead chosen to adopt the other innovation. Aggregating across both innovations, by contrast, would produce an appropriate score for all organizations.
The aggregation of complementary innovations, by contrast, is likely to provide lesser benefits from a predictive validity standpoint. In fact, aggregating across perfect complements—technologies that were always adopted together (e.g., computers, monitors and keyboards)—would have no substantive effect on the measurement of innovation, since the aggregated measure would be perfectly correlated with each of the individual innovation measures included in the aggregate. The more interesting situation arises with imperfect complements. The same technologies can be more or less complementary, given organizational contingencies and variations in primary and secondary organizational characteristics. For example, customer relationship management (CRM), supply
- 14 -

chain management (SCM), and enterprise resource planning (ERP) solutions are imperfect complements, i.e., there are no technical constraints that force adoption of all three applications.
Yet, obtaining value from implementing these imperfect, moderate complements requires greater IT innovation capability than implementing only one of these applications. In addition, we might expect organizations to react more consistently to complements than to unrelated innovations, which could moderate concerns that aggregation sometimes amounts to mixing apples and oranges.
Finally, we might have a substantive interest in organizations that tend to adopt certain clusters of complementary innovations, under an assumption that such organizations will be more likely to benefit from the adoption of each innovation in the cluster. These arguments should be especially salient for studies of IT innovation going forward, given the trend toward umbrella concepts referring to clusters of interrelated technologies.
In sum, it appears that aggregating substitutes should lead to the greatest increase in predictive validity. Aggregating moderate complements should have a lesser (though still beneficial) effect on predictive validity but should lead to results that have a clearer theoretical interpretation.
(6) Reporting Errors and Idiosyncratic Adoption
As described previously, critics have identified some circumstances for when aggregated measures may introduce noise into the measurement of innovation (e.g., where organizational characteristics vary across innovations in the set). However, aggregation may also serve to reduce noise by countering the ill effects of reporting errors and idiosyncratic adoption. On the first point, whenever informants are queried about adoption of an innovation some will mis-report their organization's true level of adoption. This can be especially worrisome in the case of more complex, abstract or multifaceted innovations often seen in the IT domain (e.g., groupware, CASE), since these innovations can mean different things to different respondents. By aggregating across innovations, these errors will tend to be smoothed out, thus producing a more reliable overall innovation score for each organization.
- 15 -

On the second point, serendipity can play an important role in when—or whether—an organization adopts an innovation, and whether it continues to assimilate that innovation. A key executive might happen upon an innovation in a trade press article and take on advocacy of the innovation. Ramiller has argued this often an issue with IT innovations (Ramiller, forthcoming). Or, a newly hired employee might serve as champion (Howell and Higgens, 1990) for an IT innovation that has provided benefits at a prior place of work. Likewise, an organization that is otherwise fairly innovative can be a laggard with respect to a particular IT innovation because an influential individual or group happens to find it threatening (Markus, 1983). By diluting the effects of nonsystematic variables, aggregation can be seen as a parsimonious way of producing measures that are more robust to these sources of noise. To the extent these sources of noise are expected to be present and can not be feasibly eradicated by other means, this should resolve the tradeoffs related to robustness of measurement and predictive validity in favor of greater aggregation.
Summary
To summarize the prior sections, it appears that there are several circumstances that should tip tradeoffs in favor of more aggregated measures (see Table 2 below). (Note to editor: this table should have check marks in the columns not the numeral "4".)
T ABLE 2: C IRCUMSTANCES F AVORABLE TO A GGREGATION
Description of Circumstances
Across
Innovations
1.
The primary research objective is to identify determinants of organizational innovativeness in general or with respect to some technology class.
2.
Important theoretical factors generalize across stages, i.e., they are expected to affect assimilation stages in the same direction.
3.
The organizational characteristics included in the study have the same value for each innovation in the set (e.g., because all innovations in the study are adopted by the same organizational unit) rather than varying across innovations
4.
The study does not control for the effects of innovation characteristics that vary across organizations.
5.
The innovations in the set include substitutes or mild complements (as
✔
✔
✔
✔
- 16 -
Across
Stages
✔
✔

opposed to strong complements or unrelated innovations).
6.
There are concerns about reliability of measurement in light of possible reporting errors or idiosyncratic adoption decisions.
✔ ✔
In prior sections, I described several measures of innovation and presented some potential tradeoffs pertaining to the use of aggregated measures. In particular, I described some circumstances that were favorable to aggregation and that should lead to greater robustness, generalizability and increased predictive validity while moderating concerns about clarity of theoretical interpretation.
As a complement to this conceptual analysis, I now present an empirical analysis that examines the extent to which aggregation improves predictive validity under favorable circumstances.
The analysis will use data on the adoption of three software process innovations: (1) relational database management systems (RDB), (2) computer aided software engineering tools (CASE), and
(3) object-oriented programming languages (OOP). Several OLS regression models will be estimated with a common set of independent variables. These variables will be used to predict different measures of innovation, including earliness of adoption, infusion, assimilation, aggregated adoption and aggregated assimilation. As described in more detail in the sections below, this study meets all of the criteria identified earlier for when aggregation is likely to improve predictive validity: (1) the theoretical model was developed to generalize to an innovation class (software process innovations) and predictor variables, with one exception, were operationalized at this level of abstraction; (2) the model is intended to generalize across stages of assimilation, (3) characteristics of organizations included as predictors can be treated as primary because they are all measured with regard to the IT department and all three technologies were adopted by IT departments, (4) the study does not include secondary characteristics of innovations as control variables, (5) it appears that the technologies in this study are moderate complements rather than strong complements or unrelated innovations, (6) some degree of measurement error and
- 17 -

idiosyncratic adoption are likely present in this study. As a result, it appears that tradeoffs described earlier should strongly favor more aggregated measures.
A Theoretical Model of Innovation with Software Process Technologies
The theoretical model employed here is summarized in Figure 1. Together with a collaborator, I
developed this model to investigate the role of organizational learning-related factors in the
assimilation of complex organizational technologies subject to knowledge barriers in diffusion
(Fichman and Kemerer 1997).
5 We viewed software process innovations (SPIs) as exemplars of such technologies.
Learning-
Related Scale
Diversity
OOP Related
Knowledge
IT Size
Specialization
Education
Organizational
Innovation with
Software
Process
Innovations
(CASE, RDB,
OOP)
F IGURE 1: O RGANIZATION I NNOVATION WITH S OFTWARE P ROCESS I NNOVATIONS
In the prior study we focused on OOP, and used three variables  Learning Related Scale , Diversity , and Related Knowledge  to test whether organizations better positioned to incur the burden of assimilating complex technologies were more likely to initiate and sustain assimilation of such technologies. The hypothesized positive effects of these variables were strongly confirmed. Three additional variables, IT Size , Specialization and Education were employed as controls. In the
5 The original study included three additional control variables: Host Organization Size, Environmental Complexity, and
Industry Sector. To shorten and simplify the analysis, these three control variables  which exhibited the weakest direct associations with innovation in the original study  were excluded from the present study.
- 18 -

present study, the distinction between theory and control is not relevant; all variables are treated simply as antecedents expected to predict innovation with software process technologies, and the three SPIs are viewed as prominent, well-understood examples of such technologies. The goal here is to compare different measures of innovation in terms of predictive validity, rather than to test a theoretical model or to draw descriptive conclusions about the state of diffusion of the technologies.
The theoretical rationales for the independent variables are summarized in Table 3 below. With one exception all independent variables were measured at the level of the IT unit or the development group within the IT unit. Since these measures are not specific to particular SPIs, they can in principle be used to predict adoption of any SPI. Also, they should predict measures of adoption that aggregate across SPIs. The exception is OOP Related Knowledge, which, as the label suggests, was operationalized with respect to OOP, and therefore should not be expected to explain variance in single-innovation models for CASE or RDBs or models that aggregate across innovations.
- 19 -

Variable
1. Learning-
Related Scale
2. Diversity
3. OOP Related
Knowledge
4. IT Size
5. Education
6. Specialization
Definition
T ABLE 3: I NDEPENDENT V ARIABLES
Rationale
Scale of activities over which learning costs can be spread.
Diversity of knowledge and activities related to applications development.
Extent of organizational knowledge in domains related to the focal innovation.
The size of the IT function.
Level of education of the IT staff at the site.
Extent of IT staff specialization.
Organizations with greater learning-related scale have a greater opportunity to amortize learning costs, and hence, can innovate more economically (Attewell, 1992).
Individuals and organizations that are more diverse in the knowledge they possess are more likely to absorb and appreciate the value of any given item of new information that is encountered, and so, have a greater capacity to innovate (Cohen and Levinthal, 1990). Diversity also increases the likelihood that an organization will have at least one domain that is sufficiently "innovation ready" for the innovation to be introduced to the organization (Swanson, 1994).
Existing organizational knowledge facilitates the absorption of new
(but related) knowledge needed to innovate successfully (Cohen and
Levinthal, 1990). It also diminishes the "distance" a firm must travel to get from its current bundle of knowledge and skills, to one that encompasses the intended innovation (Pennings and Harianto, 1992).
Larger organizations tend to have greater professionalism, more slack resources, and more specialization, all of which promote innovation
(Tornatzky and Fleischer, 1990, pg. 163).
More highly educated employees tend to be more professional
(Zmud, 1982). Increased professionalism is associated with greater boundary spanning activity, self-confidence and a commitment to move beyond the status quo ((Pierce and Delbecq, 1977), as cited in
Damanpour (1991)).
A greater variety of specialists provide a broader knowledge base
((Kimberley and Evanisko, 1981) as cited in Damanpour (1991)) and increase the sharing of ideas ((Aiken and Hage, 1971), as cited in
Damanpour (1991)).
The rationale for the selection of independent variables was that they relate to managing the organizational learning required by new technologies. Since significant learning occurs in all stages of assimilation, it is expected that the model should apply to all assimilation stages. Therefore, the model generalizes across both SPIs and stages, which suggests that it should apply to all of the measures of organizational innovation included in this study.
Survey Methods
The data set employed here was constructed using survey responses collected from over 600 IT departments located in the United States. Of the SPIs covered in the dataset, the most extensive data were captured for OOP in order to have more data available to support a separate study that used
OOP as the focal technology (Fichman and Kemerer, 1997). The sampling unit was the IT department at individual sites. Informants were instructed to consider just their own site in answering
- 20 -

questions. A probability sample of 1500 sites was extracted from a list, maintained by International
Data Corporation, of 40,000 US sites with computers installed. To qualify for the sampling frame, the site had to meet several criteria designed to ensure a well-informed respondent and the existence of custom developed applications at the site (see (Fichman and Kemerer, 1997) for details). The survey was administered in 1994 via computer disk, an approach whereby respondents insert the survey software into their PCs and are automatically led through the questionnaire items (Saltzman,
1993) .
The questionnaire provided a list of several prominent, commercially available examples of each technology to help ensure respondents were operating from common definitions of what counts as an instance of each of the technologies. A total of 608 usable responses were received, for a 45% response rate. The vast majority of responding sites were typical corporate information systems organizations, with mainframe or midrange computers as their primary host environment (81%).
The median reported size of the total on-site IT staff (including development, technical support and operations) was 16. An examination of possible response bias lead to the conclusion that this was unlikely to be a major factor in this study (see (Fichman and Kemerer, 1997) for details.)
Measures for Independent Variables
Table 4 below provides a description of the indicators for the independent constructs and how they were measured. The multi-indicator independent constructs were viewed as formative , rather than reflective . That is, the indicators were viewed as causing or composing the construct, rather than being alternative reflections of the construct (Chin, 1998; Fornell and Bookstein, 1982; Johansson and Yip, 1994)). When constructs are formative, latent variables are created using a linear combination of the indicators. In this case, an unweighted average of the standardized indicators was used because a sensitivity analysis showed that more complex weighting schemes (e.g., as suggested by principal components analysis) had a negligible impact. While there is no expectation that formative constructs must exhibit high convergent validity, as a matter of fact all of the indicators for the multi-indicator constructs had correlations of at least r=.75 with their construct
- 21 -

scores. To evaluate discriminant validity, criteria from the multitrait-multimethod (MTMM) technique were employed (Campbell and Fiske, 1959). The indicator correlation matrix was examined to find instances where cross-construct correlations for an indicator exceeded withinconstruct correlations. No such instances were found. In addition, an oblique principal components cluster analysis produced the expected clustering of indicators on their associated constructs (results available from the author).
Construct
Learning-
Related Scale
OOP Related
Knowledge
Diversity
IT Size
Specialization
Education
T ABLE 4: M EASURES FOR I NDEPENDENT V ARIABLES
Ind Indicator descriptions
L1 Log (number of application developers at site * percentage of applications-related effort attributable to new systems)
L2 Log (number of application developers at site * percentage of applications-related effort attributable to new systems and enhancements)
K1 Percentage of development staff with experience programming in
C
K2 Percentage of development staff with experience developing client-server applications
K3 Percentage of development staff with experience developing graphical user interfaces
D1 The number of different programming languages used by at least
5% of the development staff in 1993 (assembly language, Cobol,
C, other non-OO 3GL, non-OO 4GL)
D2 The number of different runtime platforms accounting for at least
5% of new development over last 3 years (centralized mainframe, centralized midrange, client-server (CS) with mainframe host, CS with midrange host, CS with desktop host, networked workstations/PCs, standalone workstations/PCs)
I1 An ordinal variable capturing level of external IT spending within respondent's span of influence (nine categories)
I2 An ordinal variable capturing number of IT employees within respondent's span of influence (nine categories)
S1 Sum of the number of six specialties for which the site has at least one full time staff member (technology evaluation, quality assurance, data administration, methods & tools, metrics and measurement, system testing)
E1 The percentage of IT staff at the site holding bachelor's degree
E2 The percentage of IT staff at the site holding master's degree
Mean sd
2.39
.77
2.71
.70
20.6
26.2
16.9
25.9
16.5
24.6
2.27
1.20
2.77
1.26
$500-
999k
10 to
24
1.35
1.61
1.88
1.78
65.6
31.8
10.0
16.1
Measures of Organizational Innovativeness
The analysis below employs seven measures of innovation selected from the dataset: OOP Time,
OOP Infusion, OOP Assimilation, RDB Assimilation, CASE Assimilation, SPI Adoption, and SPI
- 22 -

Assimilation.
6
Table 5 below provides a summary of the operationalizations; Figure 1 shows how
the measures map along the two dimensions of aggregation.
6 A few other measures were also available in this dataset, however a preliminary analysis showed they yielded no further major insights, so they are excluded here to conserve space.
- 23 -

Variable
OOP
Infusion
Measure
Infusion
OOP Time Earliness of
Adoption
T ABLE 5: I NNOVATION M EASURES
Operationalization
The sum of three indicators: (1) number of supporting technologies used, (2) number of class library types used, and
(3) number of application components covered.
Years since OOPL acquisition (nonadopters coded to zero).
Distribution of Variables
Mean SD
Indicator 1: 0.83
1.01
Indicator 2: 1.53
1.19
Indicator 3: 1.87
0.83
Years # of Sites
1 141 sites
2
3
4
5
56 sites
19 sites
21 sites
4 sites
7 2 sites
Not acquired: 361 sites
(Mean= 0.75; SD=1.22)
Stage # of sites OOP
Assimilation
Assimilation Guttman scale with seven levels:
0-not aware, 1-aware, 2-interested,
3-evaluation/trial, 4-commitment,
5-limited deployment, 6-general deployment.
5
6 sites sites
28 sites
6 sites
(Mean= 2.04; SD=1.36)
Stage # of sites RDB
Assimilation
Assimilation Guttman scale with five levels:
0-no acquisition, 1-acquisition,
2-commitment, 3-limited deployment,
4-general deployment.
sites sites
CASE
Assimilation
Assimilation Guttman scale with five levels:
0-no acquisition, 1-acquisition,
2-commitment, 3-limited deployment,
4-general deployment.
SPI
Adoption
Aggregated
Adoption
(Mean= 2.15; SD=1.66)
Stage # of sites
Number of technologies acquired across the technology set including RDBs,
CASE and OOPLs.
sites sites
(Mean= 0.65; SD=1.14)
Number # of sites
SPI
Assimilation
Aggregated
Assimilation
The sum of standardized assimilation stage for RDBs, CASE and OOPLs.
2 168
3 93 sites
(Mean= 1.27; SD=0.99)
See individual stage measures above.
- 24 -

Extensive
SPI Adoption SPI Assimilation
Aggregation
Across
Innovations
None
OOP Infusion
OOP Time
OOP Assimilation
RDB Assimilation
CASE Assimilation
Lower
Aggregation Across Stages
Higher
F IGURE 1: I NNOVATION M EASURES
OOP Infusion was measured using a summative scale with three dimensions: number of supporting
OO technologies used, number of OO class library types used, number of application component types covered (see Table 6).
7 To create the measure, the indicators for each dimension were
standardized (to a mean of zero with unit variance) then summed.
I NDICATOR
1. Supporting technologies
2. Class libraries
3. Application components
T ABLE 6: I NDICATORS FOR OOP I NFUSION
O PERATIONALIZATION
Number of supporting object technologies used from the following set: (1) object databases, (2) object CASE tools, and (3) object methodologies.
Number of class library types employed from the following set: (1) GUI support, (2) basic data structures, (3) database access, and (4) industry specific.
Number of application components for which OOP is heavily employed from the following set: (1) user interface, (2) application logic/business rules, and
(3) database access/data management.
7 The rationale for using these three dimensions for OOP Infusion is that the recommended configuration of use of OOP, as articulated by OO researchers and proponents, is one in which application of the technology is as "pure" as possible.
This usually means using OOP in conjunction with OO supporting technologies (e.g., databases, design tools), employing a reuse-oriented process based on existing class libraries, and using OOP to develop complete applications, rather than just the user interface or business logic components (Fichman and Kemerer, 1993; Goldberg and Rubin,
1995). While infusion has usually been operationalized using Guttman scales, summative scales have also been used
(Howard and Rai, 1993).
- 25 -

The earliness of adoption variable, OOP Time, was measured as the number of years since first acquisition of an OOPL. An important methodological issue affecting measures such as earliness of adoption and infusion is whether to include or exclude non-adopters from the analysis. Studies of earliness of adoption have tended to include non-adopters in the analysis either by assigning them an arbitrary score (e.g., zero) or by employing statistical techniques  referred to as "survival analysis"  that incorporate information about non-adopters without imputing a particular time of adoption. (For example, see (Grover, et al., 1997; Pennings and Harianto, 1992; Russo, 1991)).
Studies of infusion, on the other hand, have tended to exclude non-adopters. Following these conventions, I include non-adopters for OOP Time by assigning them a score of zero, and exclude them for OOP Infusion. While use of survival analysis techniques would ordinarily be preferred for the analysis OOP Time, a sensitivity analysis using these techniques produced the same pattern of significant results, and very similar levels of significance, as when non-adopters were assigned a zero value and conventional statistics were used (results available from author). Therefore, to provide better comparability across models reported here, the analysis will not use the survival analysis technique. For the same reasons, I used OLS regression to estimate all models in spite of the fact that some of the variables are measured with Guttman scales for which the multi-level logistic regression would ordinarily be the preferred procedure.
OOP Assimilation, RDB Assimilation and CASE Assimilation (all measures that aggregate across stages) were all operationalized using Guttman scales, as has been the case in past studies employing assimilation stage (Fichman and Kemerer, 1997; Meyer and Goes, 1988). (See Tables 7 and 8). The OOP Stage measure has more levels than the other two because, as already mentioned,
OOP was the primary focus of the original data collection and more detailed questions were included for this technology. While it is possible to map the seven level scale for OOP to the same five level scale used for RDBs and CASE (by collapsing the first three stages), a preliminary analysis showed that this lead to a substantial loss of predictive validity for the reduced measure
- 26 -

(i.e., variance explained for the measure in preliminary regression analyses decreased by nearly a third). Because OOPLs diffused more recently, it is especially important to preserve the information contained in the full scale.
T ABLE 7: G UTTMAN S CALE FOR OOP A SSIMILATION S TAGE
Stage Criteria to Enter Stage
0. Not aware Key decision makers are not yet aware of the SPI
1. Aware Key decision makers are aware of the SPI
2. Interest The organization is committed to actively learning more about the SPI
3. Evaluation/ trial
The organization has acquired specific innovation-related products and has initiated evaluation or trial
4. Commitment The organization has committed to use a specific SPI product in a significant way for one or more projects
5. Limited deployment
6. General deployment
The organization has established a program of regular but limited use of the SPI product
The organization has reached a state where the SPI is used on a substantial fraction of new development, including at least one large and one mission critical system
Survey Items Used to Classify
Is informant familiar with OOPL concepts or products, or aware of prior OOPL-related activities at site?
Is informant familiar with OOPL concepts or products, or aware prior OOPL-related activities at site?
Is informant aware of plans to investigate any OOPL for possible production use within next 12 months?
Has the location acquired any particular OOPL?
Is the location evaluating or trialing any OOPL?
Are any specific production projects planned, in progress, implemented or canceled that use an OOPL as the primary language?
Have at least three production projects been initiated?
Has at least one production project been completed?
Have at least three production projects been completed?
Has site implemented at least one large OOPL project requiring at least a 12 person-month effort?
Has one or more core or mission critical applications been completed?
Has there ever been a year where at least 25% of new application development projects used an OOPL?
- 27 -

T ABLE 8: G UTTMAN S CALES FOR RDB (CASE) A SSIMILATION S TAGE
Stage Criteria to Enter Stage
0. No
Acquisition
The technology has not yet been acquired.
1. Acquisition The technology has been acquired.
2. Commitment The organization has committed to use a specific SPI product.
3. Limited deployment
4. General deployment
Has completed at least one production project using the technology
Has completed one or more large, mission critical applications and used the technology on at least 25% of new application development.
Survey Items Used to Classify RDB (CASE) Stage
Has the location ever installed an RDBMS (Upper
CASE Tool) for evaluation, trial or use?
Has the location ever installed an RDBMS (Upper
Case Tool) for evaluation, trial or use?
Has the location ever approved an RDBMS (Upper
CASE) for use to develop production applications?
Has the location ever implemented a multi-user application using an RDBMS (Upper CASE)?
Has the location ever implemented a large, mission critical application using an RDBMS (Upper CASE)?
Has the location ever used an RDBMS (Upper CASE) on at least 25% of all new development in the same year?
Two measures that aggregate across innovations are included in the study. SPI Adoption (which aggregates across innovations but not stages) was measured as a count of the number of SPIs that had been acquired by the organization. SPI Assimilation (which aggregates across innovations and across stages) was created by standardizing each SPI-specific stage variable (to mean of zero with unit variance) and then summing the three standardized variables. This is the same approach that has been used previously to create aggregated measures, except for the use of standardized variables.
While averaging raw scores can lead to composite variables that have more descriptive meaning, it leaves the composite vulnerable to being biased toward technologies that have larger variances.
To begin the analysis, consider the correlations in Table 9. As can seen from the values in the lower right triangle, all independent variables are positively correlated at p ≤ .01
, with the exception of some pairs involving OOP Related Knowledge. Since OOP Related Knowledge was operationalized with respect to a particular technology, it makes sense that it should exhibit weaker associations with more general predictor variables.
- 28 -

The upper left triangle shows that all seven measures of innovation are positively correlated with each other, and with the exception of some pairs involving OOP Infusion, all relationships are significant at p ≤ .01. Organizations possess varying propensities to assimilate SPIs in general; therefore, we would expect that these measures, each of which can be viewed as capturing a more specific instance of the general propensity, to be associated with each other. Also, as would be expected, the associations among the three OOP related variables are particularly high, as are the associations between SPI Assimilation and its constituent technologies.
T ABLE 9: C ORRELATIONS (N=608)
OOP
Infuse
(N=64)
OOP
Time
OOP
Assimilation
RDB
Assimilation
CASE
Assimilation
SPI
Assimilation
SPI
Adopt
LRS Diversity
.33** OOP Time
OOP Assimilation .52** .71**
RDB Assimilation .12
.16** .20**
IT
Size
Specialization
Education
CASE Assimilation .11
.13** .19** .29**
SPI Assimilation .32* .48** .66** .72** .70**
SPI Adoption .15
.52** .62** .66** .62** .91**
.21
.30** .36** .47** .47** .62** .58** Learning-Related
Scale
Diversity
IT Size
Specialization
Education
OOP Related
Knowledge
.02
.34** .34**
.33** .28** .36**
.32**
.39**
.27** .44** .46** .39**
.44** .57** .55** .62** .36**
.13
.26** .27** .25** .29** .39** .37** .37** .33** .34**
.15
.16** .19** .19** .17** .26** .24** .20** .17** .19** .11**
.34** .27** .33** .03
-.01
.17** .14** -.01 .12** .02 .12** .23**
** significant at p ≤ .01; * significant at p ≤ .05.
The values in the shaded area show a consistent pattern of positive associations between the independent variables and the different measures of innovation, as predicted by the theoretical model. With the exception of some pairs involving OOP Infusion, all expected relationships are significant at p ≤ .01 and in the right direction (positive). Also as expected, OOP Related Knowledge has no significant correlations with innovation variables that do not incorporate OOP in some way.
- 29 -

To further explore the effects of aggregation on predictive validity, the innovation measures were each regressed on the independent variables. The cells in Table 10 contain the standardized beta coefficients for each independent variable, and the associated t-statistics (in parentheses). Two different models (Models 3 and 6) were estimated for OOP Assimilation, one including OOP
Related Knowledge as a predictor and one excluding it.
T ABLE 10: R EGRESSION M ODELS (N=608)
(1) OOP
Infusion
(N=64)
(2)
OOP
Time
(3)
OOP
Assimilation
(4)
RDB
Assimilation
(5)
Case
Assimilation
(6)
OOP
Assimilation
(7)
SPI
Assimilation
(8)
SPI
Adopt
Learning
Related Scale
Diversity
IT Size
Specialization
Education
OOP Related
Knowledge
Adjusted R 2
F-statistic
.08
(0.5)
-.02
(-0.2)
.27
(1.7)
.00
(0.0)
-.02
(-0.2)
.34**
(2.6)
.14
2.7 *
.14**
(2.9)
.20**
(4.9)
.09
(1.8)
.08*
(2.0)
.02
(0.6)
.17**
(3.7)
.15**
(3.9)
.16**
(3.7)
.06
(1.5)
.02
(0.6)
.32**
(6.9)
.13**
(3.2)
.12*
(2.6)
.04
(1.0)
.08*
(2.2)
.28**
(6.0)
.04
(1.0)
.20**
(4.5)
.10**
(2.5)
.06
(1.7)
.13**
(2.7)
.18**
(4.5)
.16**
(3.4)
.09*
(2.3)
.09*
(2.3)
.35**
(9.1)
.17**
(5.2)
.23**
(6.1)
.11**
(3.4)
.11**
(3.6)
.23**
(6.2)
.30**
(8.3)
.22
.28
.26
.27
.20
.49
.45
28.7** 40.7** 43.4** 44.8** 31.4** 118.8** 101.8**
.28**
(7.1)
.22**
(6.4)
.25**
(6.4)
.09**
(2.8)
.09**
(2.9)
** significant at p ≤ .01; * significant at p ≤ .05.
The first three regressions provide a comparison of three OOP related models. These models show that OOP Assimilation (r 2 =.28) has a larger variance explained than OOP Time (r 2 =.22) and OOP
Infusion (r 2 =.14). It therefore appears that aggregation across stages has a positive effect on predictive validity in the case of OOP.
The next four regressions provide a comparison of SPI Assimilation and its constituent technologies, RDB Assimilation, CASE Assimilation and OOP Assimilation. For SPI Assimilation, the model explains almost half the variance (r 2 =.49) and all variables are significant at p ≤ .01. The
- 30 -

single innovation models, by comparison, explain only about a quarter of the variance on average
(mean r 2 =. 24), and have only two or three significant relationships at p ≤ .01. In fact, Education, while significant in the SPI Assimilation model at p ≤ .01, is not significant in any of the individual models at p ≤ .01. Therefore, it appears that aggregation across innovations has a positive effect on predictive validity in the case of SPIs.
The final model shows the results for SPI Adoption. This model (r 2 =.45) also has a much higher variance explained that any single innovation model, and all independent effects are significant.
This, again, suggests that aggregation across innovations is highly beneficial when the goal is to predict innovation with respect to SPIs. The model has slightly less variance explained than the one for SPI Assimilation, which adds further evidence that aggregation across stages has a slightly positive effect on predictive validity.
Potential Limitations
This study is subject to some potential limitations. First, the scale for OOP Infusion was not developed according to a highly rigorous procedure and may be viewed as somewhat ad hoc . It is possible that a more rigorously developed infusion measure would have performed somewhat better in the empirical tests, although as explained in the discussion section below, there are some compelling methodological explanations for the poor performance of OOP Infusion. And in any case, this possible limitation does not call into question the main results of this study.
Second, the innovation measures all pertained to the same fairly narrow technology class. So, what the empirical results mean for studies that aggregate across broader classes of innovations (e.g., process innovations in general, IT innovations in general) is less clear. For example, Swanson
(1994) has developed a theory for why different classes of IT innovations will have different predictor variables. This theory has received some good empirical support (Grover, et al., 1997). This suggests that aggregating across different classes of IT innovations may be problematic. On the
- 31 -

other hand, Damanpour (1991) found that innovation measures had strong predictive validity even when aggregating across a diverse set of innovations. So it remains a question for future study what the effects of aggregation would be across broader classes of IT innovations.
The doubling of variance explained in models aggregating across technologies demonstrates the magnitude of potential effects of this kind of aggregation under favorable circumstances. In fact, this study meets all of the criteria identified earlier for when aggregation is likely to be beneficial.
First, the theoretical model was developed to generalize to an innovation class (SPIs), and with the exception of OOP Related Knowledge, all predictor variables were operationalized at this level of abstraction. (In fact, it is not clear how predictors other than OOP Related Knowledge would be operationalized at the level of a particular technology). Second, the model generalizes across stages of assimilation. The rationales for the strongest independent variables relate to an organization's desire and ability to accommodate the burden of organizational learning surrounding new technologies, and since significant learning occurs in all stages of assimilation, it would therefore be expected that the effects of these variables should be positive in each stage. Third, all three SPIs are adopted by the same organizational unit (the IT group) and organizational characteristics were measured with respect to the IT unit. Therefore, within the confines of this study, organizational characteristics can be treated as primary, rather than secondary. Fourth, this study does not include secondary characteristics of innovations as control variables, and as argued earlier, aggregated measures should be more robust to such omissions. Fifth, it appears that the innovations in this study are moderate complements (as will be discussed in greater detail below). Aggregating mild complements or substitutes should have a stronger effect on predictive validity than aggregating strong complements, due to the large amount of shared variance among strong complements.
Finally, it may be expected that some degree of measurement error and idiosyncratic adoption are present in this study, and aggregated measures are more robust to these non-systematic effects.
- 32 -

Substitutes and Complements
The correlations among the stage measures for individual technologies (RDB Assimilation, CASE
Assimilation and OOP Assimilation) were only r=.23, on average (see Table 9). While higher than the negative correlation one would expect for strong substitutes, this mild degree of association is still smaller than would be expected for strong complements. Therefore, it appears the technologies are only moderate complements at most. The strongest correlation is between RDB and CASE
(r=.29), suggesting this is the strongest complementary pair.
These statistical results agree with our knowledge of how these technologies have actually been used in organizations. CASE tools had support for design and generation of RDB schemas as of the late 1980s. By the time OOPLs were diffusing in earnest, gateways had been developed to permit use of leading OOPLs with SQL-compliant RDBs, so there were some complementaries between these technologies also. The least obvious link is between OOP and conventional CASE tools. It is likely that organizations adopting both tended to use these technologies to develop different kinds of systems. So the small correlation between technologies may be spurious. As argued earlier, aggregating technologies within the same class that are moderate complements, other things being equal, should have comparatively greater effect on predictive validity than aggregating strong complements. In the extreme case of perfect complements, such as one might observe between
RDBs and data dictionaries, aggregation would produce the same distribution of innovation scores as either innovation taken individually. Therefore, it appears that the particular mix of SPIs included in this dataset was a contributing factor to the increase in predictive validity due to aggregation.
Aggregated Assimilation
The variable Aggregated Assimilation implicitly combines three different kinds of innovative behaviors into a single measure, i.e., the propensity to adopt many innovations, the propensity to adopt them earlier, and the propensity to implement them in a more rapid and sustained fashion. It
- 33 -

might be argued that a better approach would be to create three separate indicators, one for each propensity, and to create a multi-indicator construct for innovation that preserves information about these different behaviors. However, this approach introduces its own problems, because of some subtle interdependencies. Since binary adoption as of a given point of time is equivalent to a dichotomous measure of earliness of adoption (for any technology that is still diffusing), aggregated adoption also picks up the propensity to adopt innovations early. Likewise, since firms that adopt earlier have more time to reach later assimilation stages, aggregated implementation also picks up an earliness effect. This perhaps explains why the correlations among SPI Assimilation, SPI
Adoption and SPI Time (the average of years since adoption) were all at r=.85 or above.
Furthermore, an index created from an unweighted average of these three variables had a correlation of r=.96 with SPI Assimilation. Therefore, it appears that finding a way to truly unbundle these innovative behaviors while still aggregating across technologies represents a challenge going forward.
Post-Adoption-Only Measures of Innovation
Among the innovation measures, OOP Infusion has the weakest predictive validity by far. It appears that two statistical artifacts—low statistical power and range restriction
8 in study
variables  contributed to this result. These same potential problems can arise for any postadoption-only measure (i.e., any measure for which an organization must have previously committed to using the technology in order to receive an innovation score.) As a result, I consider these issues in some detail here.
8 Range restriction is present in an analysis when the values for a variable of interest are concentrated in a narrower range than one would observe in a more general population. For example, the SAT scores for students admitted to selective colleges would be range restricted.
- 34 -

Regarding statistical power, OOP Infusion is only measurable for the 10% of organizations (n=64) that have already begun to implement an OOP language. Hence the model has lower statistical power than the other models where the full sample was employed (n=608).
Regarding range restriction, the organizations for which infusion can be measured are not just any organizations, but rather, comprise the 10% of the responding sample (n=64) that have committed to using an OOPL. This indicates that this subset should be relatively more innovative with respect to software process innovations (and especially OOP) than the sample as a whole, and should therefore have a high concentration of cases with high values on predictor variables and a low concentration of cases with low values. To analyze the extent of range restriction, two groups were created: those that had committed to using an OOPL (n=64), and the full sample (n=608). A composite predictor variable was computed as the average of the standardized values of the four strongest predictors in the dataset (i.e., Learning Related Scale, Diversity, IT Size and OOP Related Knowledge.) Suppose we view scores on this composite predictor as being "high" when they fall in the top quartile for the full sample and "low" when they fall in the bottom quartile. By this standard, 60% of the cases in the OOPL user group (for which OOP Infusion can be measured) had a "high" value on the composite predictor while none had a "low" value. Given this degree of range restriction, it is perhaps surprising that OOP Infusion performed as well as it did!
Two other alternative explanations could account for some of the weakness of the OOP Infusion results. First, there may be weaker operationalization of that variable compared to other innovation measures. However, the strong correlation between OOP Infusion and OOP Assimilation (r=.52), a variable that was measured quite carefully and exhibited good predictive validity, casts doubt on measurement error as the major explanation in this case.
A second alternative explanation for the weakness of the model predicting infusion is the possibility that the predictors of innovation included in this study (other than OOP Related Knowledge) might
- 35 -

in fact have no effects even if restricted range were eliminated through a controlled experiment.
Since such control is of course not possible in studies of organizational innovation, identifying the determinants of infusion (or other post-adoption measures) may be inherently problematic except where researchers can wait for broad scale diffusion to occur.
In sum, it appears that post-adoption-only measures require special attention on the part of innovation researchers to manage (as best they can) the unique conceptual and methodological issues associated with such measures.
Implications for IT Innovation Researchers
This study has several implications for IT innovation researchers. First, it argues for increased attention to aggregated measures in the study of organizational innovation with IT. Aggregating across as few as three innovations lead to much stronger results in terms of predictive validity than for any single innovation model. There were many instances where independent variables were insignificant in one or more single innovation models even though they were highly significant in the aggregated models. Since theoretical models of innovation are often developed with the intention of generalizing to broader classes of technologies, it appears that aggregation within a class can substantially reduce the possibility of Type II errors for generalizations at these broader levels. This suggests that aggregation can be a very useful tactic for the innovation researcher.
Nevertheless there may be good reasons to prefer non-aggregated measures even when aggregated measures would otherwise be appropriate. Often the substantive interest in a sponsoring organization or the IT community at large is in a particular innovation and the factors affecting its adoption and assimilation. For example, had the substantive interest in this study been directed toward CASE, it would have been crucial to include the finding that, while Diversity is a strong predictor of SPI innovation in general, it is not a strong predictor of CASE. Furthermore, including analyses of particular innovations opens up the opportunity to use measures that are closely tailored
- 36 -

to that innovation. In this study, OOP Related Knowledge, a factor that typically requires an innovation-specific measure, was shown to be a very important predictor for all the OOP related measures. Also, focusing on a particular innovation allows researchers to devote more space to careful measurement of the adoption and implementation of that innovation. For innovations that are just emerging, it may be crucial to capture richer measures of innovation in order to detect expected relationships. For example, OOP Assimilation was especially carefully measured in this study due to these sorts of concerns, and as it turned out, the richer measure of this construct had
50% stronger predictive validity than the abbreviated measure. Finally, focusing on a single innovation allows the use of measures that have some attractive properties in the right circumstances. One noteworthy attribute of earliness of adoption, for example, is that it enables the use of longitudinal research designs that incorporate time-varying factors as predictors (Russo,
1991; Singer and Willett, 1991).
The poor relative performance of OOP Infusion should serve as a warning to researchers interested in post-adoption-only measures of innovation. While studies using such measures may be attractive from a theoretical standpoint, they are prone to methodological problems, i.e., lower statistical power and range restriction in study variables. In the present study it is difficult to ascertain whether the poor predictive validity is due solely to these methodological artifacts. It might be, for example, that OOP Infusion is simply not strongly affected by the predictors of innovation included in this study. However, in the absence of evidence countering methodological explanations for weak observed relationships, researchers should be guarded about concluding that predictors found to be insignificant actually have no effects. Furthermore, researchers should take care to gather data that would allow evaluation of this issue.
In sum, it would appear that prior to selecting an innovation measure, IT researchers should perform an analysis to determine whether the circumstances of the study favor the use of aggregated
- 37 -

measures. In addition, it may be advisable to capture and analyze multiple measures of organizational innovation where possible. When the primary interest is in a particular technology, a few questions could still be devoted to capturing an aggregate measure to use in exploring alternative hypotheses for any unexpected results relating to the focal innovation. Likewise, when the primary interest is in aggregate innovation, individual level measures (which, after all, must be captured to form the aggregate measure anyway) could be analyzed separately to provide insights into the consistency of influence of predictor variables across different technologies.
Many measures of organizational innovation have been employed by IT diffusion researchers. A key consideration is whether to aggregate behaviors across innovations and/or across the assimilation lifecycle within organizations. This study has presented a conceptual analysis of the circumstances favorable to aggregation in the measurement of innovation with IT. These circumstances include the following: (1) the researcher's interest is in general innovation or a model that generalizes to the level of some class of similar innovations, (2) independent variables of interest have effects that generalize across assimilation stages, (3) characteristics of organizations can be treated as constant across the innovations in the set, (4) characteristics of innovations can not be treated as constant across organizations in the study and are not otherwise controlled for, (5) the set of innovations being aggregated includes substitutes or moderate complements (rather than strong complements or unrelated innovations), and (6) sources of noise in the measurement of innovation, such as from reporting errors or idiosyncratic adoption, may be present.
The results of the empirical analysis were consistent with the generalizations of the conceptual analysis. Under circumstances that appear favorable to aggregation, it was found that aggregating across as few as three innovations produced more than a doubling of variance explained in models predicting organizational innovation with software process technologies. It was also found that aggregating across assimilation stages had a slight positive effect on predictive validity. Taken
- 38 -

together, these results provide initial confirmation of the potentially substantial benefits of aggregated measures of innovation.
- 39 -

REFERENCES
Aiken, M. and Hage, J. "The Organic Organization and Innovation," Sociology (5), 1971, pp. 63-82.
Armstrong, C.P. and Sambamurthy, V. "Information technology assimilation in firms: The influence of senior leadership and IT infrastructures," Information Systems Research (10:4), 1999, pp. 304-327.
Attewell, P. "Technology Diffusion and Organizational Learning: The Case of Business Computing,"
Organization Science (3:1), 1992, pp. 1-19.
Bretschneider, S. and Wittmer, D. "Organizational Adoption of Microcomputer Technology: The Role of
Sector," Information Systems Research (4:1), 1993, pp. 88-109.
Campbell, D.T. and Fiske, D.W. "Convergent and discriminant validation by the multitrait-multimethod matrix," Psychological Bulletin (56), 1959, pp. 81-105.
Chin, W.W. "Issues and opinion on structural equation modeling," MIS Quarterly (22:1), 1998, pp. VII-
XVI.
Cohen, W.M. and Levinthal, D.A. "Absorptive Capacity: A New Perspective on Learning and Innovation,"
Administrative Science Quarterly (35:1), 1990, pp. 128-152.
Cooper, R.B. and Zmud, R.W. "Information Technology Implementation Research: A Technological
Diffusion Approach," Management Science (36:2), 1990, pp. 123-139.
Daft, R.L. "A Dual-Core Model of Organizational Innovation," Academy of Management Journal (21:2),
1978, pp. 193-210.
Damanpour, F. "Organizational Innovation: A Meta-Analysis of Effects of Determinants and Moderators,"
Academy of Management Journal (34:3), 1991, pp. 555-590.
Dillman, D. Mail and Telephone Surveys , John Wiley & Sons, New York, 1983.
Downs, G.W. and Mohr, L.B. "Conceptual Issues in the Study of Innovation," Administrative Science
Quarterly (21:December), 1976, pp. 700-714.
Fichman, R.G. "The Diffusion and Assimilation of Information Technology Innovations," In Framing the
Domains of IT Management: Projecting the Future Through the Past , R. B. Zmud (Ed.), Pinnaflex
Educational Resources, Inc., Cincinnati, OH, 2000.
Fichman, R.G. and Kemerer, C.F. "Adoption of Software Engineering Process Innovations: The Case of
Object Orientation," Sloan Management Review (34:2), 1993, pp. 7-22.
Fichman, R.G. and Kemerer, C.F. "The Assimilation of Software Process Innovations: An Organizational
Learning Perspective," Management Science (43:10), 1997, pp. 1345-1363.
Fichman, R.G. and Kemerer, C.F. "The Illusory Diffusion of Innovation: An Examination of Assimilation
Gaps," Information Systems Research (10:3), 1999, pp. 255-275.
Fornell, C. and Bookstein, F.L. "Two Structural Equation Models: LISREL and PLS Applied to Consumer
Exit-Voice Theory," Journal of Marketing Research (XIX:November), 1982, pp. 440-452.
Gatignon, H. and Robertson, T.S. "Technology Diffusion: An Empirical Test of Competitive Effects,"
Journal of Marketing (53:1), 1989, pp. 35-49.
Goldberg, A. and Rubin, K.S. Succeeding With Objects , Addison Wesley, Reading Mass., 1995.
Grover, V., Fiedler, K. and Teng, J. "Empirical evidence on Swanson's tri-core model of information systems innovation," Information Systems Research (8:3), 1997, pp. 273-287.
- 40 -

Grover, V. and Goslar, M.D. "The Initiation, Adoption and Implementation of Telecommunications
Technologies in U.S. Organizations," Journal of Management Information Systems (10:1), 1993, pp. 141-
163.
Hamel, G. "The challenge today: Changing the rules of the game," Business Strategy Review (9:2), 1998, pp.
19-26.
Hammer, M. "Reengineering Work: Don't Automate, Obliterate," Harvard Business Review (66:4, July-
August 1990), 1990, pp. 104-112.
Hart, P.J. and Saunders, C.S. "Emerging Electronic Partnerships: Antecedents and Dimensions of EDI Use from the Supplier's Perspective," Journal of Management Information Systems (14:4), 1998, pp. 87-111.
Howard, G.S. and Rai, A. "Promise and Problems: CASE Usage in the US," Journal of Information
Technology (8:2), 1993, pp. 65-73.
Howell, J.M. and Higgens, C.A. "Champions of Technological Innovation," Administrative Science
Quarterly (35:2), 1990, pp. 317-341.
Johansson, J.K. and Yip, G.S. "Exploiting Globalization Potential: U.S. and Japanese Strategies," Strategic
Management Journal (15:8), 1994, pp. 579-601.
Kessler, E.H. and Chakrabarti, A.K. "Innovation Speed: A Conceptual Model of Context, Antecedents and
Outcomes," The Academy of Management Review (21:4), 1996, pp. 1143-1191.
Kimberley, J.R. and Evanisko, M.J. "Organizational Innovation: The Influence of Individual,
Organizational, and Contextual Factors on Hospital Adoption of Technical and Administrative Innovations,"
Academy of Management Journal (24:4), 1981, pp. 689-713.
Lind, M.R. and Zmud, R.W. "The influence of a convergence in understanding between technology providers and users on information technology innovativeness," Organization Science (2:2), 1991, pp. 195-
217.
Markus, L.M. "Power, Politics and MIS Implementation," Communications of the ACM (30:6), 1983, pp.
430-444.
Massetti, B. and Zmud, R.W. "Measuring the extent of EDI usage in complex organizations: Strategies and illustrative examples," MIS Quarterly (20:3), 1996, pp. 331-345.
Meyer, A.D. and Goes, J.B. "Organizational Assimilation of Innovations: A Multilevel Contextual
Analysis," Academy of Management Journal (31:4), 1988, pp. 897-923.
Miller, D. and Friesen, P.H. "Innovation in Conservative and Entrepreneurial Firms: Two Models of
Strategic Momentum," Strategic Management Journal (3:1), 1982, pp. 233-250.
Nilakanta, S. and Scamell, R.W. "The Effect of Information Sources and Communication Channels on the
Diffusion of an Innovation in a Data Base Environment," Management Science (36:1), 1990, pp. 24-40.
Orlikowski, W.J. "Improvising Organizational Transformation Over Time: A Situated Change
Perspective," Organization Science (7:1), 1996, pp. 63-92.
Pennings, J.M. and Harianto, F. "Technological Networking and Innovation Implementation," Organization
Science (3:3), 1992, pp. 356-382.
Pierce, J.L. and Delbecq, A.L. "Organizational Structure, Individual Attitudes, and Innovation," Academy of
Management Review (2), 1977, pp. 23--37.
Prescott, M.B. and Conger, S.A. "Information Technology Innovations: A Classification by IT Locus of
Impact and Research Approach," Database (26:2&3), 1995, pp. 20-41.
- 41 -

Rai, A. "External Information Source and Channel Effectiveness and the Diffusion of CASE Innovations:
An Empirical Study," European Journal of Information Systems (4:2), 1995, pp. 93-102.
Rai, A. and Howard, G.S. "Propagating CASE Usage for Software Development: An Empirical Investigation of Key Organizational Correlates," Omega (22:2), 1994, pp. 133-147.
Ramiller, N.C. "Perceived Compatibility of Information Technology Innovations Among Secondary
Adopters: Toward a Reassessment," Journal of Engineering Technology Management (11:1), 1994, pp. 1-
23.
Ramiller, N.C. "Airline magazine syndrome: Reading a myth of mis-management," Information Technology
& People ), forthcoming,
Rice, R. and Rogers, E. "Reinvention in the Innovation Process," Knowledge: Creation, Diffusion,
Utilization (1:4), 1980, pp. 499-514.
Rogers, E.M. Diffusion of Innovations , The Free Press, New York, 1983.
Russo, M.V. "The Multidivisional Structure as an Enabling Device: A Longitudinal Study of Discretionary
Cash as a Strategic Resource," Academy of Management Journal (34:3), 1991, pp. 718-733.
Saga, V.L. and Zmud, R.W. "The Nature and Determinants of IT Acceptance, Routinization, and Infusion,"
Proceedings of the International Federation for Information Processing Conference on Diffusion, Transfer
& Implementation of Information Technology , Champion, Pennsylvania, 1993, pp. 67-86.
Saltzman, A. "Improving Response Rates in Disk-by-Mail Surveys," Marketing Research (5:3), 1993, pp.
32-39.
Singer, J.D. and Willett, J.B. "Modeling the Days of Our Lives: Using Survival Analysis When Designing and Analyzing Longitudinal Studies of Duration and Timing of Events," Psychological Bulletin (110:2),
1991, pp. 268-290.
Sullivan, C.H. "Systems Planning in the Information Age," Sloan Management Review (26:2), 1985, pp. 3-
11.
Swanson, E.B. "Information Systems Innovations Among Organizations," Management Science (40:9),
1994, pp. 1069-92.
Swanson, E.B. and Ramiller, N.C. "The organizing vision in information systems innovation,"
Organization Science (8:5), 1997, pp. 458-474.
Tornatzky, L.G. and Fleischer, M. "The Processes of Technological Innovation," Lexington, 1990.
Tornatzky, L.G. and Klein, K., J. "Innovation Characteristics and Innovation Adoption-Implementation: A
Meta-Analysis of Findings," IEEE Transactions on Engineering Management (EM-29:1), 1982, pp. 28-45.
Yin, R.K. Changing Urban Bureaucracies , The Rand Corporation, 1979.
Zaltman, G., Duncan, R. and Nolbeck, J. Innovations and Organizations , John Wiley & Sons, New York,
1973.
Zmud, R.W. "Diffusion of Modern Software Practices: Influence of Centralization and Formalization,"
Management Science (28:12), 1982, pp. 1421-1431.
Zmud, R.W. "An Examination of 'Push-Pull' Theory Applied to Process Innovation in Knowledge Work,"
Management Science (30:6), 1984, pp. 727-738.
Zmud, R.W. and Apple, L.E. "Measuring Technology Incorporation/Infusion," Journal of Product
Innovation Management (9:2), 1992, pp. 148-155.
- 42 -

Appendix A: Summary of Innovation Measures
Earliness of Adoption : Relative earliness of adoption in a population of potential adopters.
Internal Diffusion: The extent of use of an innovation across people, projects, tasks, or organizational units.
Measure/Definition
Infusion: The extent to which an innovation's features are used in a complete and sophisticated way.
Routinization : The extent to which an innovation has become a stable and regular part of organizational procedures and behavior.
Assimilation : The extent to which an organization has progressed through the assimilation lifecycle stretching from initial awareness to full institutionalization.
Aggregated Initiation/
Adoption/Implementation: The frequency or incidence of innovation initiation/ adoption/implementation.
Example Operationalizations
Five-item categorical scale (Rogers, 1983)
Binary adoption of laptop computers (Gatignon and Robertson, 1989)
Time since adoption of IT innovations (Grover, et al., 1997)
Five-level scale for use of software development practices (Zmud, 1982)
Number of PCs per employee (Bretschneider and Wittmer, 1993)
Percentage of supermarkets using scanners (Zmud and Apple, 1992).
Six-level scale for percentage of staff and projects using CASE tools (Rai and
Howard, 1994).
Percentage of all documents for which EDI is used; number of different transaction sets for which EDI is used (Hart and Saunders, 1998).
Three-level Guttman scale for scanner infusion (Zmud and Apple, 1992).
Four-level Guttman Scale for MRP infusion (Cooper and Zmud, 1990).
Thirteen-item summative scale for CASE features (Rai and Howard, 1994).
Ten-item summative scale for routinization of local government innovations
(Yin, 1979)
Four-item summative scale for routinization of supermarket scanners (Zmud and
Apple, 1992).
Ten-level Guttman-scale for assimilation stage achieved for healthcare innovations (Meyer and Goes, 1988).
Seven-item Guttman-scale for assimilation stage achieved for a software process innovation (Fichman and Kemerer, 1997).
Ten-level summative scale for reported relative success in applying IT to support strategy, marketing, and logistics (Armstrong and Sambamurthy, 1999).
Number of initiations/adoptions/implementations from closed set (Grover and
Goslar, 1993; Nilakanta and Scamell, 1990; Zmud, 1982).
Number of adoptions from an open set (Miller and Friesen, 1982).
Three of these measures—diffusion, infusion and assimilation—were introduced comparatively recently, and therefore warrant some comment beyond the summary provided above. Internal diffusion and infusion may be viewed as covering the "breadth of use" and "depth of use" dimensions first suggested by Tornatzky and Klein (1982). Internal diffusion addresses the spread of use of an innovation within an organization, but says little about how it used. Infusion, by contrast, captures the character of use. The definition used here is consistent with Cooper and Zmud (1990) and Zmud and Apple (1992). It is also consistent with a construct labeled "injection depth" by Howard and Rai (1993). However, there are other definitions of the term.
Sullivan (1985) defined infusion to include not only the degree of sophistication of use, but also the impact of use on organizational performance. Saga and Zmud (1993) argued that infusion has three subdimensions, which they labeled extended use, integrative use and emergent use. The first dimension is similar to the definition employed here. The third dimension overlaps with the "reinvention" concept (Rice and Rogers,
1980).
Assimilation is distinguished from other implementation measures in that it captures gradations of innovation among organizations that have yet to adopt an innovation. However the measure gives no indication of the path an organization followed to reach a particular level of assimilation. One organization may start assimilation early but progress slowly and end up with the same score as one that started late, but progressed quickly. When it is important to preserve earliness of initiation and speed of assimilation as separate behaviors, assimilation blurs these distinctions. Also, the measure may be less useful when captured for a popular innovation that has already diffused, because organizations would be concentrated in the latter stages of assimilation.
- 43 -