AE Report for ISR 2005-096 “Theorizing about the design of Information Infrastructures: design kernel theories and principles The goals of the paper are ambitious and worthwhile. Information infrastructures, when successful, hold out considerable benefits to individuals, businesses and society. Yet, they pose special challenges owing to a fundamental chicken-and-egg problem: they gain most of their value through network externalities, yet we need some way to get users to adopt them even when the network is small and network externalities have yet to become a significant factor in the value proposition. Also, IIs ultimately become quite complex, yet some way must be found to spare initial users and the designers themselves from this complexity, especially in the early stages when the network has a weaker value proposition and the fate of the II is most uncertain. Finally, successful IIs are inherently evolutionary because it is impossible to foresee the myriad of uses that will eventually be discovered for it, and so we must find some way to harness, rather than inhibit, evolutionary forces. I think an excellent case can be made, and to a good extent has been made, that IIs therefore pose a fundamentally different architectural challenge than traditional IT applications. With traditional applications the focus of design guidelines is primarily on promoting technical efficiency of one form or another. With traditional applications we want design specifications to be “complete” because it is assumed to be much more expensive to add features later. A high priority is given to reliability, security etc., right from the very start, because it is assumed the system will actually be adopted. We focus on making the system as useful as possible for each intended user in isolation, under the implicit assumption that the value of the system comes inherently from the functionality per se, rather that from the extent to which functionality has been adopted across the network. To sum up, I agree with the review panel that the goals of the research are very worthwhile, and that your design guidelines are insightful and constitute a good start on meeting the goals you set out. We all believe that this research has the potential to make a very nice contribution to the IS field. However, I also agree with the reviewers that this paper is very far from the point where it needs to be to realize this promise. Two reviewers, while noting the difficulty of the path ahead, have invited a revision. One, one while strongly encouraging the concept of the research, feels the paper is too far from a publishable state to invite a revision, and so has recommended rejection. My opinion is that the promise in the manuscript is such that it would be in the interest of the authors and the field to invite a round of revision at ISR, to see if it is possible to sufficiently address the concerns of the reviewers within the format constraints of an ISR paper. I think that you will be in the best position to judge whether the necessary revisions are possible, and so I would like to extend this opportunity to you. The reviewers have raised a number of major concerns, which I will highlight shortly. I have a fair degree of confidence that with a conscientious effort from you, most of the concerns related to content and positioning of the research can be addressed. However, there is one over arching issue that I’m not sure can be addressed, and that is conforming to the length restrictions of an ISR paper. While I think there is much potential to streamline the material that already exists, each of the reviewers has identified major areas where more content must be added. Therefore, I think you may need to find some way to limit your scope of attention if you are to develop a tight and complete case for your proposed theory of II design. It is hard for me to tell how feasible this is. In my remaining comments, I will give my own interpretation of the most important issues from the reviewers and add some thoughts of my own. In addition, I encourage to consider each of the individual reports most carefully, and to respond point-by-point to each reviewer in your response to the reviewers should you attempt a revision at ISR. You are fortunate to have the benefit of insights from a top quality review panel. 1. Concerns about Overall Paper Format and Style The paper is very difficult to master. Part of this difficultly lies in the inherent complexity and scope of the task you have set before yourself, and part is due to how you have actually structured and presented your ideas. I agree with Reviewer 1’s contention that any paper that, like this one, requires multiple close readings by domain experts to comprehend has little potential for impact among the broader audience of ISR readers. You need to find some way to convey your thoughts throughout the paper more simply, and with a much smoother logical flow. Reviewer 2 suggests the paper is too “flat”, with too many major sections. I would agree that 9 major sections a lot, but since you already have three levels of hierarchy in the paper I’m not sure what the solution is, other than finding some way to narrow your scope. Speaking of which, the paper has to be much shorter. This was a comment made by all three reviewers on the grounds of digestibility alone—there is a limit to how much effort a reader can be expected to spend to absorb the content of one paper. But just as important, the ISR guidelines specify that a paper should be no more than 32 pages of text, and 38 pages overall. This paper is nearly 47 pages long overall, of which I’d guess about 40 pages are text. I think that some progress can be made just through general streamlining, however I do not think that streamlining alone can solve the length problem, considering that all three reviewers have indicated areas where expansion may be needed. Therefore, I think you will need some way to narrow your scope (see Item 4 below) . 2. Paper Organization I like Reviewer 2’s suggestion that you put greater emphasis up front on IIs before you get to design theories, and that you organize your review around the three levels of analysis (corporate, business sector, universal service). I suspect you have put the emphasis on design theories up front because design theory is not a mainstream research approach, and you might have felt the need to legitimize your approach before getting too far into specifics. However, I agree with the reviewer that it would be better to talk about what IIs are and why we should care about their design before getting into the details of design theories. I also think it would be most helpful if you were to provide a list of prominent technologies that are (and are not) IIs, which kind they are, and why. Some quick candidates that come to mind would be: the internet, EDI networks, France’s Minitel, Japan’s DoCoMo, ISDN (in the US or elsewhere), 3G wireless, multi-application smartcards, RFID, and various enterprise technologies (ERP, CRM, SCM, PLM, groupware, knowledge management). Reviewer 2 also suggests that you start with the case studies and end with the design principles. I don’t have a clear intuition about whether this would be an improvement or not, so I’ll leave you to consider the possibility as you are making the other revisions. While we’re on the subject of the case examples, you state on page 8 “Finally, we used two case studies to derive testable hypotheses from this theory and validate them for internal and external validity”. I’m not sure I understand this sentence. How you can use the very same case studies to both derive, and then to validate ideas? Is it even true that you derived your ideas specifically from these two cases? Or did you perhaps draw them from a larger set of cases, the design literatures, etc.? Also, you don’t actually offer any testable hypotheses. 3. Terminology, Definitions and Concepts Related to IIs All the reviewers had concerns about your definitions of IIs. Reviewer 1 notes that you have defined IIs somewhat differently at different points in the manuscript. Reviewer’s 1 and 2 both note that you have incorporated into your definition of IIs elements that might more appropriately be viewed as characteristics of a good II design (e.g., standardization) rather than as a essential characteristic of all II. Reviewer 3 finds II to be an inadequate term to “reflect the content of such large and complex networks of systems. A focus on information shortchanges the richness of services, workflows, qualities (reliability, security, performance, etc.), and other capabilities provided by these infrastructures.” Reviewer 1 also wonders if II is even the best term. The reviewer suggests “platform” or “application platform” as a potential alternatives. Whether or not that’s a good idea depends on what counts and does not count as an II and how you draw the bounds on the scope of your theorizing. For example, I would call the Intel x86 architecture, MS Windows and the Oracle database all “platforms”, but I don’t think you would call them IIs, right? Possibly something like “Shared Information Services Platform” would work. This is broader than “application platform”, in that I think it would encompass the Internet. I’ll leave the final terminology decisions to you, as it ties in to other decisions regarding scope and which examples you will highlight. Reviewer 1 had made many good points about the need to clarify the sections that review theories of design and that develop your own theory of design — see the comments provided by this reviewer under the heading “2. Conceptualization and Construction of Design Theory – and Kernel Theories”. My own observation is that your Section 4.1 is very descriptive. You are not as clear as you could be in identifying the essential and distinctive features of IIs. Also, you gave no foreshadowing of the mapping of these essential, distinctive features of IIs to the selection of your kernel theories. I think these two topics have to be very tightly linked. Also, I’d like to see you link your specific design guidelines back to your kernel theories more explicitly to the maximum extent possible. It is your kernel theories that provide the boundaries on what “matters”. It’s very important for all of your guidelines to stay within the scope of your kernel theories, because otherwise this opens the door to any number of additional potentially useful guidelines, and your scope would become unmanageable. Finally, there are some other concepts that are not defined as clearly as they could be include “technology traps” (pg 19), “lock-ins (pg 25)” and “usefulness” (pg 21). Regarding the latter term, I think you might better emphasize the distinction between “inherent” usefulness or superiority that does not depend on there being other adopters, versus aspects that do depend on other adopters. 4. Incomplete Attention to Related Literatures on Architectural Design Reviewer 3 is concerned that the design guidelines, while insightful, are not sufficiently formal and complete to represent a true theory of II design. The reviewer feels that you have not drawn sufficiently on existing work on IT architectures—several books on the subject are noted. The reviewer highlights two missing elements in particular: lack of attention to security features (required to create trust) and lack of attention to how user transactions are defined and executed across a network of varied services. Since the paper is already well over the page limit, this represents a quandary. The only solution I can see is you will need to draw boundaries on your theorizing in some way that avoids the need to significantly expand your scope. This could be difficult. As the reviewer notes: “[T] the authors’ view through a lens of economic theory provides only a partial picture of a comprehensive II design theory. I do not find it surprising that the proposed design guidelines map well into the two examples of the Internet as an II success and Norway’s health care system as an II failure. However, this is not a convincing demonstration of either the completeness or accuracy of the design theory. I conjecture that by the application of other theoretical lens, such as the architectural theories mentioned above, a different and complementary set of design principles and design guidelines can be developed and evaluated on the same examples with similar success.” In a similar vein, Reviewer 2 notes that you’ve not attended to recent related work that has been done on “infrastructure flexibility” and “systems integration” and a few other topics. The concern here is that your chosen theoretical perspective privileges some aspects of design and ignores others that, for all we know, might be equally important, and might fit equally well with your two examples. Yet, I’m not sure how you will have space available to properly incorporate additional theoretical perspectives. If you can’t fit additional perspectives in, you need to be able to develop an argument for why it is appropriate to exclude them from your scope. One way out of this dilemma might be to somehow narrow your attention to a particular class of IIS that have certain properties that increase the salience of the particular theoretical perspectives you bring to bear (see Item 5 below) but not others. I’m not sure if this can be done, but it’s worth thinking about. 5. Levels of Analysis Reviewer 2 is concerned that you have not adequately distinguished the three levels of analysis (corporate, business sector, universal service) in your theorizing. For example, the corporate level is much more open to centralized control than the other two. Relatedly, governance issues are much more important for corporate infrastructures. The reviewer is concerned about whether your design principles operate equally at all levels, and particularly whether all the principles that apply at lower levels necessarily apply at higher levels. One option to consider would be to confine your theorizing to just the top, or the top two levels of analysis. This might allow you to more easily narrow your theoretical scope by attending to what is most important at the universal service or business sector level. The most extreme change would be to focus just on the top level, universal service. To do this you would have to pick a different negative case example since your current example is at the business sector level. Two possibilities for negative examples that come to mind are ISDN in the United States and multi-application smart cards in the US. In the example if ISDN in particular I think it can easily be argued that ISDN designers had no sensitivity whatsoever to the role of network externalities in standards adoption, and how this might affect design decisions. For example, there was no reason that ISDN had to “step on” the specific frequencies reserved for analog voice, but this design decision made existing phones obsolete. So this violated the principle of reusing existing infrastructure to extend possible. If you were to focus on just the top level, or just the top two levels, you could still talk about the other levels, and the possibility of generalizing the design guidelines and principles to those levels, in a section on future work. 6. Lack of Attention to the Designer Reviewer 2 has a major concern that you have no real conceptualization of the designer of information infrastructures. No individual or organizational designers are named, but you nevertheless make many claims about their intentions. The reviewer believes that quotations from designers could make your argument more persuasive. The reviewer goes on to note that this highlights a central paradox of IIs: while there are certainly designers involved, the infrastructures evolve without central management, raising a question of whether they are actually designed. This comment is increasingly weighty as you move up the levels of analysis to the universal service level. One possible way to address this paradox is to highlight the difference between design guidelines and the actual design of specific features. The internet was never “designed” specifically to do most of the things it actually does now, but, as you show, certain design principles and guidelines were implicitly followed in the overall architecture and also the subsequent design of individual components. Take a web browser. Early browsers could just as easily been designed as closed programs, but they weren’t. From the start they were designed to allow third party “plug ins”. That was a key architectural choice that was made that is consistent with your design principles and guidelines, and also your kernel theories. Allowing third-party plug-ins allows browsers to harness network externalities and to promote evolutionary growth in ways that would be impossible if one company were to control how browsers worked. 7. Other Issues Reviewer 3 raises some addition points that must be attended to, especially (1) the need to expand out the rationales for concluding that the Norway medical systems did not conform to certain design guidelines, (2) the need to better justify the contention that conformance to the design guidelines would have lead to success given the organizational and political challenges, (3) the need to justify the statement that adoption of your proposed guidelines would lead to major changes in what constitutes best practices in the design of IIs. Finally, there are a number of stylistic problems to address. You don’t use consistent labeling in the tables and in the text for your guidelines. Many citations are missing from the bibliography. There is a larger than average number of typographical errors. There were many individual sentences that I had difficulty making sense of. This all suggests the need for more attention to proofing and polishing before resubmission, and possibly the services of a professional editor. Closing As described above, much work must be done to get this paper to the point where it would be suitable for publication in ISR, and the outcome is uncertain. However, regardless of how you proceed from here, I want to reemphasize the great potential the reviewers and I see in what you are attempting to accomplish. I hope that you will persist in this endeavor! Reviewer #1 Comments on ISR #2005-096 for the Authors This is a very interesting paper in terms of its direction and potential. The paper has strengths is that it addresses a critical topic, is creatively gutsy, draws on previous work on IS design theory and builds on it, as well tries to use theoretical treatises from various disciplines to help the arguments and construction. This is a deep but complex paper in a difficult area that has much promise of having an impact on the IS community if executed well and revised. I hope that the authors will rise to the challenge. It requires both substantial rethinking and rewriting. It is a difficult paper to write – and I might add to review. It currently suffers from weaknesses that will limit both its impact and understandability. Currently it is too long and complex, and the flow of arguments and sequence is not conducive to assimilation, and in addition there are a number of confusing ambiguities that require much rethinking. If cleared this will be an excellent contribution. If not cleared, it will not have any impact. I had to read this paper repeatedly and I have thought of these issues before. Unless I was having a bad day each time I looked at it, it would seem to me that the average IS academic reader would have trouble digesting what to do with this and how to use it to move forward. I have tried to help the authors resolve some of the problems. There are issues of definition and scope, issues of conceptualization and construction, issues of clarity/exposition and flow sequence and argument. It is not ready yet and requires substantial rework. 1. Rethinking & redefining an information infrastructure (or an application platform or ?) I struggled with the names and definitions and had to take a step back. Here is how I view what we know and what we don’t know in the IS academic community around this: - We all know what an IS is and that there are classes and instances of ISs. - We all know what IT is and that it a heterogeneous concept rather than a monolith, and that IS is IT in context and contextual variables around it depending on whether you subscribe to the “IT artifact” or “organizational systems” view of the world. - We know that IT infrastructure is a blurry term that denotes hardware and software that an IS application sits on top of. - We are not quite sure what an information infrastructure is and whether it includes hardware and operating systems software, or is it like a database management program, or it what glues various IS applications together like the types of intelligent “data plumbing” architecture which are being developed in the wireless world to move data seamlessly across devices as user moves. For example, the new “iobi” platform that Verizon Wireless has deployed for moving multimedia across different mobile devices , operating systems, locations, and screen sizes. A diagram would really help to distinguish between, IS, IT, IT infrastructure, Information Infrastructure,… I would also consider the term “Platform” as another possible way to think about it in terms of something that various classes of IS can sit on top of. Platforms give the impression that they are closer to applications, and that they can still sit on top of IT infrastructure. Intel Corporation has recently adopted that term as they redefine their strategic vision and move beyond IT infrastructure. As we move towards service-oriented architectures and web services, the notion of a huge monolith application such as ERP will go away, and there will be tiny callable components that sit on top of some kind of ERP platform or mobile multimedia communication platform or … Now let me go through how the paper came through on that. I had a problem with the definition of information infrastructures (II) throughout the paper. You start by calling them heterogeneous socio-technical systems on page 2 where you mention their defining attributes (large, complex, adaptive, no one designer,…). The paper then tries to link IIs to related work on page 6 (improvisation, colonial systems, infrastructure as portfolio,…). Then on page 9 there is a formal attempt to define an II and there it becomes an installed base of IT capabilities with a defined user community and open interfaces. It then attempts to elaborate on that with a somewhat complex write-up that is descriptive rather than theoretical. Then on page 12, it tries to simplify the complexity by making it vertical and horizontal, and I am not sure that slicing works very well and that it can be cut up that way. I had a lot of problems with the classes in Table 3 and the distinguishing characteristics and seemed like a force fit. I am not sure how the verticality works in Table 3. I can see the internet or Internet 2 or serviceoriented architecture Internet or whatever else we will get going forward into the future --- as a universal supra-IT infrastructure (as convoluted as that sounds) which houses many platforms, however the other two are much more tenuous. Is verticality based on the type of information exchange standard (EDI) or the type of information integration across a user community (ERP)?? Seems like user community domain requirements, functional requirements, and technical requirements are being blurred here in terms of verticality. I think this requires much more thought. Perhaps thinking of the internet as a universal supra-IT infrastructure over which many IS platforms can be built and used might be another way to go. And IS platforms can then be defined around user communities (or industries and markets nor business sectors) such as health or financial services or manufacturing,… ) or functionalities (transactions, relationships, community). In any case, Table 3 is problematic and difficult for others to build on theoretically. Then on page 14 the paper starts a discussion of horizontal infrastructures, and while I am not sure I agree that this is “horizontal” and looks more vertical, I really like the flavor of Figure 1 and think that the paper should start from something like that to give the reader some grounding and for forcing the better articulation and definition. The internet is clearly in the domain of support infrastructures (or supra-IT infrastructures if you believe that argument) and the transport and service parts are clear. Is the operating systems software part of that or is there a middleware layer ? Then is it the applications infrastructure that sits on top of that or should that be divided into two layers: an application platform ( for a class of applications based on communities or types of information exchange…) and then applications (types of user functionalities or types of interactions). Types of interactions can be transactions (ERP), relationships (CRM), planning and rescheduling (SCM), monitoring and scanning (business intelligence). Thus the design theory would focus on “application platforms” or “xx platforms” which would still capture the evolution, heterogeneity, have both technical and functional requirements, …….. rather than on designing the internet. Seems like an application-oriented view of IT infrastructure (application platform) would be closer to the concerns of the IS community rather than computer scientist. So, I would see the Norwegian healthcare example being closer to that, but the evolution of the internet as not being a good example or illustration. In short, you may want to rethink your layers more carefully, take the business/user-oriented perspective into account more directly, and address a thinner more focused sliver of the figure 1 structure. Distinguishing between infrastructures and platforms might be a way to finesse that for the IS research community so that they can build on this paper and help advance design theory as well. Thus, like the Walls et. al 1992 paper focused on design theory for a class of information systems, this could focus on a class of application platforms. I think it is more doable that way and will have more impact. And your kernel theories still work. I really the beginning of the paper needs to lay this out very clearly (and distinguish between all these terms/concepts), so readers know what it is that we really trying to find a design theory for, and we understand where IS researchers can make an impact, and where computer scientists can. It is not an easy definition – as in a sense you are redefining our view of architecture and infrastructure, and what an application is,… but it is crucial. Unless that is done clearly and convincingly at start (and diagrams help immensely), the rest will have little impact. 2. Conceptualization and Construction of Design Theory – and Kernel Theories The review of IS design theories is reasonable although not as streamlined as it could be. The authors point out an interesting distinction between classes of information systems (authors call this vertical demarcation) and classes of design problems (authors call this horizontal demarcation). In the Walls et. al 1992 paper and rendition of an IS design theory, an IS design theory takes into account both design product and the design process. It includes both a design product set of components (meta-requirements, meta-design, kernel theories, testable design product hypotheses) coupled with a design process set of components (kernel theories, design methods, and testable design process hypotheses). Thus, what the authors call classes of design problems are derived in Walls et al. from a kernel theory and are coupled to meta-requirements for a class of information systems, rather than decoupled rules of thumb which is what design principles are if they are not based on kernel theories nor linked to meta-requirements. Thus, are there universal design principles that cut across different classes of information systems as the authors claim or are these then rules of thumb? Thus after defining what an II (or platform is) and its attributes that are different in nature than those for an IS -- there is a need to articulate the structure of IS design theory in a convincing way: is it different from the Walls et al structure ? and what is the appropriate structure and why ? I do not think the paper does that well enough. The paper identifies some attributes for IIs: - Large - Complex - Adapt to future changes in technical requirements - Adapt to future changes in functional requirements (by the way the distinction between technical requirements and functional requirements may become more blurred with new architectures like web services) - Designed as extensions or improvement to existing IIs - Diverse components not under the control of one designer (almost like open source) Then pages 17-26 tries to take on the difficult task of deriving kernel theories that can suggest what the design process might be. As one looks at those characteristics together a new dimension is added to design theory: Is it an artifact we are dealing with or is an organism that grows and changes haphazardly? So like in the field of strategy, there are design views of strategy that can be planned, as well as emergent strategy that emerges due to happenstance. One way to deal with this perhaps is to think of this as Redesign Theory rather than a design theory. This fits with many of the attributes and assumes that the successive redesigns end up being more important than the original design because of the rapid evolution over time of the technical requirements and the functional requirements. Similarly, taking a cue from the field of strategy, Michael Porter has talked about betting, hedging, and flexing, when thinking of various postures for planned strategy. The recent work by Gosain of the University of Maryland would be helpful here. I went back and re-read the Walls et. al 1992 paper as well as their more recent update in the Journal of Information Technology Theory and Applications (JITTA 2004) which you may not have seen “Assessing Information System Design Theory in Perspective: How Useful was our 1992 Rendition?” One of the issues that they mention is the distinction between different types of theories: explanatory or descriptive theories that tell us “what is, ” normative theories that tell us “what should be,” predictive theories that tell us “what will be” – while design theories tell us “how to/because.” That might help structure the way that the kernel theories are described in the II case and its evolution. Would these be predictive theories? Which suggested how IIs would evolve, depending on initial design and then the design theory would tell us how to manage this evolution to satisfy the meta-design requirements? - where managing the evolution is like redesign. Thus the design process would be a longitudinal process (like an IS lifecycle) but rather would depend on what stage in the life of II and what initial strategies were selected. Thus the design process would be contingent on the initial design conditions and the evolution that occurred. Also as different from the Walls et. al. case, the evolution would be dependent on the “crowd behavior” or “rules of engagement” of the developers (open source), and the design theory or redesign theory would need to accommodate that. I think structuring that way rather than as a set of design principles that look like rules of thumb or heuristics would be a more defensible structure for II (or platform) design theory. It is also unclear whether the design should Also, design theories are meant to be forward looking rather than backward looking. Thus they should provide guidelines for design and redesign for future classes of platforms and ISs as well. The structure of the exposition (even if the case study is backward looking) should keep that in mind. In summary both the structure and components of the design theory, and the way that the kernel theories are presented needs rethinking and much tightening into more of a better-formed theoretical argument. 4. Organization & Flow Sequence -- and Clarity and Exposition As mentioned above, it would seem that a clear definition of what an II is (or platforms) is needed at the front with careful articulation, diagrams etc.. then a review of design theories for classes of IS, and then a section on how that can be built on to serve classes of IIs / Platforms. The section 3. titled research methods does not do much for me and just hangs there. It would be better incorporated into the body of the discussion on building and testing design theories for IIs/platforms. Section 4 about kernel theories should be about that rather than about definitions of what it is we are building a design theory for. Then Section 6 6. guidelines for design do not mesh well with a design theory (even though good ideas) and are not tight, and confuse redesign with a flexibility strategies. They also seems to degenerate into rules of thumb or heuristics. In section 7, I do not find the internet example convincing for reasons already mentioned above. Pages 26-35 go on and on and I did not get much out of them. In Section 8, the Norwegian EDI example needs to be more tightly coupled with the design theory if developed differently. There a few references quoted that are not in bibliography (for example Gosain 2003). Conclusion section will change as paper changes. The entire paper is much too long and looks bundled rather than streamlined. This is a type of paper that is not easy to assimilate so there is a much greater need for clarity and flow. Good luck on making this the impactful paper it can be ! Reviewer #2 Comments on ISR #2005-096 for the Authors A key strength of this manuscript lies in what you are attempting to do: theorize about information infrastructures (such as sectoral EDI networks) in terms that would support people who are trying to design such infrastructures. The case examples are also a plus, although I believe they need to be augmented significantly. My major concern about the paper is that it has no real conceptualization of the designer of information infrastructures. In your cases, no individual or organizational designers are ever named, despite the fact that you make numerous claims about their intentions. “The Internet has constantly increased its installed base by building…” (p. 31) is a case in point. There’s no designer subject here. Quotations from named parties would go a long way to making a more persuasive argument. More critically, I believe your lack of conceptualization of the designer reflects a failure to confront the central paradox of II: yes, there are human designers who do intend to bring certain outcomes about, but almost by definition as you point out, infrastructures evolve without central management, especially in situations involving multiple organizations. Can we really say these entities are/can be designed? (See my comments below about levels of analysis and inheritance of design principles across levels.) Overall the structure of the paper did not work for me. I found it too “flat” (too many major sections) and rambling. I couldn’t figure out how the argument was constructed and how it would unfold as I read. Your paper starts with a focus on design theory (rather than II), as though the goal of developing a design theory for II is unproblematic. It seems to me that the need for design theory is something that should be established through your analysis, rather than the starting point. I’d rather see more emphasis on II at the outset (see comments below about literature support), including possibly your cases, rather than on design theory, until you’ve convinced me about the design theory goal. I’d recommend that you organize the literature review around what is known about II at each of three levels of analysis in this order: Intraorganizational Business sector Internet This not only reflects historical evolution, it affords a basis for arguing about levels of analysis effects, e.g., principles at the higher level might be inherited by lower levels, although the reverse is not likely to be true; see below. I’d rather see you start with the cases and end with the design principles that to use the design principles to analyze the cases. Other comments: You present three classes of information infrastructures (p. 13) without dealing with what I think are the most important issue here: differences in levels of analysis (cf. Klein and Kozlowski, Multilevel Theory, Research, and Methods, Jossey-Bass 2000), specifically with respect to the issue of governance, which is related to design. o In Weill’s more recent work—with Jeanne Ross, it’s clear that the architecture of infrastructures is heavily tied up with issues of governance. I believe your design theory should address governance contingencies. o Information infrastructures inside organizations can be designed (because they have central management) to a greater extent than II that cross organizational lines within a bounded community—e.g., business partners; industry sectors—and to a much greater extent than our one example (?) of worldwide II, i.e., the Internet. For example, Bill Dutton, who runs the Oxford conference on Internet Governance, believes the very term “Internet Governance” is an oxymoron—it isn’t and probably cannot be governed, he believes. o You need to consider whether your arguments apply to all levels; whether, for example, design principles relevant to organizational information infrastructures necessarily apply to the Internet and whether design principles derived from (largely unmanaged?) growth the Internet necessarily apply to the proactive design of sector infrastructures or organizational infrastructues. You might think of these relationships in terms of “inheritance” in object orientated design terms or in terms of “compilation versus composition” (Klein and Kozlowski; who also have an extended discussion of types of emergence). I do not think you’ve done a very good job of reviewing and critiquing prior work on information infrastructures. o There’s been quite a bit of recent work on (intraorganizational) “IT infrastructure flexibility” and on “systems integration” by people like Kevin Zhu, Greg Truman, and others (Byrd and Turner, JMIS 2000; Kayworth and Sambamurthy, 2000; Chung et al, CAIS 2003) that gives a different perspective from that of Weill and Broadbent. If you don’t think it’s relevant, that’s fine, but you should explain why. o Regarding the dynamics of complex systems (p. 18), it seemed to me that the ideas from Technology and Information Management about component level changes versus architectural changes (Teece, 1986) were relevant. o See INTERNET TECHNOLOGY DIFFUSION: ADOPTION OF IPV6 by A Hovav, R Patnayakuni, in CAIS as an interesting example of the legacy technology traps you discuss on page 19. o You should cite Law on “heterogeneous engineering” o I thought you might find the following articles relevant: Hargadon and Douglas, When Innovations Meet Institutions: Edison and the Design of the Electric Light, ASQ 2001 Argyres, The impact of information technology on coordination: Evidence from the B-2 ‘Stealth’ bomber, Org Sci 1999 P. 2. I do not agree that information infrastructures necessarily involve standardized interfaces. Standardization is, I believe, a design principle of a good II, not an essential characteristic of all II. Many organizational II suffer from lack of standardization and partial standardization, so that some parts cannot seamlessly interconnect with the others. P. 5. “or a set of Information Systems for designing a single IS”—unclear. P. 7 restate what you are trying to do, prior to the methodology section. Reviewer #3 Comments on ISR #2005-096 for the Authors The authors define Information Infrastructures (II) as “a shared, evolving, heterogeneous installed base of IT capabilities among a set of user communities based on open and/or standardized interfaces.” (pgs. 2 and 9) This is a large and interesting target for research. However, as with any large target, it is easy to hit but hard to bring down. This is my concise evaluation of this paper; the authors have made a number of interesting observations on how to plan and develop II’s but, in the end, make only an incomplete contribution toward a rigorous, comprehensive II design theory. To begin, I find Information Infrastructures (II) to be an inadequate term to reflect the content of such large and complex networks of systems. A focus on information shortchanges the richness of services, workflows, qualities (reliability, security, performance, etc.), and other capabilities provided by these infrastructures. A comprehensive II (I will use the term in this review) design theory must encompass an understanding of all embodied system artifacts and capabilities. The authors’ research approach draws from the theories of evolutionary economics and sociotechnical systems. From these theories, they develop a set of design principles and design guidelines for the planning and development of II’s (Table 4). As such, I find their observations insightful but still far from the goal of providing a formal II design theory. Even though research limitations are noted in the paper’s concluding remarks, I would disagree with the statement “The theory is also limited in that we strove for a higher level of generality instead of accuracy in keeping the theory relatively simple.” (p. 42) This statement implies that the proposed design principles are abstract enough to subsume all necessary and sufficient detailed (i.e., lower-level) design issues. Instead, I find significant gaps in the authors’ design theory proposals. A full understanding of complex infrastructure design must include the architectural theories of systems, software, services, qualities, and information. A few of the many relevant references to the theory and practice of architecture design are (Shaw and Garlan 1996, Maier and Rechtin 2000, Bass et al. 2003, Rozanski and Woods 2005, Erl 2005). Throughout this paper, the concepts of layered architectures are implied (e.g., Figure 1) but not rigorously presented as integral to the proposed design theory. Let me cite just two essential issues that I believe a comprehensive design theory must address. The design of security features is critical to provide a level of user trust in an II. Design quality attribute tradeoffs, such as user protections vs. performance and reliability, must be clearly described in a design theory. How are user transactions defined and executed across a network of varied services? Design issues of workflow management and Service-Oriented Architectures (SOA) are clearly relevant and must be addressed. In summary, the authors’ view through a lens of economic theory provides only a partial picture of a comprehensive II design theory. I do not find it surprising that the proposed design guidelines map well into the two examples of the Internet as an II success and Norway’s health care system as an II failure. However, this is not a convincing demonstration of either the completeness or accuracy of the design theory. I conjecture that by the application of other theoretical lens, such as the architectural theories mentioned above, a different and complementary set of design principles and design guidelines can be developed and evaluated on the same examples with similar success. As an active researcher in the field of complex system design, I agree with the research motivation of this paper. A more rigorous understanding of II design theory is needed to support the engineering of effective and efficient infrastructures. The development of a comprehensive II design theory, however, will require the utilization of a wider range of contributing theories from economic, behavioral, and technical fields to generate a more complete and testable set of design principles and guidelines. The content of this paper is a partial step in the right direction. I do not believe that the research contribution is mature enough for publication in ISR. Thus, I recommend rejection of this paper and encourage the authors to broaden their research to include additional theory bases. Other comments for improvement of the paper are: There are many, many citations in the paper that are not included in the reference list. For one example, (Tuomi 2001) is cited frequently but not listed. The word artifact (artefact) is spelled two different ways. The first is preferred. On page 5, I disagree with your claim that all design theories assume that ‘a majority of system requirements can be specified during design time.’ In the vertical and horizontal design theories that you reference before this statement, I find no clear dependence on the assumption of requirements being fully specified at design time. In particular, flexible processes assume that requirements will change. On page 9, the statement that ‘an II has no fixed purpose to justify its existence’ is an overstatement that is not true in most cases. On page 16, I fail to understand your definition of ‘cultivation’ and how it applies to II design theory. I do not find the Dahlbom and Janlert (1996) quote at all ‘crisp’. I also note that this reference is not available to the reader. Throughout your discussion on design guidelines (e.g., p. 22), I need clearer definitions and descriptions of what you mean by the terms ‘simple’ and ‘cheap’. Relative to what? There is no reference cited for the Norway health care system. Where would a reader find further information on this case study? On page 39, two short sentences are used to demonstrate that the design of the health care system violates six design theory guidelines. This is a superficial analysis that should be expanded in a future paper. I find the argument on pages 43/44 that better results would have been achieved had the designers of the Norway health care system followed this paper’s design guidelines highly debatable. The majority of the guidelines address technical design issues. The authors themselves recognize (p. 40) that organizational and political issues were of greater import to the failure of this system than technical issues. Finally, on page 45, the authors state that “If the theory was adopted, the practice of designing infrastructures would change significantly.” How?? Current best practices of complex system design are quite congruent with the design guidelines provided in this paper. Please justify this claim with examples of deficiencies in current practice. References: Bass, L., P. Clements, and R. Kazman, Software Architecture in Practice, 2nd Ed., Addison-Wesley, Inc., 2003. Erl, T., Service-Oriented Architecture: Concepts, Technology, and Design, Prentice-Hall, Inc., 2005. Maier, M. and E. Rechtin, The Art of Systems Architecting, 2nd Ed., CRC Press, 2000. Rozanski, N. and E. Woods, Software Systems Architecture, Addison-Wesley, Inc., 2005. Shaw, M. and D. Garlan, Software Architecture: Perspectives on an Emerging Discipline, PrenticeHall, Inc., 1996.