AE Report for ISR 2005-096 “Theorizing about the design of Information
Infrastructures: design kernel theories and principles
The goals of the paper are ambitious and worthwhile. Information infrastructures, when successful,
hold out considerable benefits to individuals, businesses and society. Yet, they pose special
challenges owing to a fundamental chicken-and-egg problem: they gain most of their value through
network externalities, yet we need some way to get users to adopt them even when the network is
small and network externalities have yet to become a significant factor in the value proposition.
Also, IIs ultimately become quite complex, yet some way must be found to spare initial users and
the designers themselves from this complexity, especially in the early stages when the network has
a weaker value proposition and the fate of the II is most uncertain. Finally, successful IIs are
inherently evolutionary because it is impossible to foresee the myriad of uses that will eventually be
discovered for it, and so we must find some way to harness, rather than inhibit, evolutionary forces.
I think an excellent case can be made, and to a good extent has been made, that IIs therefore pose
a fundamentally different architectural challenge than traditional IT applications. With traditional
applications the focus of design guidelines is primarily on promoting technical efficiency of one
form or another. With traditional applications we want design specifications to be “complete”
because it is assumed to be much more expensive to add features later. A high priority is given to
reliability, security etc., right from the very start, because it is assumed the system will actually be
adopted. We focus on making the system as useful as possible for each intended user in isolation,
under the implicit assumption that the value of the system comes inherently from the functionality
per se, rather that from the extent to which functionality has been adopted across the network.
To sum up, I agree with the review panel that the goals of the research are very worthwhile, and
that your design guidelines are insightful and constitute a good start on meeting the goals you set
out. We all believe that this research has the potential to make a very nice contribution to the IS
field.
However, I also agree with the reviewers that this paper is very far from the point where it needs to
be to realize this promise. Two reviewers, while noting the difficulty of the path ahead, have invited
a revision. One, one while strongly encouraging the concept of the research, feels the paper is too
far from a publishable state to invite a revision, and so has recommended rejection.
My opinion is that the promise in the manuscript is such that it would be in the interest of the
authors and the field to invite a round of revision at ISR, to see if it is possible to sufficiently
address the concerns of the reviewers within the format constraints of an ISR paper. I think that
you will be in the best position to judge whether the necessary revisions are possible, and so I
would like to extend this opportunity to you.
The reviewers have raised a number of major concerns, which I will highlight shortly. I have a fair
degree of confidence that with a conscientious effort from you, most of the concerns related to
content and positioning of the research can be addressed. However, there is one over arching
issue that I’m not sure can be addressed, and that is conforming to the length restrictions of an ISR
paper. While I think there is much potential to streamline the material that already exists, each of
the reviewers has identified major areas where more content must be added. Therefore, I think you
may need to find some way to limit your scope of attention if you are to develop a tight and
complete case for your proposed theory of II design. It is hard for me to tell how feasible this is.
In my remaining comments, I will give my own interpretation of the most important issues from the
reviewers and add some thoughts of my own. In addition, I encourage to consider each of the
individual reports most carefully, and to respond point-by-point to each reviewer in your response
to the reviewers should you attempt a revision at ISR. You are fortunate to have the benefit of
insights from a top quality review panel.
1. Concerns about Overall Paper Format and Style
The paper is very difficult to master. Part of this difficultly lies in the inherent complexity and scope
of the task you have set before yourself, and part is due to how you have actually structured and
presented your ideas. I agree with Reviewer 1’s contention that any paper that, like this one,
requires multiple close readings by domain experts to comprehend has little potential for impact
among the broader audience of ISR readers. You need to find some way to convey your thoughts
throughout the paper more simply, and with a much smoother logical flow. Reviewer 2 suggests
the paper is too “flat”, with too many major sections. I would agree that 9 major sections a lot, but
since you already have three levels of hierarchy in the paper I’m not sure what the solution is, other
than finding some way to narrow your scope.
Speaking of which, the paper has to be much shorter. This was a comment made by all three
reviewers on the grounds of digestibility alone—there is a limit to how much effort a reader can be
expected to spend to absorb the content of one paper. But just as important, the ISR guidelines
specify that a paper should be no more than 32 pages of text, and 38 pages overall. This paper is
nearly 47 pages long overall, of which I’d guess about 40 pages are text. I think that some progress
can be made just through general streamlining, however I do not think that streamlining alone can
solve the length problem, considering that all three reviewers have indicated areas where
expansion may be needed. Therefore, I think you will need some way to narrow your scope (see
Item 4 below) .
2. Paper Organization
I like Reviewer 2’s suggestion that you put greater emphasis up front on IIs before you get to
design theories, and that you organize your review around the three levels of analysis (corporate,
business sector, universal service). I suspect you have put the emphasis on design theories up
front because design theory is not a mainstream research approach, and you might have felt the
need to legitimize your approach before getting too far into specifics. However, I agree with the
reviewer that it would be better to talk about what IIs are and why we should care about their
design before getting into the details of design theories. I also think it would be most helpful if you
were to provide a list of prominent technologies that are (and are not) IIs, which kind they are, and
why. Some quick candidates that come to mind would be: the internet, EDI networks, France’s
Minitel, Japan’s DoCoMo, ISDN (in the US or elsewhere), 3G wireless, multi-application
smartcards, RFID, and various enterprise technologies (ERP, CRM, SCM, PLM, groupware,
knowledge management).
Reviewer 2 also suggests that you start with the case studies and end with the design principles. I
don’t have a clear intuition about whether this would be an improvement or not, so I’ll leave you to
consider the possibility as you are making the other revisions.
While we’re on the subject of the case examples, you state on page 8 “Finally, we used two case
studies to derive testable hypotheses from this theory and validate them for internal and external
validity”. I’m not sure I understand this sentence. How you can use the very same case studies to
both derive, and then to validate ideas? Is it even true that you derived your ideas specifically from
these two cases? Or did you perhaps draw them from a larger set of cases, the design literatures,
etc.? Also, you don’t actually offer any testable hypotheses.
3. Terminology, Definitions and Concepts Related to IIs
All the reviewers had concerns about your definitions of IIs. Reviewer 1 notes that you have
defined IIs somewhat differently at different points in the manuscript. Reviewer’s 1 and 2 both note
that you have incorporated into your definition of IIs elements that might more appropriately be
viewed as characteristics of a good II design (e.g., standardization) rather than as a essential
characteristic of all II.
Reviewer 3 finds II to be an inadequate term to “reflect the content of such large and complex
networks of systems. A focus on information shortchanges the richness of services, workflows,
qualities (reliability, security, performance, etc.), and other capabilities provided by these
infrastructures.”
Reviewer 1 also wonders if II is even the best term. The reviewer suggests “platform” or
“application platform” as a potential alternatives. Whether or not that’s a good idea depends on
what counts and does not count as an II and how you draw the bounds on the scope of your
theorizing. For example, I would call the Intel x86 architecture, MS Windows and the Oracle
database all “platforms”, but I don’t think you would call them IIs, right? Possibly something like
“Shared Information Services Platform” would work. This is broader than “application platform”, in
that I think it would encompass the Internet. I’ll leave the final terminology decisions to you, as it
ties in to other decisions regarding scope and which examples you will highlight.
Reviewer 1 had made many good points about the need to clarify the sections that review theories
of design and that develop your own theory of design — see the comments provided by this
reviewer under the heading “2. Conceptualization and Construction of Design Theory – and Kernel
Theories”.
My own observation is that your Section 4.1 is very descriptive. You are not as clear as you could
be in identifying the essential and distinctive features of IIs. Also, you gave no foreshadowing of the
mapping of these essential, distinctive features of IIs to the selection of your kernel theories. I think
these two topics have to be very tightly linked. Also, I’d like to see you link your specific design
guidelines back to your kernel theories more explicitly to the maximum extent possible. It is your
kernel theories that provide the boundaries on what “matters”. It’s very important for all of your
guidelines to stay within the scope of your kernel theories, because otherwise this opens the door
to any number of additional potentially useful guidelines, and your scope would become
unmanageable.
Finally, there are some other concepts that are not defined as clearly as they could be include
“technology traps” (pg 19), “lock-ins (pg 25)” and “usefulness” (pg 21). Regarding the latter term, I
think you might better emphasize the distinction between “inherent” usefulness or superiority that
does not depend on there being other adopters, versus aspects that do depend on other adopters.
4. Incomplete Attention to Related Literatures on Architectural Design
Reviewer 3 is concerned that the design guidelines, while insightful, are not sufficiently formal and
complete to represent a true theory of II design. The reviewer feels that you have not drawn
sufficiently on existing work on IT architectures—several books on the subject are noted. The
reviewer highlights two missing elements in particular: lack of attention to security features
(required to create trust) and lack of attention to how user transactions are defined and executed
across a network of varied services.
Since the paper is already well over the page limit, this represents a quandary. The only solution I
can see is you will need to draw boundaries on your theorizing in some way that avoids the need to
significantly expand your scope. This could be difficult. As the reviewer notes: “[T] the authors’ view
through a lens of economic theory provides only a partial picture of a comprehensive II design
theory. I do not find it surprising that the proposed design guidelines map well into the two
examples of the Internet as an II success and Norway’s health care system as an II failure.
However, this is not a convincing demonstration of either the completeness or accuracy of the
design theory. I conjecture that by the application of other theoretical lens, such as the
architectural theories mentioned above, a different and complementary set of design principles and
design guidelines can be developed and evaluated on the same examples with similar success.”
In a similar vein, Reviewer 2 notes that you’ve not attended to recent related work that has been
done on “infrastructure flexibility” and “systems integration” and a few other topics.
The concern here is that your chosen theoretical perspective privileges some aspects of design
and ignores others that, for all we know, might be equally important, and might fit equally well with
your two examples. Yet, I’m not sure how you will have space available to properly incorporate
additional theoretical perspectives. If you can’t fit additional perspectives in, you need to be able to
develop an argument for why it is appropriate to exclude them from your scope.
One way out of this dilemma might be to somehow narrow your attention to a particular class of IIS
that have certain properties that increase the salience of the particular theoretical perspectives you
bring to bear (see Item 5 below) but not others. I’m not sure if this can be done, but it’s worth
thinking about.
5. Levels of Analysis
Reviewer 2 is concerned that you have not adequately distinguished the three levels of analysis
(corporate, business sector, universal service) in your theorizing. For example, the corporate level
is much more open to centralized control than the other two. Relatedly, governance issues are
much more important for corporate infrastructures. The reviewer is concerned about whether your
design principles operate equally at all levels, and particularly whether all the principles that apply
at lower levels necessarily apply at higher levels.
One option to consider would be to confine your theorizing to just the top, or the top two levels of
analysis. This might allow you to more easily narrow your theoretical scope by attending to what is
most important at the universal service or business sector level. The most extreme change would
be to focus just on the top level, universal service. To do this you would have to pick a different
negative case example since your current example is at the business sector level. Two possibilities
for negative examples that come to mind are ISDN in the United States and multi-application smart
cards in the US. In the example if ISDN in particular I think it can easily be argued that ISDN
designers had no sensitivity whatsoever to the role of network externalities in standards adoption,
and how this might affect design decisions. For example, there was no reason that ISDN had to
“step on” the specific frequencies reserved for analog voice, but this design decision made existing
phones obsolete. So this violated the principle of reusing existing infrastructure to extend possible.
If you were to focus on just the top level, or just the top two levels, you could still talk about the
other levels, and the possibility of generalizing the design guidelines and principles to those levels,
in a section on future work.
6. Lack of Attention to the Designer
Reviewer 2 has a major concern that you have no real conceptualization of the designer of
information infrastructures. No individual or organizational designers are named, but you
nevertheless make many claims about their intentions. The reviewer believes that quotations from
designers could make your argument more persuasive.
The reviewer goes on to note that this highlights a central paradox of IIs: while there are certainly
designers involved, the infrastructures evolve without central management, raising a question of
whether they are actually designed. This comment is increasingly weighty as you move up the
levels of analysis to the universal service level.
One possible way to address this paradox is to highlight the difference between design guidelines
and the actual design of specific features. The internet was never “designed” specifically to do
most of the things it actually does now, but, as you show, certain design principles and guidelines
were implicitly followed in the overall architecture and also the subsequent design of individual
components. Take a web browser. Early browsers could just as easily been designed as closed
programs, but they weren’t. From the start they were designed to allow third party “plug ins”. That
was a key architectural choice that was made that is consistent with your design principles and
guidelines, and also your kernel theories. Allowing third-party plug-ins allows browsers to harness
network externalities and to promote evolutionary growth in ways that would be impossible if one
company were to control how browsers worked.
7. Other Issues
Reviewer 3 raises some addition points that must be attended to, especially (1) the need to expand
out the rationales for concluding that the Norway medical systems did not conform to certain
design guidelines, (2) the need to better justify the contention that conformance to the design
guidelines would have lead to success given the organizational and political challenges, (3) the
need to justify the statement that adoption of your proposed guidelines would lead to major
changes in what constitutes best practices in the design of IIs.
Finally, there are a number of stylistic problems to address. You don’t use consistent labeling in the
tables and in the text for your guidelines. Many citations are missing from the bibliography. There is
a larger than average number of typographical errors. There were many individual sentences that I
had difficulty making sense of. This all suggests the need for more attention to proofing and
polishing before resubmission, and possibly the services of a professional editor.
Closing
As described above, much work must be done to get this paper to the point where it would be
suitable for publication in ISR, and the outcome is uncertain. However, regardless of how you
proceed from here, I want to reemphasize the great potential the reviewers and I see in what you
are attempting to accomplish. I hope that you will persist in this endeavor!
Reviewer #1 Comments on ISR #2005-096 for the Authors
This is a very interesting paper in terms of its direction and potential. The paper has strengths is
that it addresses a critical topic, is creatively gutsy, draws on previous work on IS design theory
and builds on it, as well tries to use theoretical treatises from various disciplines to help the
arguments and construction. This is a deep but complex paper in a difficult area that has much
promise of having an impact on the IS community if executed well and revised. I hope that the
authors will rise to the challenge. It requires both substantial rethinking and rewriting. It is a difficult
paper to write – and I might add to review.
It currently suffers from weaknesses that will limit both its impact and understandability. Currently it
is too long and complex, and the flow of arguments and sequence is not conducive to assimilation,
and in addition there are a number of confusing ambiguities that require much rethinking. If cleared
this will be an excellent contribution. If not cleared, it will not have any impact. I had to read this
paper repeatedly and I have thought of these issues before. Unless I was having a bad day each
time I looked at it, it would seem to me that the average IS academic reader would have trouble
digesting what to do with this and how to use it to move forward. I have tried to help the authors
resolve some of the problems. There are issues of definition and scope, issues of
conceptualization and construction, issues of clarity/exposition and flow sequence and argument. It
is not ready yet and requires substantial rework.
1. Rethinking & redefining an information infrastructure (or an application platform or ?)
I struggled with the names and definitions and had to take a step back. Here is how I view what we
know and what we don’t know in the IS academic community around this:
- We all know what an IS is and that there are classes and instances of ISs.
- We all know what IT is and that it a heterogeneous concept rather than a monolith, and that IS is
IT in context and contextual variables around it depending on whether you subscribe to the “IT
artifact” or “organizational systems” view of the world.
- We know that IT infrastructure is a blurry term that denotes hardware and software that an IS
application sits on top of.
- We are not quite sure what an information infrastructure is and whether it includes hardware and
operating systems software, or is it like a database management program, or it what glues various
IS applications together like the types of intelligent “data plumbing” architecture which are being
developed in the wireless world to move data seamlessly across devices as user moves. For
example, the new “iobi” platform that Verizon Wireless has deployed for moving multimedia across
different mobile devices , operating systems, locations, and screen sizes.
A diagram would really help to distinguish between, IS, IT, IT infrastructure, Information
Infrastructure,… I would also consider the term “Platform” as another possible way to think about
it in terms of something that various classes of IS can sit on top of. Platforms give the impression
that they are closer to applications, and that they can still sit on top of IT infrastructure. Intel
Corporation has recently adopted that term as they redefine their strategic vision and move beyond
IT infrastructure. As we move towards service-oriented architectures and web services, the notion
of a huge monolith application such as ERP will go away, and there will be tiny callable
components that sit on top of some kind of ERP platform or mobile multimedia communication
platform or …
Now let me go through how the paper came through on that. I had a problem with the definition of
information infrastructures (II) throughout the paper. You start by calling them heterogeneous
socio-technical systems on page 2 where you mention their defining attributes (large, complex,
adaptive, no one designer,…). The paper then tries to link IIs to related work on page 6
(improvisation, colonial systems, infrastructure as portfolio,…). Then on page 9 there is a formal
attempt to define an II and there it becomes an installed base of IT capabilities with a defined user
community and open interfaces. It then attempts to elaborate on that with a somewhat complex
write-up that is descriptive rather than theoretical. Then on page 12, it tries to simplify the
complexity by making it vertical and horizontal, and I am not sure that slicing works very well and
that it can be cut up that way. I had a lot of problems with the classes in Table 3 and the
distinguishing characteristics and seemed like a force fit.
I am not sure how the verticality works in Table 3. I can see the internet or Internet 2 or serviceoriented architecture Internet or whatever else we will get going forward into the future --- as a
universal supra-IT infrastructure (as convoluted as that sounds) which houses many platforms,
however the other two are much more tenuous. Is verticality based on the type of information
exchange standard (EDI) or the type of information integration across a user community (ERP)??
Seems like user community domain requirements, functional requirements, and technical
requirements are being blurred here in terms of verticality. I think this requires much more thought.
Perhaps thinking of the internet as a universal supra-IT infrastructure over which many IS platforms
can be built and used might be another way to go. And IS platforms can then be defined around
user communities (or industries and markets nor business sectors) such as health or financial
services or manufacturing,… ) or functionalities (transactions, relationships, community). In any
case, Table 3 is problematic and difficult for others to build on theoretically.
Then on page 14 the paper starts a discussion of horizontal infrastructures, and while I am not sure
I agree that this is “horizontal” and looks more vertical, I really like the flavor of Figure 1 and think
that the paper should start from something like that to give the reader some grounding and for
forcing the better articulation and definition. The internet is clearly in the domain of support
infrastructures (or supra-IT infrastructures if you believe that argument) and the transport and
service parts are clear. Is the operating systems software part of that or is there a middleware layer
? Then is it the applications infrastructure that sits on top of that or should that be divided into two
layers: an application platform ( for a class of applications based on communities or types of
information exchange…) and then applications (types of user functionalities or types of
interactions). Types of interactions can be transactions (ERP), relationships (CRM), planning and
rescheduling (SCM), monitoring and scanning (business intelligence). Thus the design theory
would focus on “application platforms” or “xx platforms” which would still capture the evolution,
heterogeneity, have both technical and functional requirements, …….. rather than on designing
the internet. Seems like an application-oriented view of IT infrastructure (application platform)
would be closer to the concerns of the IS community rather than computer scientist. So, I would
see the Norwegian healthcare example being closer to that, but the evolution of the internet as not
being a good example or illustration. In short, you may want to rethink your layers more carefully,
take the business/user-oriented perspective into account more directly, and address a thinner more
focused sliver of the figure 1 structure. Distinguishing between infrastructures and platforms might
be a way to finesse that for the IS research community so that they can build on this paper and
help advance design theory as well. Thus, like the Walls et. al 1992 paper focused on design
theory for a class of information systems, this could focus on a class of application platforms. I
think it is more doable that way and will have more impact. And your kernel theories still work.
I really the beginning of the paper needs to lay this out very clearly (and distinguish between all
these terms/concepts), so readers know what it is that we really trying to find a design theory for,
and we understand where IS researchers can make an impact, and where computer scientists can.
It is not an easy definition – as in a sense you are redefining our view of architecture and
infrastructure, and what an application is,… but it is crucial. Unless that is done clearly and
convincingly at start (and diagrams help immensely), the rest will have little impact.
2. Conceptualization and Construction of Design Theory – and Kernel Theories
The review of IS design theories is reasonable although not as streamlined as it could be. The
authors point out an interesting distinction between classes of information systems (authors call
this vertical demarcation) and classes of design problems (authors call this horizontal
demarcation). In the Walls et. al 1992 paper and rendition of an IS design theory, an IS design
theory takes into account both design product and the design process. It includes both a design
product set of components (meta-requirements, meta-design, kernel theories, testable design
product hypotheses) coupled with a design process set of components (kernel theories, design
methods, and testable design process hypotheses). Thus, what the authors call classes of design
problems are derived in Walls et al. from a kernel theory and are coupled to meta-requirements for
a class of information systems, rather than decoupled rules of thumb which is what design
principles are if they are not based on kernel theories nor linked to meta-requirements. Thus, are
there universal design principles that cut across different classes of information systems as the
authors claim or are these then rules of thumb? Thus after defining what an II (or platform is) and
its attributes that are different in nature than those for an IS -- there is a need to articulate the
structure of IS design theory in a convincing way: is it different from the Walls et al structure ? and
what is the appropriate structure and why ? I do not think the paper does that well enough.
The paper identifies some attributes for IIs:
- Large
- Complex
- Adapt to future changes in technical requirements
- Adapt to future changes in functional requirements
(by the way the distinction between technical requirements and functional requirements may
become more blurred with new architectures like web services)
- Designed as extensions or improvement to existing IIs
- Diverse components not under the control of one designer (almost like open source)
Then pages 17-26 tries to take on the difficult task of deriving kernel theories that can suggest what
the design process might be. As one looks at those characteristics together a new dimension is
added to design theory: Is it an artifact we are dealing with or is an organism that grows and
changes haphazardly? So like in the field of strategy, there are design views of strategy that can
be planned, as well as emergent strategy that emerges due to happenstance. One way to deal
with this perhaps is to think of this as Redesign Theory rather than a design theory. This fits with
many of the attributes and assumes that the successive redesigns end up being more important
than the original design because of the rapid evolution over time of the technical requirements and
the functional requirements. Similarly, taking a cue from the field of strategy, Michael Porter has
talked about betting, hedging, and flexing, when thinking of various postures for planned strategy.
The recent work by Gosain of the University of Maryland would be helpful here.
I went back and re-read the Walls et. al 1992 paper as well as their more recent update in the
Journal of Information Technology Theory and Applications (JITTA 2004) which you may not have
seen “Assessing Information System Design Theory in Perspective: How Useful was our 1992
Rendition?” One of the issues that they mention is the distinction between different types of
theories: explanatory or descriptive theories that tell us “what is, ” normative theories that tell us
“what should be,” predictive theories that tell us “what will be” – while design theories tell us “how
to/because.” That might help structure the way that the kernel theories are described in the II case
and its evolution. Would these be predictive theories? Which suggested how IIs would evolve,
depending on initial design and then the design theory would tell us how to manage this evolution
to satisfy the meta-design requirements? - where managing the evolution is like redesign. Thus
the design process would be a longitudinal process (like an IS lifecycle) but rather would depend
on what stage in the life of II and what initial strategies were selected. Thus the design process
would be contingent on the initial design conditions and the evolution that occurred. Also as
different from the Walls et. al. case, the evolution would be dependent on the “crowd behavior” or
“rules of engagement” of the developers (open source), and the design theory or redesign theory
would need to accommodate that. I think structuring that way rather than as a set of design
principles that look like rules of thumb or heuristics would be a more defensible structure for II (or
platform) design theory. It is also unclear whether the design should
Also, design theories are meant to be forward looking rather than backward looking. Thus they
should provide guidelines for design and redesign for future classes of platforms and ISs as well.
The structure of the exposition (even if the case study is backward looking) should keep that in
mind.
In summary both the structure and components of the design theory, and the way that the kernel
theories are presented needs rethinking and much tightening into more of a better-formed
theoretical argument.
4. Organization & Flow Sequence -- and Clarity and Exposition
As mentioned above, it would seem that a clear definition of what an II is (or platforms) is needed
at the front with careful articulation, diagrams etc.. then a review of design theories for classes of
IS, and then a section on how that can be built on to serve classes of IIs / Platforms.
The section 3. titled research methods does not do much for me and just hangs there. It would be
better incorporated into the body of the discussion on building and testing design theories for
IIs/platforms. Section 4 about kernel theories should be about that rather than about definitions of
what it is we are building a design theory for. Then Section 6 6. guidelines for design do not
mesh well with a design theory (even though good ideas) and are not tight, and confuse redesign
with a flexibility strategies. They also seems to degenerate into rules of thumb or heuristics.
In section 7, I do not find the internet example convincing for reasons already mentioned above.
Pages 26-35 go on and on and I did not get much out of them. In Section 8, the Norwegian EDI
example needs to be more tightly coupled with the design theory if developed differently. There a
few references quoted that are not in bibliography (for example Gosain 2003). Conclusion section
will change as paper changes. The entire paper is much too long and looks bundled rather than
streamlined. This is a type of paper that is not easy to assimilate so there is a much greater need
for clarity and flow.
Good luck on making this the impactful paper it can be !
Reviewer #2 Comments on ISR #2005-096 for the Authors
A key strength of this manuscript lies in what you are attempting to do: theorize about information
infrastructures (such as sectoral EDI networks) in terms that would support people who are trying
to design such infrastructures. The case examples are also a plus, although I believe they need to
be augmented significantly.
My major concern about the paper is that it has no real conceptualization of the designer of
information infrastructures. In your cases, no individual or organizational designers are ever
named, despite the fact that you make numerous claims about their intentions. “The Internet has
constantly increased its installed base by building…” (p. 31) is a case in point. There’s no designer
subject here. Quotations from named parties would go a long way to making a more persuasive
argument.
More critically, I believe your lack of conceptualization of the designer reflects a failure to confront
the central paradox of II: yes, there are human designers who do intend to bring certain outcomes
about, but almost by definition as you point out, infrastructures evolve without central management,
especially in situations involving multiple organizations. Can we really say these entities are/can be
designed? (See my comments below about levels of analysis and inheritance of design principles
across levels.)
Overall the structure of the paper did not work for me. I found it too “flat” (too many major sections)
and rambling. I couldn’t figure out how the argument was constructed and how it would unfold as I
read.
Your paper starts with a focus on design theory (rather than II), as though the goal of developing a
design theory for II is unproblematic. It seems to me that the need for design theory is something
that should be established through your analysis, rather than the starting point. I’d rather see more
emphasis on II at the outset (see comments below about literature support), including possibly your
cases, rather than on design theory, until you’ve convinced me about the design theory goal. I’d
recommend that you organize the literature review around what is known about II at each of three
levels of analysis in this order:
 Intraorganizational
 Business sector
 Internet
This not only reflects historical evolution, it affords a basis for arguing about levels of analysis
effects, e.g., principles at the higher level might be inherited by lower levels, although the reverse is
not likely to be true; see below.
I’d rather see you start with the cases and end with the design principles that to use the design
principles to analyze the cases.
Other comments:

You present three classes of information infrastructures (p. 13) without dealing with what I
think are the most important issue here: differences in levels of analysis (cf. Klein and




Kozlowski, Multilevel Theory, Research, and Methods, Jossey-Bass 2000), specifically
with respect to the issue of governance, which is related to design.
o In Weill’s more recent work—with Jeanne Ross, it’s clear that the architecture of
infrastructures is heavily tied up with issues of governance. I believe your design
theory should address governance contingencies.
o Information infrastructures inside organizations can be designed (because they
have central management) to a greater extent than II that cross organizational
lines within a bounded community—e.g., business partners; industry sectors—and
to a much greater extent than our one example (?) of worldwide II, i.e., the
Internet. For example, Bill Dutton, who runs the Oxford conference on Internet
Governance, believes the very term “Internet Governance” is an oxymoron—it isn’t
and probably cannot be governed, he believes.
o You need to consider whether your arguments apply to all levels; whether, for
example, design principles relevant to organizational information infrastructures
necessarily apply to the Internet and whether design principles derived from
(largely unmanaged?) growth the Internet necessarily apply to the proactive
design of sector infrastructures or organizational infrastructues. You might think of
these relationships in terms of “inheritance” in object orientated design terms or in
terms of “compilation versus composition” (Klein and Kozlowski; who also have an
extended discussion of types of emergence).
I do not think you’ve done a very good job of reviewing and critiquing prior work on
information infrastructures.
o There’s been quite a bit of recent work on (intraorganizational) “IT infrastructure
flexibility” and on “systems integration” by people like Kevin Zhu, Greg Truman,
and others (Byrd and Turner, JMIS 2000; Kayworth and Sambamurthy, 2000;
Chung et al, CAIS 2003) that gives a different perspective from that of Weill and
Broadbent. If you don’t think it’s relevant, that’s fine, but you should explain why.
o Regarding the dynamics of complex systems (p. 18), it seemed to me that the
ideas from Technology and Information Management about component level
changes versus architectural changes (Teece, 1986) were relevant.
o See INTERNET TECHNOLOGY DIFFUSION: ADOPTION OF IPV6 by
A Hovav, R Patnayakuni, in CAIS as an interesting example of the legacy
technology traps you discuss on page 19.
o You should cite Law on “heterogeneous engineering”
o I thought you might find the following articles relevant:
 Hargadon and Douglas, When Innovations Meet Institutions: Edison and
the Design of the Electric Light, ASQ 2001
 Argyres, The impact of information technology on coordination: Evidence
from the B-2 ‘Stealth’ bomber, Org Sci 1999
P. 2. I do not agree that information infrastructures necessarily involve standardized
interfaces. Standardization is, I believe, a design principle of a good II, not an essential
characteristic of all II. Many organizational II suffer from lack of standardization and partial
standardization, so that some parts cannot seamlessly interconnect with the others.
P. 5. “or a set of Information Systems for designing a single IS”—unclear.
P. 7 restate what you are trying to do, prior to the methodology section.
Reviewer #3 Comments on ISR #2005-096 for the Authors
The authors define Information Infrastructures (II) as “a shared, evolving, heterogeneous installed
base of IT capabilities among a set of user communities based on open and/or standardized
interfaces.” (pgs. 2 and 9) This is a large and interesting target for research. However, as with any
large target, it is easy to hit but hard to bring down. This is my concise evaluation of this paper; the
authors have made a number of interesting observations on how to plan and develop II’s but, in the
end, make only an incomplete contribution toward a rigorous, comprehensive II design theory.
To begin, I find Information Infrastructures (II) to be an inadequate term to reflect the content of
such large and complex networks of systems. A focus on information shortchanges the richness of
services, workflows, qualities (reliability, security, performance, etc.), and other capabilities
provided by these infrastructures. A comprehensive II (I will use the term in this review) design
theory must encompass an understanding of all embodied system artifacts and capabilities.
The authors’ research approach draws from the theories of evolutionary economics and sociotechnical systems. From these theories, they develop a set of design principles and design
guidelines for the planning and development of II’s (Table 4). As such, I find their observations
insightful but still far from the goal of providing a formal II design theory. Even though research
limitations are noted in the paper’s concluding remarks, I would disagree with the statement “The
theory is also limited in that we strove for a higher level of generality instead of accuracy in keeping
the theory relatively simple.” (p. 42) This statement implies that the proposed design principles are
abstract enough to subsume all necessary and sufficient detailed (i.e., lower-level) design issues.
Instead, I find significant gaps in the authors’ design theory proposals.
A full understanding of complex infrastructure design must include the architectural theories of
systems, software, services, qualities, and information. A few of the many relevant references to
the theory and practice of architecture design are (Shaw and Garlan 1996, Maier and Rechtin
2000, Bass et al. 2003, Rozanski and Woods 2005, Erl 2005). Throughout this paper, the
concepts of layered architectures are implied (e.g., Figure 1) but not rigorously presented as
integral to the proposed design theory. Let me cite just two essential issues that I believe a
comprehensive design theory must address.
 The design of security features is critical to provide a level of user trust in an II. Design
quality attribute tradeoffs, such as user protections vs. performance and reliability, must be
clearly described in a design theory.
 How are user transactions defined and executed across a network of varied services?
Design issues of workflow management and Service-Oriented Architectures (SOA) are
clearly relevant and must be addressed.
In summary, the authors’ view through a lens of economic theory provides only a partial picture of a
comprehensive II design theory. I do not find it surprising that the proposed design guidelines map
well into the two examples of the Internet as an II success and Norway’s health care system as an
II failure. However, this is not a convincing demonstration of either the completeness or accuracy
of the design theory. I conjecture that by the application of other theoretical lens, such as the
architectural theories mentioned above, a different and complementary set of design principles and
design guidelines can be developed and evaluated on the same examples with similar success.
As an active researcher in the field of complex system design, I agree with the research motivation
of this paper. A more rigorous understanding of II design theory is needed to support the
engineering of effective and efficient infrastructures. The development of a comprehensive II
design theory, however, will require the utilization of a wider range of contributing theories from
economic, behavioral, and technical fields to generate a more complete and testable set of design
principles and guidelines. The content of this paper is a partial step in the right direction. I do not
believe that the research contribution is mature enough for publication in ISR. Thus, I recommend
rejection of this paper and encourage the authors to broaden their research to include additional
theory bases.
Other comments for improvement of the paper are:

There are many, many citations in the paper that are not included in the reference list. For
one example, (Tuomi 2001) is cited frequently but not listed.

The word artifact (artefact) is spelled two different ways. The first is preferred.

On page 5, I disagree with your claim that all design theories assume that ‘a majority of
system requirements can be specified during design time.’ In the vertical and horizontal
design theories that you reference before this statement, I find no clear dependence on the
assumption of requirements being fully specified at design time. In particular, flexible
processes assume that requirements will change.

On page 9, the statement that ‘an II has no fixed purpose to justify its existence’ is an
overstatement that is not true in most cases.

On page 16, I fail to understand your definition of ‘cultivation’ and how it applies to II
design theory. I do not find the Dahlbom and Janlert (1996) quote at all ‘crisp’. I also note
that this reference is not available to the reader.

Throughout your discussion on design guidelines (e.g., p. 22), I need clearer definitions
and descriptions of what you mean by the terms ‘simple’ and ‘cheap’. Relative to what?

There is no reference cited for the Norway health care system. Where would a reader find
further information on this case study?

On page 39, two short sentences are used to demonstrate that the design of the health
care system violates six design theory guidelines. This is a superficial analysis that should
be expanded in a future paper.

I find the argument on pages 43/44 that better results would have been achieved had the
designers of the Norway health care system followed this paper’s design guidelines highly
debatable. The majority of the guidelines address technical design issues. The authors
themselves recognize (p. 40) that organizational and political issues were of greater import
to the failure of this system than technical issues.

Finally, on page 45, the authors state that “If the theory was adopted, the practice of
designing infrastructures would change significantly.” How?? Current best practices of
complex system design are quite congruent with the design guidelines provided in this
paper. Please justify this claim with examples of deficiencies in current practice.
References:
Bass, L., P. Clements, and R. Kazman, Software Architecture in Practice, 2nd Ed., Addison-Wesley,
Inc., 2003.
Erl, T., Service-Oriented Architecture: Concepts, Technology, and Design, Prentice-Hall, Inc.,
2005.
Maier, M. and E. Rechtin, The Art of Systems Architecting, 2nd Ed., CRC Press, 2000.
Rozanski, N. and E. Woods, Software Systems Architecture, Addison-Wesley, Inc., 2005.
Shaw, M. and D. Garlan, Software Architecture: Perspectives on an Emerging Discipline, PrenticeHall, Inc., 1996.