Luhmann meets Weick: Information interoperability and Situational

advertisement
Luhmann meets Weick: Information interoperability and Situational awareness
ABSTRACT
Technology and technological applications are increasingly driving our perception of reality.
Reality is the product of interaction between random social and technological systems and
entities, and of the information exchanged and shared in that interaction. Shared meaning
of new information can contribute to the creation of ‘situational awareness’ in complex
situations. Does a combination of social systems theory and the theory of sensemaking
contribute to the theoretical development and creation of this communication process
between random systems and entities?
Keywords: enactment, interoperability, interpenetration, sensemaking, situational
awareness, systems theory
0
0
Luhmann meets Weick: Information interoperability and Situational awareness
INTRODUCTION
Technology and technological applications are increasingly driving our perception of reality
within and outside organizations. Information interoperability enables connections between
man and machine (technology and technological applications), between machines amongst
themselves, and between machines and man. Information interoperability hence creates
new opportunities for information exchange and sharing between random autonomous
systems and entities, and to use that information for further production, functioning or
generation. Exchanging and sharing a growing volume of information between random
systems and entities in coalitions and networks does, however, also throw up questions
about the way meaning is assigned to this information within organizations. Especially
under exceptional circumstances, such as disasters or crises, or with operations where
large volumes of complex information must be processed under great pressure, this
process of assigning meaning becomes very important. This essay attempts to, on the one
hand, formulate a theoretical basis for the exchange and sharing of information based on
Luhmann's social systems theory, and, on the other hand, link that theory to Weick's theory
of sensemaking.
Sociologist and administration expert Niklas Luhmann's social systems theory has the
discernible unit of communication comprising information, utterance and understanding at
its core. This discernible unit of communication, which is engendered by or from a system,
can be received or rejected by another system. Luhmann draws on the concept of
interpenetration to show that systems are involved in a reciprocal process of contributing to
the development of systems in their environment. Luhmann claims that interpenetration
comes about when systems are willing and able to open their created complexity up to
other systems in their environment. System interpenetration thus creates new information
within the system in the form of ‘Sinn’ (meaning), bringing about new and further activities.
Luhmann uses the neologism sensemaking to refer to this process. The concept of
sensemaking was introduced by social psychologist and organizational theorist Karl E.
Weick. Weick's sensemaking concept departs from a social system in the form of an
organization with an ability to assign meaning to incoming information. The receiving
system will take action on the basis of the meaning assigned. Defining and assigning
meaning to incoming information is, in Weick's view, a process of enactment. Through
1
enactment the social system opens itself up to its environment, while creating a new selfconstructed reality. This self-constructed social environment and reality determine the
possibilities and constraints for further actions. Weick considers technology an integral part
of the organization as a system, and with that an integral part of the process of
sensemaking. But he adds that it is not technology in itself that causes the process of
enactment to change, but rather that the increasing volumes of data produced by
technology are making great demands upon organizations' capacity to deal with that data
and turn it into meaningful information.
In various sectors of our society, decisions are increasingly being based on large volumes
of information produced by interconnected (digital) systems and entities. Examples include
operating rooms, contingency planning or defense. Situational awareness is, put simply by
Endsley, ‘knowing what's going on around you’. Situational awareness can be created and
fostered by selecting the right information and using that selection to provide all parties
involved with unequivocal and meaningful information.
This essay shall first draw on the insights presented by Luhmann and Weick to describe
how the exchange and sharing of information between random systems and entities, and
the ensuing process of assigning meaning to that information, can be framed in a theory.
That theory will subsequently be further explored to see whether it can be used to help
create and develop shared ‘situational awareness’.
INTEROPERABILITY
The role of technology and technological applications in our society is becoming an ever
more crucial one. Within a time span of only a few decades, information technology and the
applications carrying that technology, such as computers, Internet, and cell phones, have
drastically changed our society. Communicating, exchanging and sharing information using
technological applications, interlinked in networks such as the Internet, has become second
nature to whole generations. The number of cell phones keeps growing across the globe,
rapidly making it our main means of communication. Cooper (2005) observes that the
postmodern subject prefers to be continuously connected to his postmodern object - the cell
phone -, taking it with him wherever he goes, wanting to be able to connect to the network
anywhere and anytime, which causes the postmodern subject to lose his footing both
individually and collectively whenever he is disconnected from his technological object.
Cooper accounts for this need to connect or be connected within a human context as a
2
necessary strategy enabling people to present a coherent world into which non-connected
elements are also assimilated. Slowly but surely these new technologies and technological
applications are turning into an integral part of our everyday reality. Clark (2003) claims that
this unquestioning acceptance of technological applications into our day-to-day reality is
determined by the extent to which the use of these technological applications will help us
achieve our goals.
“What matters is that as far our conscious awareness is concerned, the tool itself fades into
the background, becoming transparent in skilled use. In this respect the technology
becomes, to coin a phrase ‘pseudo-neural’.” (2003:45).
New technologies and technological applications are heralded by and emerge from, for
example, nanotechnology or biotechnology, a development that is gaining shape at great
pace across the globe. Technological applications that have been around for longer, such
as RFID chips, are increasingly deployed in, for example, passports, public transport
passes, and in product or book packaging. What these new technological applications have
in common is that they are not only getting smaller, but consequently also getting less
visible and less tangible. Their invisibility and intangibility leads to these applications
surrounding us more and more without us realizing or experiencing it as such. The
development towards networked objects is sometimes referred to as the ‘Internet of
Things’, with every move an object makes in a physical space reflected in the digital space.
The European Union defines the ‘Internet of Things’ as follows:
“The Internet of Things links the objects of the real world with the virtual world, thus
enabling anytime anyplace connectivity for anything and not only for anyone.” (2010:11)
These technological developments will not only lead to all subjects on the planet being
interconnected, but also see objects in those same networks connected through their
unique identification. The hybrid networks, consisting of subjects and objects, that are
arising are making it possible to strike up connections between both subjects on the one
hand and objects on the other, but also between subjects and objects, and that
independently of time and place. The possibility and ability to make connections, and
ensuing possibilities of anytime and anywhere information exchange and sharing, is
triggering an evolution in our perception of reality. Reality as perceived by us is developing
into a more hybrid reality, made up of interlinked subjects and objects that mutually
exchange and share information, and act, produce or create based on that information. This
3
possibility of random subjects and objects being able to exchange and share information
can also be referred to as information interoperability. Interoperability is a linguistic
compound that can mean several things. ‘Inter’ stands for applying mutual links between
systems and entities that have been or will be designated. ‘Operability’ stands for
producing, executing or influencing based on exchanged and shared information. The
mutual relation that thus arises between subjects and objects, and between the physical
and the digital world, requires a new and different approach to the relations between those
phenomena. Verbeek (2005) uses the term ‘post’phenomenology for that new approach:
“From the postphenomenological perspective, reality cannot be reduced to interpretation,
language games or contexts. To do so would amount to affirming the dichotomy between
subject and object, with the weight merely being shoved to the side of the subject. Reality
arises in relations as do the human beings who encounter it.” (2005:113)
Due to the proliferation of connections, the process of organizing, decision making, and
carrying out activities within and between organizations increasingly becomes dependent
on ever larger volumes of information produced by and through networked subjects and
objects. This essay will now continue by describing the development of connections
between systems and entities from the perspective of Luhmann's (1995) systems theory. As
soon as connections between systems and entities have been established successfully,
information can and will be exchanged and shared between objects and subjects in and
between organizations. But incoming information will only acquire meaning for the
organization when meaning is assigned to that information within the receiving organization.
This process of assigning meaning will be discussed from the perspective of Weick's theory
of sensemaking (1979:1995) later in this essay. Finally, this essay will conclude with an
exploration of the influence technology and technological applications, and the information
they generate, have on how people work in organizations.
INTEROPERABILITY AND SYSTEMS THEORY
Luhmann (1995) bases his systems theory on the principle of self-referential and
autonomous systems. He views a system as self-referential when it is capable of forming
elements that function as functional units, and when relations between these functional
units and the system can be perceived as units and relations that were engendered by the
system itself. The system thus continuously reproduces itself through the creation of
functional units and their mutual relations. Luhmann used Maturana & Varela’s concept
4
‘autopoiesis’, made up of the concepts ‘auto’ (self) and ‘poiesis’ (creation or production), to
denote this principle. Luhmann’s theory states that a self-referential system is able to
produce itself, i.e. reproduce, through new elements that stem from the system. Selfproduction of elements enables self-referential and autonomous systems to set up
relations: ‘with themselves and to differentiate these relations from relations with their
environment’.
Double contingency
In order to be able to tackle the issue of how a self-referential, autopoietic and autonomous
system can interact and communicate with one or several systems, Luhmann was forced to
shift the focus of his analysis from: ‘the orientation of a single given actor to the
consideration of two or more interacting actors as a system.’ Luhmann refers to this change
using the theorem ‘double contingency’, which basically means that two random black
boxes are connected through a random event and are looking to harmonize. Each black
box assumes the other black box has the same intentions. Each black box designs its own
behavior through a range of complex and self-referential operations within its limits. The
relation between the systems becomes more effective as the mutual assumptions ensuing
from their system/environment relation increase, and as they become willing to observe
themselves on the basis of these assumptions. The black boxes attempt to influence each
other on the basis of what they register, and can learn from each other on the basis of the
acquired information. Luhmann refers to such a developing form and structure as a social
system.
System and Environment
According to Luhmann, the distinction between system and environment constitutes the
central paradigm of systems theory. And he adds the condition that information is only
really information the moment it is more than an existing distinction between system and
environment: ‘it is information only if it instigates a change of state in the system,’ he states.
And the latter is in his eyes only the case when he, following the example of Bateson
(1972), states: ‘the perception of a difference creates a difference in the system. Something
was not known; then information arrives, namely that these, and none other, are the facts of
matter’. In other words: the difference that is referred to here comes into being when
perception of information actually leads to changes in the perceiving systems. Within the
theory of self-referential systems, the environment is mainly a condition for the identity of
5
the system, because identity is only possible if there are differences. Everything that occurs
is part of a system (or a range of systems) and ‘always at the same time’ comes under the
realm of ‘the environment of other systems’. Every kind of categorization presupposes a
reduction. Every perception, description and conceptualization of a certain category
requires a system reference, within which something can be considered part of a system or
its environment. Every change to the system is a change to the environment of other
systems, every increase in complexity in one area will increase the complexity of the
environment of all other areas.
Communication and action
Communication contains information, according to Luhmann: ‘and thus is enriched with
environmental meaning whenever this information comes from the environment; actions
however are more easily determined as belonging to the system or not’. This means that
every system has to take into consideration other systems in its environment, and every
system depends on the profundity with which the environment can be perceived. If the
system we depart from has the ability to understand this, this system will be able to discern
another system in its environment and distinguish it from the environment they have in
common. The relation between the system and the environment has to be reproduced on a
higher level of system complexity with increased possibilities and restrictions. Luhmann
feels that communication is based too much on the principle of sending and receiving
messages or information between senders and recipients. In his opinion, the metaphor of
sending and receiving positions the essential part of communication within the action of
sending, i.e.: ‘the utterance’ or the communicated message. This focuses too much
attention on, and demands skillfulness of the system that makes the utterance.
Communication is more than just sending and receiving, with selective attention from both
sides. But the selectivity of information is in itself part of the communication process,
because this selective attention is only updated in relation to the great selection of
information that is available to us. The third part of the selection process consists of the
concept ‘understanding’. Luhmann ascertains that the understanding of communication
contains a distinction between the informative value of the content and the reason why this
content is uttered. Either side can be emphasized. The understanding process can focus
more on the information itself or focus on the way the information is expressed. But this
always depends on the fact that both facets are experienced as a selection, and therefore
separated from each other. In other words: one needs to be able to accept that information
6
as such is not understood, but that it requires separate decisions. Luhmann is convinced
that communication transforms the distinction between information and utterance, into one
between acceptance or rejection of the utterance, i.e. a transformation from and into. In his
view, communication is a fully independent, autonomous, self-referential closed way of
making selections that will, however, never lose its specific characteristic as a selection.
Interpenetration
The communicative unit can be rejected or received by the receiving system. When
systems possess a reciprocal willingness and ability to accept the communicative unit, and
grant communicative acts from other systems access to their system, a form of
interpenetration comes about.
“Interpenetrating systems converge in individual elements – that is they use the same ones
– but they give each of them a different selectivity and connectivity, different past and
futures.” (1995:215)
Luhmann (1995) uses the concept of ‘interpenetration’ to pinpoint the special way in which
systems contribute to the shaping of the system within the environment of the system.
Interpenetration is more than just a general relation between system and environment, but
rather an inter-system relation between two systems that make up an environment for each
other, and through which a system makes its own complexity available to build other
systems. Interpenetration therefore only really occurs when these processes are evenly
matched. That is the case when both systems enable each other to introduce their own and
existing complexity to the other side. The concept of interpenetration presupposes
therefore, according to Luhmann, the ability to connect different forms of autopoiesis, such
as life, consciousness and communication. The concept of interpenetration is Luhmann's
answer to the question of how double contingency between different systems is enabled,
and a new system based on communication comes into being with sufficient frequency and
density. Making connections between two or more systems leads to the evolutionary
creation of a new and higher form of system formation. This new system formation consists
of interlinked autonomous and self-referential systems, and is basically a higher form of
interlinked systems that only manifests itself as it comes into being, i.e. as it enters into and
maintains a communicative association. In Luhmann's view, system evolution is only
facilitated by the concept of interpenetration, i.e. in the form of reciprocity. In the systems
theoretical approach, reciprocity turns evolution into a self-perpetuating circular process:
7
“Therefore evolution is possible only by interpenetration, that is only by reciprocity. From
the systems theoretical viewpoint, evolution is a circular process that constitutes itself in
reality.” (1995:216)
Every system that participates in the concept of interpenetration must be willing and able to
allow a difference between system and environment created by another system access to
itself without that leading to the erasing of its own difference between system and
environment. The concept of interpenetration does not connect execution, but shapes
connections every system uses to stabilize its own internal complexity. The difference
adopted by the system is shaped by the unit of communication consisting of a combination
of information, utterance and understanding. Systems want to quickly obtain new and
relevant information from their environment, and be able to adequately apply this
information within their own complexity. New information must therefore be acceptable for
the system, and enable the system to assign meaning to the information. Luhmann (1996)
borrows the neologism ‘sensemaking’ coined by US scientists to refer to this process of
assigning meaning. By assigning meaning to information, i.e. sensemaking, a system is
enabled to perpetuate existing executions, and to pass the ambivalence between knowing
and not knowing on to a subsequent situation. A system benefits internally from new
information based on what a system can or wants to do with this new information.
In today's information society, Luhmann questions the possibility of rational behavior,
instead favoring intelligent behavior. Within the context of the information society, we will
always lack information to be able to come to purely rational decisions. Luhmann states that
within that context, the direct connection between the concepts of information and decision
needs to be reconsidered. Reconsidering this connection is important for the further
development of the information society, because our changing society has to deal with new
and interrelated structuring issues, which, in turn, lean heavily on the available information.
And that is precisely when, in a changing environment, decisions need to be able to revert
to a system memory that helps select what needs to be remembered and what can be
forgotten. This selection process evolves as the information society evolves, because
bridging this gap between virtual and actual information leads to a connection between
globally available information on the one hand, and increasing local and contact-related
production of this information on the other hand. Only the way in which the available
information is retrieved by systems makes information meaningful information. The
information society is hence built on results following from questions that, in terms of
8
structure and operations, are not available anywhere else, and derive or lose content only
through communication.
SENSEMAKING
Karl E. Weick considers an organization a ‘complex system’ (1979:212), a yet to be
identified social entity that strives to achieve several goals through coordinated activities
and relations between subjects and objects. Such an open entity is open ended, and its
survival depends on other subjects and (sub)systems within a greater social entity, such as
a society. The organization as a social entity operates and executes activities using
information as the raw material. How the organization obtains this information and where it
comes from is irrelevant to the organization. In its processed form this information can
come in different shapes, sizes and compositions. Within the organization, the connections
within the process of information processing are introduced in the form of nouns, which in
turn constitute the basis for the processing of the information. This basis is fused with the
interests and activities of the subjects and objects involved, which are also present in and
around the process. Based on this processing process, organizing can be considered as a
set of organizational rules and regulations that together ensure orderly connections
between strings of events and ensuing interactions within the organization. According to
Weick, many of the ‘things’ we see in organizations spring from relations and relational
links, variables that are or will be connected in the processing process in a systematic
manner. Current affairs in organizations are therefore highly dependent on the way in which
connections are established within the process, the direction from which these affairs are
influenced, and the time it takes to move information through the available network of
connections. Weick negates the existence of such a thing as ‘the organization’, and only
sees interconnected events and interactions between subjects and between subjects and
objects within given boundaries.
“Just as the skin is a misleading boundary for marking off where a person ends and the
environment starts, so are the walls of an organization. Events inside organizations and
organisms are locked into causal circuits that extend beyond these artificial boundaries.”
(1979:88).
In Weick's theory, the process of organizing is very similar to the evolution-enabling process
of natural selection (a self-perpetuating process of selection). Likening organizing to the
process of natural selection is Weick's way of modeling and shaping organizing as an
9
activity based on four elements, namely: ‘ecological change’, ‘enactment’, ‘selection’ and
‘retention’.
Ecological change
Changes stemming from connections between the organization as a system and its
environment create what Weick calls a meaningful environment. Incoming changes require
the organization to act in the form of assigning meaning (enactment) by subjects or objects.
Many of the incoming changes are triggered by the interpenetration of information from the
environment. This penetrating information, in turn, constitutes the raw material for a process
of sensemaking within organizations. Weick, like Luhmann, follows Bateson's (1972)
epistemology, which states as follows:
“Ecology, in the widest sense, turns out to be the study of the interaction and survival of
ideas and programs (i.e. differences, complexes of differences etc.) in circuits.” (1972:491).
The concept of enacted environment, where changes from the environment interpenetrate
into the organization as a system, is, in Weick's view, not the same as the concept of a
perceived environment. If a perceived environment were to be the core, this phenomenon
would have been called ‘enthinkment’ and not ‘enactment’ (the act of assigning meaning).
Weick considers reality a product resulting from an active process of social construction,
and sees the concept of ‘enactment’ as the starting point of that process. Weick joins
Berger and Luckmann (1966) in stating that observations of our environment from different
viewpoints does not lead to everyone observing a common world in the same way.
Similarities in our perception of this common world are caused by the fact that we use
language as a common system. Berger and Luckmann point out that man uses language to
construe his social reality. The concept of an ecological environment and the ensuing social
construction of reality is based on the fact that knowledge is developed through connections
between subjects and between subjects and objects. The subject observes the object, and
that observation is subsequently processed cognitively by the subject, labeled in different
ways, and linked to various other isolated or external events. Weick states that there is too
little focus on the possibility that the development of knowledge can also move into a
seemingly opposite direction, namely the potential effect of the subject on the object. The
potential effect of the subject on the object makes knowledge development into an activity
where the subject, partly through his own interaction, establishes the object both within his
environment and within existing relations in that environment. That means, in Weick's
10
theory, that we can assume that there is a reciprocal relation between subject and object.
Mutual effects between subject and object are what Weick sees as a model for the relation
between enactment and ecological change. Weick mainly identifies this kind of reciprocal
influence in organizations with activities that lean heavily on technology and technological
applications. These have to shape enactment around and while taking account of the
(im)possibilities of the technology. The high level of entanglement with technology and
technological applications causes the process of enactment at organizations to change. But
arguing that enactment reduces when the intensity of technology use increases goes too
far, in Weick's view. Weick claims that that would ignore the fact that it is not technology in
itself that engenders these changes, but rather that it is the information produced by that
technology and the information that is increasingly fashioned and processed using and
through technological applications that drives changes. Technology generates ever greater
volumes of raw data, which is a development that is also making ever greater demands on
organizations to assimilate this raw data into their own context, in such a way that this data
can be turned into usable and manageable information (bracketing).
Enactment
Weick compares the term enactment, i.e. the act of assigning meaning, when used in the
context of organizing, to the relation that evolution theory established between the term
variation (the existence of differences within a kind) and natural selection. He prefers
enactment over variation, as enactment has a more active connotation, and better reflects
the active role people in organizations play in creating their environment, while also being
willing to impose the created environment on themselves. As stated earlier, the act of
assigning meaning is closely linked to the principle of ecological change. Especially where
new differences arise within existing knowledge and experience in the organization, such as
through the arrival of new information from the environment, this requires action from one or
several actors to isolate and further scrutinize this new difference in order to eventually
assign meaning to it. This kind of bracketing of new differences is merely one manifestation
of enactment. Another manifestation of enactment can, for example, come about when an
actor does something that leads to a new ecological change, i.e. a change that
subsequently limits his scope of actions in the environment, which, in turn, reproduces a
next ecological change, making this an endless sequence.
11
The process of assigning meaning is the only process through which the organism or the
organization approaches its external environment. The perspective of the ability to assign
meaning gives people in organizations greater self-confidence and more willingness to
reflect on their day-to-day activities, on how they influence their environment, and on how
their environment influences them. On the other hand, the organization should be more
concerned about, and have greater awareness of, the environment and the ensuing
influence on the reality constructed by the organization, and with that on the (un)intended
(side-)effects on organizational processes and the ensuing information developed and
shaped by organizations. If man and organization were to be more aware of the fact that
they construct their own environment and hence their own reality, they would be able to
influence that process more. When environments are approached from the perspective of
active meaning assignation, the focus shifts from the question of what's true and what's not,
to the question whether the presented or conceived version of reality is more reasonable or
less reasonable. That would prevent endless discussions and questions aimed at showing
whether things are perceived and judged correctly and whether they are true or not. From
the perspective of assigning meaning, such discussions can, in Weick's view, be replaced
by questions along the lines of: what have we done? what meaning can we, and do we
want to, assign to certain actions and information? and which actions did we refrain from?
This way, people are, on an individual level, challenged to analyze whether the meaning
they assigned to changes in their environment has led to the right form of common meaning
or sensemaking for that change. As pointed out earlier, an organization can be considered
as a network consisting of random subjects and objects that mutually exchange and share
meaning supported by the development and use of a common language and everyday
social interaction. Exchanging and sharing meaning enables people in organizations to
make sense of changes that come in from the outside. That makes sensemaking in
organizations a social process that is a breeding ground for interpenetrations and
interpretations. The process of sensemaking is not something that starts somewhere,
because sensemaking never stops. Subjects are always in the middle of something that
subsequently turns into something else, and can only be perceived when focusing on the
past of that something, from a point where they have just passed that something.
Selection
In Weick's theory, selection is the part of the process of natural selection that is least
understood. In his view, selection in an organizational context implies imposing different
12
kinds of structures on the result of sensemaking actions in organizations. These structures
can be used to explain and apply meanings and the way these are shaped from actions of
subjects, as well as to cut down on the possible number of multiple interpretations of the
assigned meaning. Where the subject is concerned, structures imposed on the outcome of
sensemaking emerge from what Weick calls cause maps:
“Beliefs are cause maps that people impose on the world after which they see what they
have already imposed.” (1979:135)
Weick defines individual cause maps as insights compiled by the subject based on earlier
acquired and applied knowledge that determines how the subject interprets observations
and approaches the outside world. The subject bases individual cause maps on a collection
of experiences and agreements that he has accumulated in the past by being part of an
organization, and which are laid down in the technical, social and cultural system of the
organization, as one of the many different meanings assigned to reality. Selection is hence
the organizational process that generates answers to questions such as: ‘what is going on
here?’ According to Weick, social systems use two forms of selection, namely criteria
relevant to internal functioning, and criteria relevant to the external functioning of the
system in its environment. The environment of the social system is here one of the crucial
principles that should, from the point of view of the subject, sooner be considered output
than input. Based on sensemaking and interpretations of assigned meanings, people
construct a common picture of the environment in which actions are produced. The
selected environment is, however, not necessarily the environment that set off the act of
assigning meaning, but can subsequently still serve as an explanation of why and how
applied actions and meanings came about. Selection thus determines what people do the
moment meaning is assigned, for example when using and applying technology, when they
improvise, develop and implement alternatives, and in the interpretation of incoming
information. When seemingly strange actions lead to adaptation to changing circumstances
relatively easily, certain selections will start to become a permanent and frequent part of the
selection process. Removing plurality from meaning is therefore essentially both a solitary
and a social process. In Weick's words:
“Sensemaking is largely solitary in the sense that structures contained within individual
minds are imposed on streams of individual elapsed experience that are capable of an
infinite number of individual reconstructions.” (1979:142).
13
A complicating factor in the removal of plurality is that interconnected subjects and objects
produce and process pieces of information within the organization at any given moment. All
these pieces are in different stages of adaptation and processing within the process of
organizing. Different pieces of information can impact on each other in that the
interpretation of one piece of information can influence the meaning of another piece of
information. Potentially interacting information takes up system capacity for plurality
reduction, meaning that there is no capacity - providing it is used in the process - available
for other parts of the process. That brings about many different cycles, rules, cause maps
and convictions that each influence the present streams of events. Each identifiable part of
the process of organizing is intensively conditioned by the context or the subject this
specific part of the process is embedded into and takes place in. This context of the
process of organizing is, in turn, influenced by the input taken up in the specific process of
organizing, as well as the available cycles and interpretations that are, or are not,
acceptable at that specific point in time. The frequency with which ecological changes
occur, the extent of these changes, and the way these changes are instigated by the act of
assigning meaning, are created by the existing context. At the same time, the different parts
of the process of organizing also play a crucial role in their respective stages of
development. The existing context and the stage of development both, in turn, also play a
role in the limitation of enactment.
Retention
Retention constitutes relatively straightforward storage of products that have successfully
come out of the sensemaking process. These products have come through the selection,
and can as a result be called enacted environments. An enacted environment is therefore a
detailed and connected summary of a previously plural reflection of assigned meaning. It is
an enacted version of that which the plurality was based on, and from which other versions
could also have been constructed. Weick uses concepts such as enacted environment or
cause maps to denote retained meaningful environments to be able to refer to meaning
incorporated into and assigned to the products at an earlier stage. The concept of enacted
environment is used to stress that it is all about the result of the process of organizing and
not about the input for that process. The cause map concept is used to stress that retained
and enacted products are produced by a process that was shaped previously. Cause maps
contain the variables used by the individual to establish the causal links that have resulted
in the meaning. Eden (1992) argues that individual cause maps can be analyzed and
14
described using cognitive schemata that reflect connections between variables in the
organization (actors and the meaning they assign) and the way in which these variables
have been and are introduced by the individual:
“Cognitive maps can be seen as a picture or visual aid in comprehending the mapper’s
understanding of particular and selective elements of the thoughts (rather than thinking) of
an individual, group or organization.” (1992:262)
Such an inner reflection of supposed reality emanating from the individual is potentially
applied to every act of sensemaking, as occurred previously. Analysis and reflection of this
reality enables the questioning or adaptation of the applied selections.
SITUATIONAL AWARENESS
Technology and technological applications are, in Weick's view, an ever more essential part
of organizations, and it is key that they are covered in all discussions about sensemaking at
organizations. In particular the increasingly close relation between IT and sensemaking in
organizations is something Weick considers a fundamental new part of the process of
sensemaking. In his words:
“What is emerging as a growing issue for sensemaking is the disparity between the speed
and complexity of information technology and the ability of humans to comprehend the
outputs of the technology.” (1995:177).
His general point of concern is here mainly caused by the fact that the development and
application of information technology in organizations is driven by a decision-making
rationality and not by an action or narrative rationality. In this context, Weick considers
technology as an ecological change or an enacted environment for members of the
organization. Under exceptional circumstances, such as during disasters or in times of
crisis, the chance of the process of assigning meaning faltering increases due to the huge
volume and complexity of interpenetrating information from the environment, which can in
turn lead to the situation getting out of hand even further. As stated earlier, the act is the
central part of the process of assigning meaning because, also in exceptional situations
such as disasters and crises, subjects thus assign meaning to incoming information, or as
Weick puts it:
15
“that when people act, they bring events and structures into existence and set them in
motion. People who act in organizations often produce structures, constraints, and
opportunities that were not there before they took action.” (1988:306).
Assigning meaning is therefore both a process, i.e. the act of assigning meaning, and a
product, i.e. a new environment that has been given meaning. The environment the
meaning has been incorporated into is both of a public and a private nature. In its public
manifestation it is a construction that is visible to other actors, while its private form is that
of a list of if-then assumptions, within which acts or actions are connected to their
consequences.
“In any crisis situation there is a high probability that false hypotheses will develop and
persist. It is largely through open exchange of messages, independent verification, and
redundancy that the existence of false hypotheses can be detected.” (1990:583).
If under exceptional circumstances, such as a crisis, a diversity of information is distributed
among a large number of parties, each with their own interpretation of what is happening,
then a great effort will eventually be required to bring all these different interpretations back
into line and prevent differences and lack of clarity between the parties. If the parties
involved are unable to align the different ideas of what is happening while also agreeing on
what should be done next, the effectiveness of any actions will be undermined. In our
information society, the success of operations in times of crisis, and sometimes even
surviving such situations, increasingly depends on an ability to select, take in and
understand large volumes of data quickly, according to Endsley (2001). The data is to
provide information about the circumstances in the (external) environment in which the
activities are set, the status of activities on site, and the condition of the resources needed
to carry out these activities. This requires timely and adequate transformation of the
available data into the right information that the subjects involved can assign meaning to.
All available data and information is processed and absorbed by subjects and objects in a
process of sensemaking, as described earlier. Subjects and objects each have an individual
task in executing their activities, and therefore a dynamic and changing information need
ensuing from that individual task. All subjects and objects involved must be able to interpret
and understand information, available for the execution of activities, within the context of,
for example, a common mission (such as a pilot, co-pilot, air traffic control). The problem of
being able to understand and process a continuously growing amount of information is,
16
however, a major one, and becoming even greater. In Endsley's view, this not only goes for
pilots, but also for professionals with desk-based activities. Supporting the growing
information needs of all interlinked parties, while also launching a coordinated effort to fulfill
those needs, in order to ensure everyone can avail of adequate and unequivocal
information at the same time, is an ever greater challenge. Endsley et al. (2001) refer to this
collectivity and simultaneity in having the same information as 'situational awareness',
which in their view basically boils down to knowing what is happening around you and what
individuals need to know to be able to carry out their activities:
“Situation awareness (SA) can be thought of as an internalized mental model of the current
state of the operators’ environment. All of the incoming data from the many systems, the
outside environment, fellow crew members and others must be all brought together into an
integrated whole. This integrated picture forms the central organizing feature from which all
decision making and action takes place.” (2001:3).
Even though Endsley states that the elements that make up situational awareness differ
from one domain to the next, the mechanisms through which situational awareness can be
achieved can still be described in generic terms. Situational awareness is, in his view,
essentially based on a reflection of the situation in the environment as constructed by the
subject himself. In order to be able to construct his individual image of reality, the subject
will gather and compile information from sensory input, i.e. visual, auditive, tactile, olfactory
and taste input. Information that is gathered and compiled from that input is then used to
make decisions and determine what actions are required. Some of the available information
shall be openly perceptible, while other information is only present on a subconscious level.
The combination of information processed by the different senses, which is generally
perceived as normal, throws up new issues when we have to increasingly go by the
assumption that functionaries interlinked in networks operate remotely and are able to
exchange and share information within their networks independent of place and time.
Endsley therefore considers it a huge challenge to also provide these functionaries with
sufficient and adequate information using a physically remote interface that is intended to
compensate for the absence of directly perceived sensory information. A crucial point here
is that true situational awareness can only exist in the brain of the subject, which is why
presenting ever greater volumes of data for absorption, interpretation and quick conversion
into meaningful information by the subject will, in Endsley's view, not be the solution. He
states that successful improvement and development of situational awareness can only
17
come about when technology and technological applications that present information are
developed and shaped based on clearly formulated requirements from the domain and the
professional himself. Furthermore, clear description is needed of which system and
environmental circumstances can influence information from the environment, and with that
the situational awareness. The design process of situational awareness will have to be
shaped from a system-driven approach. An air traffic controller and a motorist, but a doctor
or an army commander too, operate in different domains and under different circumstances,
but all need compounded information. That information is needed to acquire a situational
awareness specific to their situation, which will help them be more effective in carrying out
their activities. Further research into how large volumes of diverse data can be combined
into new and meaningful information for specific activities or work will, according to Endsley,
boost the development of situational awareness. Instead of overloading an operator or
decision maker with hundreds of pieces of randomly categorized data, clearly defined
requirements for situational awareness can help compress data flows into information that
is needed to reflect and understand information in specific situations. The importance of the
creation of situational awareness in different domains and under different circumstances is
corroborated by the following examples:
“Command & Control at the battalion level
“The Operating Room is a critical and
is performed by a team consisting of
complex work environment consisting of
groups of experts and staff from a variety
people, devices and tools, actions and
of domains including leadership and
events. Effective teamwork in the OR is
management, military science, logistics,
critical to patient safety because health
enemy intelligence, field artillery (fire
workers interact with each other in order to
support) and telecommunications. They
achieve successfully and safely a common
explore and integrate their specialized
goal. To accomplish the common goal, all
knowledge to create plans that will
members of the surgery team must perform
accomplish the intent of the battle mission
their roles and tasks with full and continuous
and are achievable within the constraints of
comprehension and awareness of the
the situation. Throughout these activities
dynamic situation. To do this they rely on
they communicate and share information,
continuous communication, and information
ideally, developing a shared understanding
sharing with their team mates. Accurate
of the mission and battle and working in a
information sharing among people and
coordinated fashion to achieve the
technology is essential to effectively
18
Both examples highlight that apart from the individual need for situational awareness, the
need to provide several individuals with the same information and meaning at the same
time also caused complexity to increase. Salas (1995) points out that there is more to
working in groups or teams than simply adding up the situational awareness of the
individual team members. The greater complexity Salas lays bare is, in his view, down to,
among other things, the unique activities that working in a team brings with it, such as
coordination and information exchange and sharing.
Case 1: Command & Control
Sonnewald and Pierce (2000) state that a team must be capable of gathering information
from a range of different sources, in order to subsequently examine, combine, and convert
the available information into plans using their specialist knowledge, and finally see to the
execution of those plans. It is impossible for one individual within a dynamic changing
environment, such as a battlefield, to obtain and process all the information needed to
develop and execute an effective plan. During planning preparations, individual experts
therefore need to join forces with others in finding, combining and spreading the available
information within the entire process of command and control. The experts involved in this
process can hail from different disciplines, different branches of the armed forces, and even
from different countries. Every individual that takes part in the command and control
process will possess specialist knowledge and experience in his specific domain. Each
individual within the team therefore develops specialist and unique situational awareness,
comprising a combination of individual knowledge of and skills for the specific situation. As
a result, these individuals are enabled to support the efforts relating to their domain. In
addition to individually developed situational awareness, experts also need to develop and
maintain intra-group or shared situational awareness across all team members. Intra-group
or shared situational awareness is realized through a network of interwoven individual
meanings, each stemming from individual, intra-group, and inter-group levels and jointly
capable of constituting the required knowledge of the current situation. Collaboration in the
group can hit a snag when team members use different (specialist) jargon and terminology,
aim for different organizational targets, set different priorities, and have different past
experiences in their respective fields. In these environments, the use of information
technology is on the rise, with technology taking up an increasingly vital role in the efficient
and effective execution of activities. Each team member uses IT on an ever larger scale, as
well as technological applications based on IT, which is leading to man-computer,
19
computer-computer and computer-man interfaces growing rapidly in terms of size and
complexity, in their role of links between random subjects and objects within networks. The
development of these random combinations incurs further questions about how, for
example, an integrated idea of the current situation can be compiled from these
combinations. In order to be able to develop an integrated idea, certain questions will need
answering, such as whether and how participating subjects and objects can provide realtime information, whether they can vouch for the information provided, and whether the
right decisions can be made based on the information compiled, which leads Toner (2009)
to note:
“At this point it appears that there is lots of technology, but relatively little science.” (2009:3).
Stanton et al. (2006) shift the attention in the development of ‘situational awareness’ in
distributed teams (spread across time and geographical locations), consisting of interlinked
subjects and objects, to the development of a situational awareness at system level. In
such a system, or a system of systems, they see a developing cognitive process at system
or network level, rather than on an individual level. All elements needed for distributed
situational awareness are on hand in the system or network, while specific knowledge
about the situation is compiled from separately operating subjects and objects, and from the
knowledge and information exchanged and shared between these subjects and objects.
The mutual exchange and sharing of information between different subjects and objects
within these systems or networks will only take place on a limited scale, and certainly not on
the basis of detailed individual cause maps. Distributed situational awareness is therefore
not the same as shared situational awareness, as the latter presupposes the existence of
different, but still mutually exchangeable, requirements and goals.
“Therefore our approach assumes that distributed situational awareness can be defined as
activated knowledge for a specific task within a system.” (2006:1291).
In the theory of Stanton et al., the concept of activated knowledge enables the procurement
of knowledge, both of the environment in which the situation is set and of the latest changes
to the situation, from a random collection of subjects and objects for any possible situation
that requires specific knowledge. The idea that a network can consist of a combination of
knowledge objects that together form a system and ensure situational awareness at system
level is one that Stanton et al. see leading to completely new insights on shared awareness.
First and foremost that the system as a whole, and not each individual, has all the required
20
information, that each individual is presented with a reflection of reality that suffices in
enabling him to execute his specific activities, and that the development of that shared
awareness is not conditional on there being communication between individuals. What's
important for the separate units in the network is to know who has the specific information
that can be of importance for an adequate and shared idea of the situation, and whether
that shared idea of the situation can lead to the provision of usable information to other
units in the network. The realization of distributed situational awareness makes connections
between random subjects and objects crucial because communication between the
separate elements of the network is not possible without connections.
Case 2: Operating Room
The importance of developing situational awareness in, for example, an operating room is
described by McIlvaine (2007). He claims situational awareness is key in such an
environment, because there too, albeit in one room, more and more subjects and objects
are interconnected and depending on each other for the execution of their tasks. A first step
towards situational awareness is to regard all subjects and objects present in this space in
terms of their mutual relations. Within these mutual relations, a key role is played by the
data flows from electronic monitors and related comments and warnings by team members.
The development of situational awareness in this environment requires, in the view of
McIlvaine, care and continuous attention because situational awareness is an active
construction here as well, both on an individual level and on team level:
“There is an active flow of information from the environment to the individual and then
between the individuals who constitute the team. Each member has a responsibility to the
patient and to other team members to keep.” (2007:169).
Responsibility and commitment to the patient and other team members increase as
operations take longer and team members rotate, both inside and outside the operating
room. Continued understanding of the current situation on both an individual level and on
team level is conditional on the available data and information, but the reliability of that data
and information also depends on the weakest link in this context. The importance of
continued understanding of what is going on in the operating room is further underlined by
Parus et al. (2010), who state that situational awareness in a complex work environment,
such as an operating room, depends on mutual communication and the exchange and
sharing of information between all members of the team. The ability to efficiently and
21
effectively exchange and share information between man and technological applications is
essential in order to be able to properly execute and coordinate interdependent activities
and tasks. In this environment too, there is a need to combine and collectively present the
increasing amount of data and information coming from different sources, so that all team
members can avail of a continuous stream of joint and shared information. The key to being
able to develop situational awareness is again the knowledge of what the team members
need to know at what moment and what data and information are critical for effective and
safe collaboration in a team.
CONCLUSIONS
Biologist Ludwig von Bertalanffy is considered one of the originators of systems theory, and
he observed:
“System theorists agree that the concept of ‘system’ is not limited to material entities, but
can be applied to any ‘whole’ consisting of interacting ‘components’.” (1969:106).
Arguing from the perspective of Von Bertalanffy, there is no distinction between social,
biological and/or technological systems. He sees every system as a whole made up of
interconnected and interacting components. This essay has established that new
connections between man, organization and technology are cropping up all the time.
Connections that, in turn, form the basis for the ability to exchange and share information
between subjects and objects in random coalitions and networks. Based on these
connections, and on the information exchanged and shared between subjects and objects,
new systems emerge. Existing systems either evolve or cannot cope with changes and
eventually disappear. The possibility of exchanging and sharing information between
random systems and entities breeds new forms of interaction between different systems
and entities, forms of interaction that are not bound by time and place. These new forms of
interaction will inevitably and without us noticing cause our constructions and perception of
reality to evolve and change into a new form of reality. This new form of reality is then
partially defined by invisible and omnipresent forms of technology and technological
applications, which will adapt our perception of reality to the coalitions and networks that we
are part of. Both Luhmann and Weick argue their respective theories from the system as
the basis and from the interlinked and interacting components in the system. Luhmann
does so more from a sociological/administration sciences perspective, while Weick
22
approaches systems from a social psychological/organizational sciences perspective. Such
differences in scientific approaches led Von Bertalanffy to conclude:
“All scientific constructs are models representing certain aspects of perspectives of reality.”
(1969:94)
Both Luhmann and Weick depart from the assumption that reality, and with that knowledge,
is constructed from and through communication and interaction between systems. It is
interesting to take on an interdisciplinary and holistic approach and look for the synergy that
can result from the - in essence complementary - theories of Luhmann and Weick. Where
Luhmann focuses on the process of communication between systems, Weick's focus is on
the process of how a receiving system makes sense of the information interprenetrating into
the system. They both assume the perspective of ecological change and development, and
lean heavily on Bateson's epistemology. Within this process of evolution and ecological
change in society and organizations, the increasing use and application of technology and
technological applications plays a key role. And with that the generation, adaptation and
processing of information. As a result of the widespread application and use of technology
and technological applications, the process of evolution and ecological change will lead to a
continuously changing reality, and at the same time to changes in our thinking about
organizing and organizations. New forms of organizations will have to be developed and
shaped from new combinations of organization and technology in order to make the most of
the growing information flow between and within organizations. Fifty years back, a similar
combination of social and technological developments, such as nuclear energy, space
travel and rapidly developing computer mainframes, were the basis for the development of
general systems theory:
“Modern technology and society have become so complex that traditional ways and means
are not sufficient anymore, but approaches of a holistic or systems, and generalist or interdisciplinary nature became necessary.” (1969:xx).
The development of more and more anytime and anywhere connections between systems
and entities, and the corresponding proliferation of possibilities for information exchange
and sharing, requires new and more holistic approaches, consisting of new combinations of
man, organization and technology using that information. New technological opportunities
and technological applications are generating an increasing amount of information within
and between organizations. That information has to be made sense of. New meanings
23
coming out of these combinations in turn lead to new insights. As outlined here, the
proliferation of information in many areas is throwing up new questions about how
meaningful information can be created from this growing mountain of data, and how this
meaningful information can be captured in new interfaces between man and machine.
Meaningful information is a prerequisite for the creation of situational awareness in those
functionaries who rely on that information in being able to do their work properly.
24
REFERENCES
Allard-Poesi F. (2005) The paradox of sensemaking in organizational analysis. Organization,
12, 169 - 196.
Bateson G. (1972) Steps to an ecology of mind. Chicago. The University of Chicago Press.
ISBN 0226039064
Beringer D.B. and Hancock P.A. (1989) Exploring situational awareness: A review and the
effects of stress on rectilinear normalization. Proceedings of the fifth international
symposium on aviation psychology. Columbus Ohio University
Berger P. and Luckmann Th. (1966) The social construction of reality. A treatise in the
sociology of knowledge. London. Penguin Books. ISBN 9780140135480
Bertalanffy v. L. General Systems Theory (1969) revised edition. New York. George Braziller.
ISBN 0807604534
Clark A. (2003) Natural-born cyborgs. Minds, technologies and the future of human
intelligence. Oxford. Oxford University Press. ISBN 9780195177510
Cooper R. (2005) Peripheral Vision: Relationality. Organization Studies, 26, 1689 - 1710.
Cooper R. (2006) Making present: Autopoiesis as human production. Organization, 13, 59 81.
Cooper R. (2007) Organs of Process: Rethinking human organization. Organization Studies,
28, 1547 - 1573.
Donnenwald D.H. and Pierce L.G. (2000) Information behavior in dynamic group work
contexts: interwoven situational awareness, dense social networks and contested
collaboration in command and control. Information Processing & Management, 36, 461479.
Endsley M.R. (2001) Designing for situation awareness in complex systems. Second
international workshop on symbiosis of humans artifacts and environment. Kyoto,
Japan.
Endsley M.R. (2000) Theoretical underpinnings of situation awareness. in Endsley M.R. and
Garland D.J. (Ed.) Situation Awareness analysis and measurement Mahwah, NJ,
Lawrence erlbaum Associates.
European Commission, Information Society and media (2010). CERP-IOT Cluster of
European research project on the internet of things. Vision and Challenges for realizing
the internet of things March 2010 Edited by Sundmaeker H., Guillemin P., Friess P.,
Woelffle S.
25
Heidegger M. (1954) The question concerning technology. in Lovvit, W. (Ed.) The question
concerning technologys and other essays. 1977 ed., Harper Torchbooks.
Heidegger M. (1998) Zijn en Tijd, Nijmegen, SUN. Dutch edition. ISBN 906168675x
Ihde D. (1967) Some parallels between analysis and phenomenology. Philosophy and
phenomenological research, 27, 577-586.
Ihde D. (2000) Technoscience and the 'other' continental philosophy. Continental philosophy
review, 33, 59 - 74.
Ihde D. (2002) Bodies in technology, Minneapolis. University of Minnesota press. ISBN
0816638454
Ihde D. (2009) Postphenomenology and technoscience The Peking University Lectures. New
York. State University of New York. ISBN 9781438426211
Ihde D. and Amato I. (2000) Timeline Travails, Source Science, new series, 287, 803.
Ihde D. and Selinger E. (2003) Chasing technoscience. Indiana University Press. ISBN
0253342538
Lier van B. (2009) Luhmann meets ‘the Matrix’ Uitwisselen en delen van informatie in
netcentrische omgevingen. Eburon, Delft. ISBN 9789059723085
Lubitz V. D. and Wickramasinghe N. (2006) Networkcentric healthcare and bioinformatics:
Unified operations within three domains of knowledge. Expert systems with
applications, 30, 11 - 23.
Luhmann N. (1983) Insistence on systems theory: Perspectives from Germany - an Essay.
Social Forces, 61, 987-998.
Luhmann N. (1986) The autopoieis of social systems. Journal of Sociocybernetics, 6, 84-95.
Luhmann N. (1986) Autopoiesis: What is Communication. Communication Theory.
Luhmann N. (1992) The concept of society. Thesis eleven,, 31, 67 -80.
Luhmann N. (1995) Social Systems. Stanford. Stanford University Press. ISBN 0804726256
Luhmann N. (1996) On the scientific context of the concept of communication. Social Science
Information, 35, 257 - 267.
Luhmann N. (1997) The control of intransparency Systems Research and Behavioral
Science,, 14, 359 - 371.
Luhmann N. (2006) Systems as difference. Organization,13, 37 - 57.
Magala S. (1997) Karl E. Weick: Sensemaking in organizations - Book review essay.
Organization studies,18, 317-338.
Mathews M.D & Pleban R.J. and Endsly M.R. & Strater L.D. (2000) Mesaure of infantry
situation awareness for a virtual mout environment. Proceedings of the human
26
performance, Situation awareness and automation: User centered design for the new
millennium conference.
Mcilvaine W.B. (2007) Situational awareness in the operating room: a primer for the
anesthesiologist. Seminars in Anesthesia, Perioperative medicine and Pain, 26, 167172.
Parush C.S. (2010) Communication and team situation awareness in the OR: Implications for
augmnetive information display. Journal of Biomedical Informatics, 1-9.
Salas E. Prince C. Bakker D.P. & Shrestha L. (1995) Situation Awareness in team
performance: Implications for measurement and training Human Factors, 37, 123-136.
Sonnenwald D.H. and Pierce L.G. (2000) Information behavior in dynamic group work
contexts: interwoven situational awareness, dense social networks and contested
collaboration in command and control. Information processing and Management, 36,
461-479.
Stanton N.A. C.S. (2006) Distributed situation awareness in dynamics systems: theoretical
development and apllication of an ergonomic methodology. Ergonomics, 49, 12881311.
Toner E.S. (2009) Creating situational awareness: A systems approach. workshop on medical
surge capacity Institute of medicine forum on medical and public health preparedness
for catastrophic events.
Verbeek P.P. (2005) What things do: Philosophical reflections on technology, agency and
design. Pennsylvania. The Pennsylvania state university press. ISBN 9780271025407
Weick K. E. (1988) Enacted Sensemaking in Crisis situations. Journal of Management studies,
25, 305 - 317.
Weick K. E. (1990) The vulnerable system: an analysis of the Tenerife air disaster. Journal of
Management, 16, 571 - 593.
Weick K. E. (1995) Sensemaking in organizations. Thousand Oaks. Sage Publications. ISBN
0803971761
Weick K. E. (2002) Real-time reflexivity: Prods to reflection Organization Studies, 23, 893 898.
Weick K.E. (1979) The social psychology of organizing. New York. McGraw-Hill Inc. ISBN
0075548089
Weick K.E. (1988) Enacted sensemaking in crisis situations. Journal of management studies
25, 305-317.
27
Weick K.E. (1995) What theory is not, theorizing is. Administrative Science Quarterly 40,
3385-390.
Weick K.E. Sutcliffe K.M. and Obstfeld D. (2005) Organizing and the process of sensemaking.
Organization Science, 16, 409 - 421.
Wickens C.D. (2008) Situation Awareness: review of Mica Endsley's 1995 articles on situation
awareness theory and measurement. Human Factors: The journal of the Human
Factors and Ergonomics Society, 50, 397-403.
28
Download