Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
Visualisation of Knowledge Mapping for Information Systems Evaluation:
A Manufacturing Context
Amir Sharif
Brunel University
Brunel Business School
amir.sharif@brunel.ac.uk
Muhammad M. Kamal
Brunel University
Brunel Business School
muhammad.kamal@brunel.ac.uk
Abstract
Information Systems (IS) are primary enablers in
instigating organisational change, increasing
organisational responsiveness and decreasing supply
chain overheads. This paper aims to contribute
through exploring and visualising Knowledge
Mapping (KM) from the perspective of Information
Systems Investment Evaluation (ISIE). Complexity of
IS evaluation process increases with the increase in
intricacy of IS. A number of interrelating factors (e.g.
costs, benefits and risks) contribute towards the
intricacy of IS evaluation. There seems to be a
growing need to evaluate the IS investment decisionmaking processes, to better understand the often far
reaching
repercussions
related
with
IS
implementation and interconnected Knowledge
Components (KC). In seeking to edify the often vague
IS evaluation process, this paper attempts to,
emphasise the increase of knowledge and learning
through the application of a Fuzzy Cognitive
Mapping (FCM) technique. The resulting FCM
determines the intricate and developing behaviour of
causal relationships within the knowledge
management area.
1. Introduction
In today’s global economic environment, the
world-wide business organisational ambience has
significantly transformed which contributed to
today’s competition in many sectors of industry,
commerce and government [1]. In such a hypercompetitive state, organisations need to be aware that
their operations would abruptly cease to function
should the technology that underpins their activities
and help to automate organisational workflows, ever
come to a halt. Thus, business organisations need to
constantly explore state-of-the-art ways to reorchestrate their products and delivery of services. In
recent years, enterprise IS have played a significant
role in supporting organisational agility, dealing with
improbability in decision-making, supplementing
their competitiveness, and coordinating information
in the supply chain [22, 46]. According to Stockdale
and Standing [44] a substantial growth in enterprise
IS investments has enforced a number of business
organisations to focus on the usefulness and
assessment of their business processes and
approaches. The latter argument is supported by
Sharif et al., [38] who state that in essence, enterprise
Zahir Irani
Brunel University
Brunel Business School
zahir.irani@brunel.ac.uk
IS evaluation is a decision-making process, which
enables an organisation in defining costs, benefits,
risks and more importantly, in highlighting the
repercussions of investing in IS. According to
Hedman and Borell [10] the evaluation of enterprise
IS is essentially based upon knowledge of the
organisation and strategic, tactical and operational
prerequisites.
Such
enterprise
IS
support
organisations in capturing and storing knowledge of
human experts and then imitating human reasoning
and decision-making in the design, production and
delivery of manufactured goods [22, 44].
Regardless of using any approach in an
organisation, the evaluation process aims to ascertain
a link between the anticipated value of an investment
and analysis [often quantitative] of the costs, benefits
and risks [13]. To address the requisite for an
organised evaluation instrument for supporting the
top management in better understanding the human,
organisational and technical repercussions of their
investment decisions, academics and practitioners
have approached investment decision-making from a
variety of perspectives. For example, Expert Systems
(ES) perform a number of tasks that are carried out
by humans with certain knowledge and experience.
Assessing performance necessitates an understanding
of human expert performance and in the way it can
be assessed. The knowledge and investigational
learning that is a prerequisite within a decisionmaking process, is thus essential to the ending result.
Kim et al., [20] points here that sharing and
managing knowledge in all its forms requires being
stable and controlled to make the most of its effect.
The motivation for this paper is to attempt to map
out and visualise the range and aspects of knowledge
that are relevant to the ISIE process in the
manufacturing context, based upon the extant
literature and managerial, operational, organisational,
technological and strategic aspects of an
organisation’s strategy. As such, the motivation for
this research rests on understanding what aspects of
the relevant expert knowledge ultimately drive this
knowledge-intensive evaluation task, thereby
highlighting some of the dynamic inter-relationships
inherent within the field and in a practical context.
This paper therefore seeks to investigate and map
factors influencing the decision-making process for
ISIE and their pertinent KC. The mapping of ISIE
factors and related KCs is achieved by using a FCM
technique, resulting in exploring the interrelationships and intricacies of decision-making
1
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
factors in a manufacturing context. FCM was
selected to explore the cognitive perspective of ISIE
factors – the focus is to examine the link based
context, not the agent based context. Moreover, the
authors are not limiting or neglecting the usefulness
of other techniques, instead we are seeking to apply
FCM as it has limited application to the ISIE related
context as compared to agent based modelling.
2. Research Design
A robust research design (based on four steps)
was constructed and proposed, which acted as a
blueprint for the research process. Each step in this
process acts as a basis for the next step:
 Step 1 is about investigating factors that define
ISIE in the manufacturing sector. This was
achieved by studying the extant general IS and
manufacturing literature in-depth. This research
exercise facilitated the authors in understanding
of ISIE practices in manufacturing organisations
and as a result, supported the identification and
defining of 15 influential factors. These factors
are classified according to the proposed
‘MOOTS’
dimensions
–
Managerial,
Organisational, Operational, Technological, and
Strategic.
 Step 2 is about identifying and correlating KC
with the relevant ISIE factor. These KCs are
identified using the proposed five-step Pairwise
IS Theory Equivalence (PIE) framework (Figure
1). The proposed PIE process is further divided
into five sub-steps. For example, for each ISIE
factor a supposition is developed, then, two
relevant IS theories are identified for each ISIE
factor – this allowed more flexibility in
extracting a relevant KC. Then a rationale is
developed that supports the identification of the
dependent and independent constructs relevant
to each IS theory. From these constructs only
those are selected that clearly associate the ISIE
factor with the two chosen IS theories. After
identifying the constructs, a relevance check is
conducted – this sub-step is to ensure the whole
process is moving rightly, resulting in
identifying the gap. This void is then translated
into a single KC for each ISIE related factor.
 Step 3 details the process by which the MOOTS
and the PIE classification approach is combined
with expert knowledge to construct a matrix (of
ISIE factors. Through pairwise comparison –
the so-called Field Anomaly Relaxation (FAR)
as stated by Rhynne [37] – these factors then
determine the scope of the knowledge to be
mapped. Each of these factors is then assigned
fuzzy weightings using a range of positive to
negative values. A directed graph can be
constructed of these pairwise fuzzy values –
which ultimately becomes the FCM.
 Step 4 involves the algorithmic process of the
FCM simulation. This needs several simulation
scenarios to be identified. These scenarios are
effectively vectors which denote the initial
states of the ISIE factors from Step 3. These
vectors are details of expert knowledge encoded
into numerical fuzzy values per factor. These
vectors are, in turn, fed into the simulation
algorithm where the successive nodal states of
each factor in the directed graph are updated
from the preceding nodal state until equilibrium
is achieved. The output values for each node,
hence the ISIE factor, are plotted against
iterative steps. Then updated FCM is created
through calculating the inverse of the fuzzy
weight matrix and the final ISIE nodal values.
3. Information Systems Investment Evaluation
Information systems constitute a substantial
financial investment for organisations [12], thus, they
should be justified, evaluated and managed with
caution [4]. To invest in new IS and or enhance the
efficiency of existing IS and technological
infrastructure, managements are required to consider
investment risks and payoffs and acquire knowledge
of existing IS inefficiencies, respectively [21]. In this
context, evaluating IS investments becomes a
requisite for management, for the purpose that
enterprise IS implementation influence the way
organisations operate and impact their strategies,
tactics and operational decisions. Evaluation is vital
to rationalise higher IS investment costs,
improbability of returns from IS investments and act
as a control and management mechanism [4].
However, Stockdale and Standing [44] argue here
that in evaluating IS, a crucial task is to develop
frameworks that are effectively standardised and can
be applied to a broad range of applications but at the
same time thoroughly detailed to offer effective
support. In this context, systematic but equally
coherent methodologies are a prerequisite to
determine IS justification concerns emerging from
the complexity of topical technologies [8]. Though,
successfully pursuing ISIE can lead an organisation
to sustain its corporate viability and success.
3.1 ISIE in Manufacturing Context
To improve business operations and productivity
of the supply chain, manufacturing organisations
have developed themselves as responsive industrial
entities by implementing pertinent IS. The latter
argument is supported by Mondragon et al., [26] who
report that the operations of agile organisations
require the existence of efficient IS within their
supply chains. Wang et al., [47] report that
information systems are vital for (a) instigating
transformational changes within and between
2
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
manufacturing organisations and (b) the efficient
functioning of their operations i.e. design,
production, and delivery of manufactured goods.
According to Coronado et al., [5] IS implementation
enhances the agility of manufacturing organisations
and enables the development of resilient and
collaborative relations. Despite the significance of IS,
there are a number of cases where manufacturing
organisations have been unsuccessful in completing
their IS projects. This has resulted in manufacturing
organisations failing to gratify their key stakeholders.
For example Irani et al., [16] in their study on SME
manufacturing enterprises focused on gaining
insights SME IS implementation failures. The
reasoning concluded from their IS failure was failing
to realise the human and organisational factors
influencing the evaluation/implementation process.
Perera and Costa [33] also alleged that most of the IS
investments by manufacturing organisations have not
reaped the desired returns. This is because selecting
an appropriate IS is primarily complicated and vital
decision for manufacturing organisations [2].
Such commentaries on IS performance show that
while manufacturing organisations have benefited
from IS, many of them have been less than satisfied.
One of the apparent reasons for this discontentment is
reported by Small [41] who states that inapt IS
investment justification practices can result in
organisations not able to differentiate the benefits that
manufacturing IS may
be capable of conferring,
whilst, there is a lack of evidence on returns on IS
investment as managements have failed to prove the
tangible returns on the resources deployed to plan,
develop, implement and operate IS. Advocates claim
that ISIE when managed and pursued effectively can
have a positive impact on organisational performance
and productivity [9]. Thus, a formal justification
proposal must be prepared and accepted by decisionmakers, prior to IS investments [14].
3.2 MOOTs Classification of ISIE Factors
In Table 1, the authors present 15 factors (based
on the MOOTS dimension) that define ISIE in the
manufacturing context. This list of factors may not
be comprehensive, nevertheless, these 15 factors are
identified based on the literature specifically
focusing on IS, ISIE, manufacturing organisations
and supply chain management. These factors also
directly related to the context of this research.
4. From Knowledge to Knowledge Management
Knowledge has long been deliberated as a vital
organisational asset supporting the top management
in their decision-making process and augmenting
organisational competitiveness [34]. Advocates
highlight that its effective management is essential
for organisational success [7]. Knowledge is a blend
Dimension
ISIE Factors
Management Commitment (MC)
Managerial
Management Style (MS)
Managerial Capability (MC*)
Organisational Cultural (OC)
Organisational
Organisational Performance (OP)
Organisational Size (OS)
Employee Commitment (EC)
Training and Education (TE)
Operational
Information Systems & Manufacturing Agility
(ISMA)
Enterprise Integration in Manufacturing (EIM)
Information Systems & Organisational Fit (ISOF)
Technological
Information Systems Quality Output &
Performance (ISQOP)
Strategic Information Systems Impact (SISI)
Strategic
Business Strategy & IS Alignment (BSISA)
Strategic IS Business Partnership (SISBP)
Table 1: MOOTS Classification of Factors Defining ISIE
of experience, values, circumstantial information and
professional understanding that support the
evaluation and incorporation of new experiences and
information. Researchers have thus studied
Knowledge Management (KM) so as to define its
involvement
in
managing
and
leveraging
organisational knowledge [28]. Organisations that
focus on KM tools and techniques recognise the
significance of exchanging knowledge, as this
sharing of exchange increases efficiency and
sustained competitiveness. Such use of knowledge
tools and techniques to generate economic benefits is
one facet of knowledge economy [42]. According to
Powell and Snellman [35], the key component of
knowledge economy is a higher dependence on
intellectual competencies of human resources than
contribution from other physical or natural resources.
From the economic perspective, Shin [42] claims that
employing KM tools and techniques for knowledge
sharing should not be assumed on a simple conjecture
that organisational competitiveness is positively
interrelated with knowledge sharing. Shin [42]
further argues that knowledge sharing has a negative
link with competitiveness and a positive one.
Considering both the knowledge-sharing aspects, the
economic viewpoint may offer a way to explore how
to curtail barriers and promote enablers so as to
acquire knowledge sharing benefits in organisations.
Thus, three important aspects can be drawn from
the above discussion i.e., People (organisational and
cultural aspects of the use of knowledge); Process
(methods and techniques for managing the flow of
knowledge); and Technology (tools and infrastructure
that provide access to knowledge). From these three
key aspects, a rational theme has been to relate
explicit and tacit forms of knowledge together [17].
In the context of organisational knowledge, Nonaka
[30] reported that this level of knowledge is created
through a series of socialisation, externalisation,
combination and internalisation that transform
3
Ref.
[29]
[29]
[6]
[19]
[45]
[25]
[27]
[3]
[31]
[31]
[15]
[36]
[11]
[36]
[48]
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
same process in order to identify related knowledge
components. As highlighted in Figure 1, the steps
include (with an example of how a KC for
‘management commitment’ factor is identified from
Step 1 to Step 5 in Table 2) the following:
 Step
1
is
about
identifying
the
assumptions/starting point – For each ISIE
factor an assumption is developed. The
assumption is divided into the ‘Focus’ and
‘Dependence’ of the ISIE factor. For example,
Focus signifies the central theme i.e. the
management is committed to ISIE – this
indicates the Focus of management; whereas,
Dependence shows the state of being
determined, influenced or controlled by
something else i.e. MC is dependent on the
availability and utilisation of resources. From
this assumption (i.e., dependence), the keywords
extracted are availability, utilisation, resources,
and evaluation.
 Step 2 is about identifying the relevant IS
theories, models or frameworks – In this step,
the authors identified two relevant IS theories
for each ISIE factor. The decision to select two
appropriate IS theories, models, or framework
was made on the understanding that it would
allow flexibility in extracting a relevant KC. For
example, for the ‘management commitment’
factor, the resource-based view and contingency
theory were considered relevant based on their
dependent and independent constructs.
As
sum
on
pti
s
ISIE Factors
In an attempt to determine the KC resulting from
ISIE, a five-step Pairwise IS Theory Equivalence
(PIE) process is followed (Figure 1). Each ISIE factor
defined under MOOTS classification will follow the
Th
eor
ies
STEP 1
STEP 2
Identifying the
assumptions /
starting point
for ISIE Factor
Identifying relevant IS
theories, models or
frameworks for each
ISIE Factor
Dependent /
Independent Constructs
STEP 3
nc
e
Rationale – Identifying
the Key Dependent/
Independent Constructs
that link ISIE Factor
with a relevant Theory
el
ev
a
STEP 4
Relevance Check –
Identifying the Link
between Step 1 and
Step 3
STEP 5
Identifying the
Gap as a result of
Step 4
Gap Analysis
ISIE related Knowledge
Components
4.1 Process of Identifying ISIE related KC
IS
R
knowledge between the tacit and explicit modes. In
line with this dynamic process of knowledge
creation, the process of transmitting tacit knowledge
to explicit (and the other way round), is a collective
act, where knowledge is relocated and shared with
other individuals in the organisation. Thus, two key
points can be extracted from the aforesaid knowledge
conceptualisation:
 First, knowledge is in a reformed mode;
nevertheless, to make an individual’s
knowledge beneficial for others, it must be
conveyed effectively and efficiently and that it
is understandable and open to others. This can
be attained by using expert systems – the
fundamental idea is that knowledge from the
human mind is stored in the computer and users
call upon the computer for specific advice as
and when necessary. The computer can make
extrapolations and reach to a decision. Then like
a human expert, the computer offers suggestions
and explains, if need be, the rationality behind
the suggestion. Expert systems offer an effective
and flexible means of exploring solutions to
problems that often cannot be dealt with by
other, more orthodox approaches.
 Second, hoard of information is of limited value
to organisations – it is only when this
information is dynamically processed in an
individual’s mind through a process of
discussion and learning that it can be effective.
At this point in time, KM plays a vital role by
performing as a meticulous process for managing
intellectual and knowledge assets to meet
organisational aim and objectives. KM aims to make
knowledge approachable and re-utilisable for and by
the organisation. Therefore, knowledge not only
exists in documents and repositories, but it becomes
rooted in individuals’ minds over time and is
revealed through their actions and behaviours.
Likewise, in an organisational context, decision
makers and their decision-making processes are
influenced by the knowledge that is generated as a
result of evaluating organisational IS investments
[18]. Kulkarni et al., [24] highlight that knowledge is
entrenched and streams across various units within an
organisation. For example, experts with specific
domain capability, categorical best practice measures,
or lessons learned from related experiences,
documents, daily operational practices, and IS. It is,
therefore, important to understand the different types
of KCs so as to uncover its likely contribution to the
performance of an organisation [32].
KC
Figure 1: A Three-Step PIE Process for Identifying ISIE
related KC
 Step 3 is about developing the rationale –
Identifying the main dependent and independent
constructs relevant to an IS theory, model or a
4
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
STEP 4 –
Relevance Check
 Assets
 Capabilities
 Resources
 Competitive
Advantage
 Organisational
Performance
Contingency
Theory





Strategy
Technology
Task
Organisational Size
Structure & Culture
 Efficiency
 Organisational
Performance




Assets and Resources
Efficiency
Capabilities
Organisational Performance




Availability
Utilisation
Resources
Evaluation
 Effective Use of Resources
Table 2: Example of Managerial Dimension ISIE Factor
(Management Commitment) and related KC
 Step 5 is about identifying a gap as a result of
Step 4. Based on the relevance check of
keywords and dependent/independent constructs
a KC is extracted. The authors assert that a KC
is based upon relevant IS theories, models or
Managerial
Dimension
Technological Operational Organisational
Dimension
Dimension
Dimension
Resource
Based
View
frameworks that, in turn, support that KC. For
example, for the ‘management commitment’
factor, the knowledge component identified is
Effective Use of Resources.
A similar process for extracting the relevant KCs
is followed for each of the other 14 ISIE factors. The
remaining ISIE factors and their related KCs are
presented in Table 3.
Strategic
Dimension
MANAGERIAL DIMENSION
(a) FOCUS: Management is committed to
evaluating their IS investments.
(b) DEPENDENCE: MC is dependent on
the availability and effective utilisation of
financial and other organisational resources
e.g. if there is enough investment to
implement and evaluate IS, management
will be committed towards promoting /
pursuing the evaluation.
STEP 3 – Main Dependent
& Independent Constructs
STEP 5 –
for Selecting Theories STEP 2 –
KC
IS
related to Relevant
Relevant
Theories
M
Keywords Constructs Dependent Independent
from Step 1 from Step 3
STEP 1 –
Assumptions
framework. However, from these available
constructs, only those constructs were selected
that associate an individual ISIE factor with the
two relevant IS theories, models or frameworks.
For example, the constructs that were deemed
relevant from the two IS theories, models or
frameworks are assets, resources, efficiency,
capabilities and organisational performance.
 Step 4 is about conducting a relevance check –
Identifying the link between Step 1 and Step 3.
This relevance check enabled the authors to
relate the keywords extracted from the
assumption to the dependent and independent
constructs of each IS theory, model or
framework to identify a gap.
ISIE Factors
Knowledge Components
MC
Effective Use of Resources
MS
Past Management Experience
EIM
Improved Performance &
Management of Resources
Effective Organisational
Benchmarks & Performance
Performance Management
Metrics
Better Management of
Resources
Actual Use of IS
Skills Identification & System
Training
Consistent Information
Output
Radical Transformation
ISOF
Use of System
ISQOP
Quality Production &
Performance Measurement
MC*
OC
OP
OS
EC
TE
ISMA
SISI
BSISA
SISBP
Enhancing Organisational
Competitiveness
Strategic Alignment/Fit
Procedures
Development & Effectiveness
of Relationship
Table 3: All ISIE Factors and their related KCs
5. Visualising the Knowledge Map
The 15 ISIE knowledge components presented in
Tables 2 and 3 were grouped into six key thematic
areas as shown as follows, i.e., constructed to be
within a morphological field of factors [37].
 ICT – Actual Use of IS, Use of System.
 MGMT – Past Management Experience, Better
Management of Resources.
 PERF – Improved Performance and
Management
of
Resources,
Effective
Organisational Benchmarks and Performance,
Performance Management Metrics, Consistent
Information Output, Radical Transformation, &
Quality Production, Performance Measurement.
 RES – Effective Use of Resources,
Development and Effectiveness of Relationship.
 SKILLS – Skills Identification & Systems
Training.
5
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
 STRAT
–
Enhancing
Organisational
Competitiveness,
Strategic
Alignment/Fit
Procedures.
By doing so, the authors wished to carry out a
pairwise analysis to determine and remove any
redundant / duplicated factors. This approach has
been successfully used before [39]. In comparing any
and every two sets of factors, a reduced
morphological field was generated leading to an 83%
reduction of ISIE constructs (i.e. from 90 to 15). The
method for doing so was based on identifying those
ISIE pairwise combinations where four or more
similar dependent / independent sub-constructs
existed. The resulting reduced morphological field of
ISIE factors is therefore shown in Table 4.
ICT
MGMT
Actual Use
of IS
Past
Management
Experience
Better
Management of
Resources
upon the PIE construct, to yield the fuzzy weight
matrix in Table 5. Subsequently the FCM in Figure 2
was constructed as a directed graph, where the
strength of causality between each node (henceforth
USEIT – TRANSF in the weight matrix) was
determined by the thickness of the line connecting
each factor. A thicker line / thinner line denotes
stronger / weaker causal relationships, respectively,
and a value of 0 or no line indicates no relationship.
PERF
Effective
Organisational
Benchmarks &
Performance
Radical
Transformation
Table 4: ISIE Factors Following Pairwise Analysis
This set of ISIE factors was then used as the basis for
constructing the fuzzy cognitive map. Causal
relationships were developed by the authors based
Actual Use of IS
Past Management
Experience
Better Management of
RES
Resources
Effective Organisational
PERF
Benchmarks & Performance
TRANSF Radical Transformation
USEIT
PAST
MGMT
Figure 2: FCM of ISIE Factors
USEIT
PAST MGMT
RES
PERF
TRANSF
Better
Effective Organisational
Actual Use Past Management
Radical
Management
Benchmarks &
of IS
Experience
Transformation
of Resources
Performance
0
0.333
0.667
0
1
1
0
-0.333
-0.333
0
0.333
0
0
0.333
0.667
0.333
0
1
0
1
-0.667
0.333
0
0.667
0
Table 5: Fuzzy Weight Matrix for the ISIE Factors
5.1 Knowledge Mapping / Simulation of the FCM
The mapping and simulation of the FCM follows
the technique defined by Kosko [23] and as denoted
by Sharif and Irani [39]. The authors as a result used
the TAPE framework [40] to identify two scenarios
in manufacturing ISIE (with user’s and manager’s
perspective of evaluating an IS). These were used as
‘seed’ factors in the FCM – wherein each of the FCM
nodes were classified into explicit or tacit KCs.
5.2 Analysis
In both scenarios (Figures 3 & 7) the overall
results show that the dynamics are founded on high to
low causal responses ranging from TRANSF, PERF,
RES, PAST MGMT through to USEIT. First
scenario, there is a major shift in terms of the
negative causal response related to USEIT and RES;
with opposite causal responses from other factors.
Second scenario, there is a constant large negative
reduction in the USEIT variable (from 0.93 to 0.09
showing little or no causal effect); and a
corresponding fall in causal effect for RES and
TRANSF. But, there are causal increases for PAST
MGMT and PERF. This is more prominent when the
starting and ending nodal values are plotted for each
scenario (Figures 5, 6, 9 and 10), respectively.
6
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
Figure 3: FCM Results for Scenario 1
(----)
Figure 8: Resulting FCM for Scenario 2
(--)
USEIT
(----)
(++++)
(----)
PERF
(++++)
PAST
MGMT
(++++)
(---)
(----)
(----)
(----)
(---)
(++++)
(----)
(--) (--)
TRANSF
(--)
(----)
(----)
(++++)
RES
Figure 9: Starting Nodal Values for Scenario 2
(----)
Figure 4: Resulting FCM for Scenario 1
Figure 10: End Nodal Values for Scenario 2
Figure 5: Starting Nodal Values for Scenario 1
Figure 6: End Nodal Values for Scenario 1
Plotting the separate responses on a polar and/or
Cartesian scale, magnifies the respective scenario
results further as shown in Figures 11 and 12. Here,
across both scenarios there appear to be two key
dynamic factors which dominate the results – namely
the interactions between nodes PERF and RES which
show an out-of-phase relationship with one another
(highlighting that as the causality of organisational
benchmarks and performance increases, the causality
of better management of resources decreases (and
vice versa). Underlying all of these factors, the
dynamic of the past management experience node
(PAST MGMT) appears to have a stabilising effect
on all of the other nodes – i.e., when this nodal
response stabilises, all the other nodes stabilise soon
afterwards as well. Hence, in mapping the knowledge
involved in ISIE, the PAST MGMT factor appears to
have a tacit controlling impact over other factors.
But, analysing the FCM results which at last encode
the knowledge within the ISIE process in this vein
reveals that this is only part of the overall picture.
Figure 7: FCM Results for Scenario 2
7
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
RES
PERF
TRANSF
0.000
PAST
MGMT
-0.883
-0.938
-0.785
-0.968
-0.546
0.000
0.617
0.617
0.369
-0.561
0.729
0.938
-0.293
0.851
0.620
0.000
0.253
0.785
-0.561
0.000
0.372
-0.748
0.253
0.000
USEIT
USEIT
PAST
MGMT
RES
PERF
TRANSF
Table 7: Final fuzzy weight matrix for Scenario 2
Figure 11: Scenario 1 Phase Plot
Figure 12: Scenario 2 Phase Plot
Figures 4 and 8 show the respective reconstructed
FCM diagrams as following each of the simulation
runs. The redrawn FCM in both cases were
constructed through matrix manipulation of the
computed results, scenario vectors and the original
fuzzy weight matrix. This yielded the computed
fuzzy eight matrices in Tables 6 and 7 for scenarios 1
and 2, respectively. The FCMs were once again
drawn and constructed based upon the pairwise
relationship between each node and the strength of
each of the causal links determined the thickness of
the lines connecting nodal points. Ultimately for both
scenarios and FCMs, these diagrams show that each
scenario involves an inherent range of interrelationships belying the initial FCM.
USEIT
USEIT
PAST
MGMT
RES
PERF
TRANSF
0.000
PAST
MGMT
0.956
RES
PERF
TRANSF
0.915
0.977
0.842
First, analysing scenario 1 through the redrawn
FCM (Figure 4) shows that strong causal
relationships continue to exist between PERF – RES
and USEIT-TRANSF, a range of others have either
strengthened, weakened significantly or new
relationships have emerged (as shown in Figure 13).
The resulting FCM shows a unique set of internal
‘knowledge loops’ arising from the USEIT node
supplemented by inputs into the PERF component
also. The strength of these causal inter-relationships
appear to suggest that factors of USEIT, PERF and
RES are given more prominence by users of IT in
any IS evaluation, based upon how they may perceive
the utility and benefit of IS in the work that they do.
In this FCM it is also interesting to note that there are
considerably weak causal relationships in the ‘outer
loop’ of relationships (between PERF-PAST MGMT
and PERF-RES especially) that reveal the challenges
that management faces in overcoming the ‘benefit of
hindsight’ effect concerning past experiences of IT
benefits and risks. This is consistent with literature
where tactical/operational considerations within the
IS evaluation process shared alike relationship [16].
Aspect
Decrease of
Causal
Decrease
of
Knowledge
Causal
Relationships
Knowledge
Relationships
Increase of
Causal
Knowledge
Relationships
New Causal
Relationships
-0.948
0.000
-0.445
-0.445
-0.671
-0.742
-0.897
-0.915
-0.552
-0.809
-0.988
0.000
-0.972
-0.977
-0.742
0.000
-0.994
-0.859
-0.972
0.000
Total
Negative
Causal
Relationships
Table 6: Final Fuzzy Weight Matrix for Scenario 1
Total Positive
Causal
Relationships
FCM
N/A
N/A
15
Scenario 1
Scenario 2
USEIT-TRANSF
PAST MGMT-TRANSF
RES-USEIT
RES-TRANSF
PERF-USEIT
PERF-RES
PERF-TRANSF
TRANSF-USEIT
TRANSF-PAST MGMT
TRANSF-PERF
USEIT-PAST MGMT
USEIT-RES
USEIT-TRANSF
PAST MGMT-USEIT
RES-USEIT
PERF-RES
PERF-TRANSF
TRANSF-PERF
TRANSF-PERF
USEIT-PAST MGMT
USEIT-RES
PAST MGMT-PERF
PAST MGMT-TRANSF
PERF-USEIT
TRANSF-USEIT
TRANSF-PAST MGMT
USEIT-PERF
RES-PASTMGMT
RES-PERF
PERF-PASTMGT
TRANSF-RES
USEIT-PERF
PAST MGMT-RES
PERF-PAST MGMT
TRANSF-RES
17
9
5
10
3
12
Figure 13: Summary of Causal Relationships
8
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
Second, analysing scenario 2 through the redrawn
FCM in Figure 8 shows that there are a range of
strong
causal
inter-relationships
principally
emanating from the PERF and TRANSF nodes.
These knowledge loops subsequently jointly feed into
the USEIT and PAST MGMT nodes (with the latter
having a strong reinforcing loop back to PERF).
Here, it is evident that while there may be implicit
and weak causal links between the impact of
resources applied to IS and the impact that IS has on
organisational performance, it is interesting to note
that the majority of the negative causal loops in the
resulting FCM are related to those factors which
implicitly involve the interactions between users and
how resources may be used to derive benefit from IT.
As identified within Figure 13, even though the initial
FCM started off with a greater proportion of
positive/strong to negative/weak causal links (i.e. 12
vs. 3) between the scenarios explored, there are 88%
more weak causal links that emerged overall as a
result of each of each of the FCM simulations. There
are almost three times as many weak relationships as
strong relationships in scenario 1 (i.e. 17 vs. 5),
although there is a better balance between these in
scenario 2 (i.e. 10 vs. 9). This shows that ultimately a
mapping of the knowledge inherent in these IS
evaluation scenarios gives an indication of the
dynamics of just how difficult and complex it is to
overcome organisational culture and technology
adoption factors (encapsulated by the explicit
knowledge PERF component) as well as integrated
and effective efficiency strategies (encapsulated by
the tacit knowledge TRANSF component).
6. CONCLUSIONS
This paper sought to extend the view and
understanding of knowledge by seeking to apply a
cognitive technique to explore knowledge-based
decisions involved in evaluating IS investments. This
has also been with a view to researching into how
strategic IS is used in manufacturing where a paucity
of research into the breadth and impact of how
knowledge is used in such a context has previously
been identified [7]; and, how knowledge may be
represented in knowledge-intensive environments
[41]. In light of the examination of the literature, this
paper presented a MOOTS classification of factors
defining ISIE in the manufacturing organisations.
Therefore, the authors identified the key KCs related
to each ISIE factor, using the five-step PIE
framework. Subsequently, the MOOTS and the PIE
classification approach is pooled with expert
knowledge to construct a matrix of ISIE factors – this
step led towards the development of the FCM of ISIE
factors. The latter process allowed the authors to
identify the inter-relationships and intricacies of
decision-making constructs in this research case.
The FCM approach used sought to identify those
knowledge constructs which are relevant to the ISIE.
This paper has been able to present this technique as a
means of exploring and therefore mapping the
knowledge inherent in IS evaluation from both user
and managerial perspectives. The resulting
knowledge mapping through the application of an
FCM has shown the intricate and developing
behaviour of causal relationships within the
knowledge area. The main relationships and
knowledge within ISIE have been shown to be driven
by a mixture of managerial and user perspectives.
These are eventually balanced by strong (and weak)
driving elements centring around: the actual
(intended) usage of IS and the instant operational /
tactical benefits this can provide from a user
perspective (FCM scenario 1); and a clear set of
relationships based upon how organisational culture,
technology adoption and the integration of IS for the
benefit of the organisation from a manager’s
perspective (FCM scenario 2).
7. References
[1] Caldeira, M. M. and Ward, J. M. “Understanding the
successful adoption and use of IS/IT in SMEs: an explanation
from Portuguese manufacturing industries” Information
Systems Journal, 2002, 12(2), pp. 121-152.
[2] Cebeci, U. “Fuzzy AHP-based decision support system for
selecting ERP systems in textile industry by using balanced
scorecard”, Expert Systems with Applications, 2009, 36, pp.
8900-8909.
[3] Choi, D., Kim, J. and Kim, S. “ERP training with a webbased electronic learning system: the flow theory perspective”,
International Journal of Human Computer Studies, 2007,
65(3), pp. 223-243.
[4] Chou, T-Y., Chou, S-C. T. and Tzeng, G-H. “Evaluating
IT/IS investments: A Fuzzy Multi-criteria decision model
approach”, European Journal of Operational Research, 2006,
173(3), pp. 1026-1046.
[5] Coronado, A. E., Lyons, A. C. and Kehoe, D. F. “Assessing
the value of information systems in supporting agility in hightech manufacturing enterprises”, International Journal of
Operations and Production Management, 2004, 24(12), pp.
1219-1246.
[6] Fink, L. and Neumann, S. “Exploring the perceived
business value of the flexibility enabled by information
technology infrastructure”, Information & Management, 2009,
46, pp. 90-99.
[7] Gunasekaran, A. and Ngai, E. W. T. “Knowledge
management in 21st century manufacturing”, International
Journal of Production Research, 2007, 45(11), 2391-2418.
[8] Gunasekaran, A., Ngai, E. W. T. and McGaughey, R. E.
“Information Technology and Systems Justification: A Review
for Research and Applications”, European Journal of
Operational Research, 2006, 173(3), pp. 957-983.
[9] Gunasekaran, A., Love, P. E. D., Rahimi, F. and Miele, R.
“A model for investment justification in information
technology projects”, International Journal of Information
Management, 2001, 21, pp. 349-364.
[10] Hedman, J. and Borell, A. “Narratives in ERP systems
evaluation”, Journal of Enterprise Information Management,
2004, 17(4), pp. 283-290.
9
Proceedings of the 47th Hawaii International Conference on System Sciences - 2014
[11] Hendricks, K. B., Singhal, V. R. and Stratman, J. K. “The
impact of enterprise systems on corporate performance: A
study of ERP, SCM, and CRM system implementations”,
Journal of Operations Management, 2007, 25, pp. 65-82.
[12] Irani, Z. “Investment evaluation within project
management: an information systems perspective”, Journal of
Operational Research Society, 2010, 61, pp. 917-928.
[13] Irani, Z. and Love, P. E. D. “The propagation of
technology management taxonomies for evaluating
information systems”, Journal of Management Information
Systems, 2001, 17(3), pp. 161-177.
[14] Irani, Z., Sharif, A. M., Love, P. E. D. and Kahraman, C.
“Applying concepts of fuzzy cognitive mapping to model the
IT/IS Investment evaluation process”, International Journal of
Production Economics, 2002, 75(1), pp. 199-211.
[15] Irani, Z. “Information Systems Evaluation: Navigating
through the Problem Domain”, Information & Management,
2002, 40(1), pp. 11-24.
[16] Irani, Z., Sharif, A. M. and Love, P. E. D. “Transforming
failure into success through organisational learning: An
analysis of a Manufacturing Information System”, European
Journal of Information Systems, 2001, 10(1), pp. 55-66.
[17] Irani, Z., Sharif, A. M. and Love, P. E. D. “Linking
Knowledge Transformation to Information Systems
Evaluation”, European Journal of Information Systems, 2005,
14(3), pp. 213-228.
[18] Irani, Z., Sharif, A. M. and Love, P. E. D. “Knowledge
Mapping and Information Systems Evaluation in
Manufacturing”, International Journal of Production
Research, 2007, 45(11), pp. 2435-2457.
[19] Ke, W. and Wei, K. K. “Organizational culture and
leadership in ERP implementation”, Decision Support Systems,
2008, 45(2), pp. 208-218.
[20] Kim, S., Hong, J. and Suh, E. “A Diagnosis Framework
for Identifying the Current Knowledge Sharing Activity Status
in a Community of Practice”, Expert Systems with
Applications, 2012, 39(18), pp. 13093-13107.
[21] Kim, Y. J. and Sanders, G. L. “Strategic actions in
information technology investment based on real option
theory”, Decision Support Systems, 2002, 33, pp. 1-11.
[22] Koduru, H., Xiao, F., Amirkhanian, S. and Juang, C.
“Using Fuzzy Logic and Expert System Approaches in
Evaluating Flexible Pavement Distress: Case Study”, Journal
of Transportation Engineering, 2010, 136(2), pp. 149-157.
[23] Kosko, B. Neural Networks and Fuzzy Systems, Saddle
River, NJ: Prentice-Hall, 1991.
[24] Kulkarni, U. R., Ravindran, S. and Freeze, R. “A
Knowledge Management Success Model: Theoretical
Development and Empirical Validation”, Journal of
Management Information Systems, 2006, 23(3), 309-347.
[25] Love, P. E. D. and Irani, Z. “An exploratory study of
information technology evaluation and benefits management
practices of SMEs in the construction industry”, Information &
Management, 2004, 42(1), pp. 227-242.
[26] Mondragon, A. E. C., Lyons, A. C. and Kehoe, D. F.
“Assessing the value of information systems in supporting
agility in high-tech manufacturing enterprises”, International
Journal of Operations & Production Management, 2004,
24(12), pp. 1219-1246.
[27] Moynihan, D. P. and Pandey, S. K. “Finding Workable
Levers over Work Motivation Comparing Job Satisfaction, Job
Involvement,
and
Organizational
Commitment”,
Administration & Society, 2007, 39(7), pp. 803-832.
[28] Nevo, D. and Chan, Y. E. “A Delphi study of Knowledge
Management Systems: Scope and Requirements”, Information
& Management, 2007, 44, pp. 583-597.
[29] Ngai, E. W. T., Law, C. C. H. and Wat, F. K. T.
“Examining the critical success factors in the adoption of
enterprise resource planning”, Computers in Industry, 2008,
59(6), pp. 548-564.
[30] Nonaka, I. “A Dynamic Theory of Organizational
Knowledge Creation”, Organization Science, 1994, 5(1), pp.
14-37.
[31] Panetto, H. and Molina, A. “Enterprise integration and
interoperability in manufacturing systems: trends and issues”,
Computers in Industry, 2008, 59(7), pp. 641-646.
[32] Pemberton, J. and Stonehouse, G. “Organizational
learning and knowledge assets – an essential partnership”, The
Learning Organization, 2008, 7(4), pp. 184-193.
[33] Perera, H. S. and Costa, W. K. “Analytic Hierarchy
Process for Selection of ERP Software for Manufacturing
Companies”, The Journal of Business Perspective, 2008,
12(4), pp. 1-11.
[34] Polanyi, M. The Tacit Dimension, Garden City, NY:
Doubleday, 1967.
[35] Powell, W. W. and Snellman, K. “The Knowledge
Economy”, Annual Review of Sociology, 2004, 30,199-220.
[36] Raymond. L., Croteau, A. -M. and Bergeron, F. “The
Strategic Role of IT: An Empirical Study of its Impact on IT
Performance in Manufacturing SMEs”, Proceedings on the
Sixth International Conference on Internet and Web
Applications and Services, 2011, pp. 89-97.
[37] Rhynne, R. “Field Anomaly Relaxation – The Arts of
Usage”, Futures, 1995, 27(6), pp. 657-674.
[38] Sharif, A. M., Irani, Z. and Weerakkody, V. “Evaluating
and modelling constructs for e-Government decision making”,
Journal of the Operational Research Society, 2010, 61 (6), pp.
928-952.
[39] Sharif, A. M. and Irani, Z. “Applying a FuzzyMorphological approach to Complexity within management
decision-making”, Management Decision, 2006, 44(7), pp.
930-961.
[40] Sharif, A. M. Knowledge representation within
Information Systems in Manufacturing Environments. PhD
Thesis, Brunel University, UK, 2004.
[41] Shaw, D. and Edwards, J. S. “Manufacturing knowledge
management strategy”, International Journal of Production
Research, 2006, 44(10), pp. 1907-1925.
[42] Shin, M. “A Framework for evaluating economics of
knowledge
management
systems”,
Information
&
Management, 2004, 42(1), pp. 179-196.
[43] Small, M. H. “Planning, justifying and installing advanced
manufacturing technology: a managerial framework”, Journal
of Manufacturing Technology Management, 2007, 18(5), pp.
513-537
[44] Stockdale, R. and Standing, C. “An interpretive approach
to evaluating information systems: A content, context, process
framework”, European Journal of Operational Research,
2006, 173(3), pp. 1090-1102.
[45] Velcu, O. “Exploring the effects of ERP systems on
organizational performance: Evidence from Finnish
companies”, Industrial Management & Data Systems, 2007,
107(9), pp. 1316-1334.
[46] Vundavilli, P. R., Parappagoudar, M. B., Kodali, S. P. and
Benguluri, S. “Fuzzy logic-based expert system for prediction
of depth of cut in abrasive water jet machining process”,
Knowledge-Based Systems, 2012, 27, pp.456-464.
[47] Wang, E., Klein, G. and Jiang, J. J. “IT support in
manufacturing firms for a knowledge management dynamic
capability link to performance”, International Journal of
Production Research, 2007, 45(11), pp. 2419-2434.
[48] Zhang, M. Sarker, S. and Sarker, S. “Unpacking the effect
of IT capability on the performance of export-focused SMEs: a
report from China”, Information Systems Journal, 2008, 18(4),
pp. 37-380.
10