RISK ANALYSIS IN AN EXTENDED ENTERPRISE ENVIRONMENT:
IDENTIFICATION OF KEY RISK FACTORS IN B2B E-COMMERCE RELATIONSHIPS
Vicky Arnold
University of Connecticut/University of Melbourne
Clark Hampton
University of Connecticut
Deepak Khazanchi
University of Nebraska-Omaha
Steve G. Sutton
University of Connecticut/University of Melbourne
September 2003
Preliminary Draft
Please Do Not Quote Without Permission
We gratefully acknowledge financial support from the Institute of Internal Auditors.
RISK ANALYSIS IN AN EXTENDED ENTERPRISE ENVIRONMENT:
IDENTIFICATION OF KEY RISK FACTORS IN B2B E-COMMERCE RELATIONSHIPS
SYNOPSIS
In October 2001, the International Federation of Accountants (IFAC 2001) released a
proposed International Audit Standard on the impact of electronic commerce on the audit of financial
statements. The proposal was accepted as an International Audit Practice Statement (recommended
procedures—not required) in March 2002. The audit statement focuses on four aspects of ecommerce risks: (1) legal and regulatory issues, (2) transaction integrity, (3) transaction security, and
(4) process alignment (IFAC 2001, 2002b). Sutton and Hampton (2003) note that this statement is
critical in two respects: (1) it is the first explicit recognition by an audit standards board of potential
risks exuding from e-commerce activities, and (2) while the first three issues are arguably covered
under general internal control standards, process alignment emerged for the first time as a
fundamental risk category in the audit of financial statements.
Within the U.S., the Sarbanes-Oxley Act has brought IT and e-commerce risks to the
forefront as companies struggle to address Section 404 requirements related to reporting on internal
controls. Organizations are slowly coming to the realization that requirements under Sarbanes-Oxley
extend internal controls within the audit context from direct controls over financial activities to a
broader enterprise risk management frame that includes strategic, operational, reputational,
regulatory, and information risks (Katz 2003; Banham 2003). In a recent survey of CIOs, the largest
planned increases in expenditures related to investments in security and e-commerce initiatives
despite tight IT budgets (Ware 2002). Ware further notes, “Companies will take steps to shore up
their security to give customers and business partners peace of mind. More than half (56%) of the
panelists plan to increase their spending on security software and only 6% plan to decrease spending
in this area.”
In this study, the focus is on identifying the critical risk factors that can be used to assess the
impact of B2B e-commerce on overall enterprise risks. The Khazanchi and Sutton (2001) framework
for B2B e-commerce assurance is applied as the organizing conceptual model for the study. The
framework focuses on three primary risk categories: (1) technical level risks, (2) application level
risks, and (3) business level risks. These categories encapsulate, but broaden, the three categories
identified by IFAC (2002a) in a white paper released around the same time as the e-commerce audit
practice statement was accepted—the three categories being (1) IT infrastructure, (2) IT applications,
and (3) IT business processes. To identify a critical set of B2B risk factors, structured focus groups
were conducted with three internal constituency groups (corporate groups consisting of IS security,
internal IT audit, and e-commerce development managers) and two external constituency groups (ecommerce consultants and external IT auditors). Tests of consistency between the internal
constituency groups and then between internal constituencies and each of the external constituency
groups confirm strong agreement on the identified critical B2B risk factors. Tests were also
conducted on the relative importance of the critical B2B risk factors with the only substantial
inconsistencies existing at the internal constituency groups versus external IT auditors group for the
business-level factors. This would appear to indicate that internal business concerns over ecommerce might not have the same prioritization as do external auditors assessing enterprise risk in a
financial audit environment.
1
BACKGROUND
Since electronic data interchange (EDI) capability was first implemented in organizations
over 30 years ago, interorganizational systems have evolved and have steadily tightened the bond
between customers and buyers. The resulting relationships are often viewed as partnering or
collaborative relationships—particularly in the current Internet-driven business-to-business
(B2B) e-commerce environment where buyers are increasingly dependent on suppliers to
perform (McIvor et al. 2003). Nonetheless, the bulk of these interorganizational systems has
been EDI-driven; and, over the past decade, researchers have begun to examine the benefits and
costs that arise in such situations.
Researchers increasingly find that the benefits of basic B2B relationships are negligible
and the true benefits come from collaboration (Lee et al. 2003); without widespread adoption of
B2B application there is little payback (Iacouvou et al., 1995). The result has been that many
organizations that are in a preferred power position with their suppliers (e.g., automotive
manufacturers) have a tendency to force suppliers to implement B2B capability (Teo et al. 2003).
Yet, forcing key suppliers to implement B2B may lead to distrust which can inhibit voluntary use
and extensions of B2B applications (Hart and Saunders 1997) and in the worst case degenerate
into conflict and yield a worse rather than better business linkage (Kumar and van Dissel 1996).
Still, to sustain competitiveness in the contemporary business environment, B2B integration is
imperative (Kurnia and Johnston 2000), necessitating a greater focus on assisting business
partners in integrating new technologies and/or carefully considering the capability and
willingness of new business partners to engage in effective B2B integration (Angeles and
Ravinder 2000).
2
This selection and/or integration of business partners has become only more critical as
organizations have implemented a variety of strategies for reducing business cycle time—such as
vendor managed inventory (VMI), just-in-time (JIT) manufacturing and quick response retailing
(QR) (Khazanchi and Sutton 2001). In short, the impact is that suppliers and customers become
vital components of the value chains in the emerging business environment where competition is
increasingly supply chain versus supply chain rather than company versus company (Papazoglou
et al. 2000). This competition of supply chains induces organizations to become more selective
of their business partners and to attempt to lock-in these suppliers to ensure stability of the
supply chain (Grieger 2003). As a result, many companies are limiting their number of suppliers
in order to improve the coordination of value/supply chains and create relationships that
transcend specific transactional relationships—a concept referred to as relationalism (Grover et
al. 2002). The challenge is to develop and integrate the technology across the value chain that
will support the integration of business processes, information standards and information systems
of all partners in the value chain (Shin and Leem 2002). While the process can be painful and
costly, the implementation of such technology ultimately cements the relationship between
business partners and provides stability in value chain partnering relationships (Grover et al.
2002).
Despite all the perceived benefits of forging integrated business relationships, for many
organizations there is still some trepidation in entering into these relationships (Iacovou et al.
1995). This trepidation can come from costs, but even more frequently is a result of perceived
enterprise risks. Kumar and van Dissel (1996) summarize these risks as the costs associated with
exposure to being exploited in the relationship. These risks include transaction specific capital
(i.e., investment by one party that has little or no value outside of the business relationship),
3
information asymmetries (i.e., problems in monitoring performance which yields risk to shirking
by a business partner), and loss of resource control (i.e., resources that are transferred in a
relationship that cannot be returned or controlled in the event of termination of the relationship).
These risks primarily center on loss of investment, however, which can have negative
financial ramifications for an organization but are not likely in most cases to cripple an
organization. Yet, in an era where the focus has been on enhancing core business processes,
outsourcing activities that other organizations can do better, and developing integrated value
chains, breakdowns in relationships may have far greater ramifications than simply financial
losses and/or inefficiencies (Sutton and Hampton 2003). For instance, in a just-in-time
environment where a vendor is responsible for managing the materials and parts inventory,
failure by the vendor to deliver parts for a prolonged period of time can potentially lead to
extended shutdowns of manufacturing processes. These interruptions can put the manufacturing
company at risk due to the inability to produce goods, inability to meet obligations downstream
in the supply chain, loss of other business partners’ trust, and decline in general reputation. Such
risks could potentially jeopardize an organization’s long-term standing.
Various organizations have recognized the increased prevalence of such risks and have
endeavored to meet the needs of various information users wishing to mitigate the risk of ecommerce across the value chain. Harbinger, Inc., for instance has evolved from simply a valueadded network provider supporting EDI transactions to become an information source on
preparedness of potential partner organizations. Within the automotive industry, Harbinger now
provides reports that highlight the B2B capability and the degree of integration with underlying
business processes of various small- and medium-sized enterprises (SMEs) that serve as
suppliers to the major U.S. auto manufacturers. The reports help automobile manufacturers
4
identify potential suppliers that are more likely to be able to operate in their just-in-time
manufacturing environments and to more effectively monitor internal business risk from partner
relationships (Yost 1999). Similarly, the International Federation of Accountants (IFAC), which
sets international audit practice guidelines, has recently released two documents that advise
external auditors to evaluate the business risks that an organization assumes through e-commerce
relationships. This risk assessment should be done as part of an overall assessment of the client
organization’s business risks, financial viability, and going concern status (IFAC 2001; 2002a,
2002b).
The wide variety of concerns in B2B integration led Massetti and Zmud (1996) to the
conclusion, “What seems absent is a rich, tactical understanding that links strategic expectations
regarding [B2B] with operational plans for potential implementation.” The focus of Massetti and
Zmud’s study was on deriving factors for assessing the benefits from B2B linkage. As
interorganizational systems have become more tightly coupled, a focus on the opposite side of
the equation (i.e. associated business risks) seems particularly critical. While prior research has
addressed a variety of general risk factors in interorganizational systems linkages (e.g.,
Papazoglou et al. 2000, Unal 2000, McIvor et al. 2003, Hempel and Kwong 2001, Westland
2002, Kumar and van Dissel 1996), a focused effort on identifying the specific risk factors
within various general categories can aid managers in risk management, aid auditors and
monitoring organizations in the measurement of potential risk, and inform future development
and innovation in interorganizational systems.
The purpose of this study is to explore and identify the key risk factors involved in ecommerce driven interorganizational systems that can potentially escalate an organization’s
overall enterprise risk. This paper documents a study that targets directly the identification of the
5
key risk factors in B2B relationships. The paper uses the Khazanchi and Sutton (2001) model for
B2B e-commerce assurance services as the conceptual model for viewing specific risk
components. Initially, three structured focus groups were conducted with information systems
security, IT audit, and e-commerce development staff from each of three large corporations that
are heavy users of B2B e-commerce across the value chain in order to identify the key risks
associated with these relationships and to determine whether a consensus exists across
organizations as to what are the key risk factors. Two additional focus groups were conducted
with an e-commerce consulting firm and an external audit firm in order to explore whether
differences existed between corporate teams and external professionals. The results provide an
agreed upon set of key risk factors for each of the three risk dimensions put forth by Khazanchi
and Sutton (2001): (1) technical level risk, (2) application-user level risk, and (3) business level
risk.
The remainder of this paper is presented in four parts. First, an overview and discussion
of the B2B e-commerce assurance service model is presented. Second, the structured focus group
methodology applied in the study is discussed. The third part presents the results of the study and
the fourth part discusses the implications of the results for e-commerce managers and
researchers.
THEORETICAL MODEL
The Khazanchi and Sutton (2001) B2B e-commerce assurance model utilized in this
study was developed based on an analysis of 90 SMEs that were engaged in EDI-based B2B
relationships. The model was developed and refined using a combination of surveys, phone
interviews and written descriptions from participating organizations and consists of three
primary assurance components (see Table 1 and Figure 1): technical level assurance, application
6
user level assurance, and business level assurance. Each of these three levels is discussed further
in the following subsections.
[Insert Table 1 about here]
[Insert Figure 1 about here]
Technical Level Assurance
Technical level B2B assurance deals with assisting decision makers in ensuring that the
necessary technical B2B elements are in place and that integration with external and internal
applications is feasible given the availability of financial and technological resources. This
includes a variety of technical services such as selecting appropriate internal applications for
B2B linkages, integrating multiple trading partners, mapping customer/supplier data for direct
use in internal applications, ensuring that the business transaction process works and includes all
electronic transaction sets, and using appropriate B2B intermediaries to support the processes.
Review of B2B technical capability should also consider the level of integration with back office
systems and the reliability of those systems in ensuring integrity and security of the data captured
in B2B transmissions. (Khazanchi and Sutton 2001).
As defined, technical level assurance captures a range of risks identified in various
studies. The five key problems identified by Papazoglou and Tsalgatidou (2000) which hamper
widespread use of e-commerce all fit in this category (i.e. incomplete implementations, rigid
requirements for protocols, limited interoperability of systems, insufficient security, and lack of
integration with business models). The technical level also embodies more general concerns such
as cost effectiveness, reliability, security (Unal 2000), and trust in a partners control mechanisms
(Tan and Thoen 2002).
7
Technical level risks may be of the greatest concern to SMEs who have been forced to
adopt e-commerce with a limited number of partners (perhaps even one). It is also important to
larger organizations that have significant concerns over partners’ controls related to data integrity
and security which could lead to legal disputes over privacy and requirements for safe
maintenance of data.
Application-User Level Assurance
The application-user level relates to ensuring that decision makers’ choices and rationale
for B2B implementation are appropriate. Risks in this area focus on understanding potential
benefits of B2B linkages, assessing the current business environment and internal processes,
obtaining general information about B2B options, assessing organizational readiness for
adopting B2B, relying on paper-based transactions for internal processes, dealing with
impersonal nature of e-commerce transactions, and reliability of internal transaction processing.
Finally, questions about the adequacy in preparation of an organization’s staff for B2B activities
along with related education and training programs should be assessed.
Application-user level risks include risks previously identified in the literature such as
Angeles and Ravinder’s (2000) key factors for customers selecting among vendors for EDI
relationships—i.e. readiness for high-level EDI, trading partner flexibility, and communication,
and Iacovou et al.’s (1995) three factors affecting small organizations adoption of B2B ecommerce—i.e. perceived benefits of application, organizational readiness, and external pressure
to adopt.
These risks are of concern to SMEs who are considering adoption—including those under
coercive pressures. However, larger organization may be particularly concerned about these risks
relative to smaller business partners. Smaller organizations may not be able to integrate the
8
technology effectively in order to enhance flexibility in ordering and to further cut cycle times
across the value chain. Weaknesses of a partner organization in either of these latter areas could
warrant reconsideration by the dominant partner as to the long-term viability of the business
relationship.
Business Level Assurance
Business level risks relate to an organization’s ability to appropriately reengineer
traditional business processes to incorporate an e-commerce driven business . These risks may
center around a variety of issues including the appropriateness of e-commerce for an
organization, assessment of direct/indirect benefits actually being realized from e-commerce
usage, adherence to legal requirements (electronic orders, signatures, trading partner agreements,
information privacy laws, etc.), proper monitoring of data and transmission security/auditability,
and appropriateness of workflow procedures to achievement of efficiency gains. Accordingly,
internal control systems should be assessed for viability in assuring continuous maintenance of
controls over privacy of data, reliability of systems, and security of electronic transmissions.
(Khazanchi and Sutton 2001).
The importance of the business level dimension has been noted in part in other recent
research studies. Business model integration has been noted as a key concern by Papazoglou et
al. (2000) and Hempel and Kwong (2001). Kumar and van Dissel (1996) emphasize the
importance of individual partner’s performance when there is sequential interdependence among
a chain of partners. Breakdowns by a single partner will inevitably affect adjacent partners in the
sequential chain and may possibly impact all subsequent downstream partners in the value chain.
Assessment of business level risks would appear to be of critical importance to
organizations potentially at risk. All organizations should be concerned that their own systems is
9
properly implemented and that transactions are secure, electronic funds transfers are complete,
internal recording of business transactions are accurate and complete, and inefficiencies are
identified. At the same time, similar failures by business partners can also have major effects on
an organization. Thus, risk assessment of partner systems is also of great importance—
particularly as systems become increasingly complex and partners become increasingly
dependent on each other for successful execution of the supply chain.
Identifying Key Risk Factors
The three components of the B2B e-commerce assurance model appear to provide a
reasonable means for assessing risks associated with an organization’s B2B e-commerce
activities. While the model provides a good conceptual foundation for examining e-commerce
risks, understanding the key risk factors in each category is still general and non-specific.
Identifying a set of specific factors for each of the three risk components could enhance
the value of the model to managers and researchers assessing the riskiness of potential trading
partners. Before the risks can be quantitatively measured, the specific risk factors that should be
measured must be identified. The factors that make up the set of critical factors for assessing ecommerce risk within each of the three categories are unknown? Is there a consistent set of
factors applicable to most or all organizations?
In seeking to identify whether such a set of factors exist, this study adopts a philosophy
consistent with that of total quality management (TQM). Such a philosophy is based on the belief
that the best way to monitor and improve a process is to engage the individuals who regularly
perform the process in the development of quality factors (Havelka et al. 1998). These
individuals are in the best position to understand the key factors that affect a systems’ ability to
function properly and in a quality manner (Lampe and Sutton 1994a). In the case of e-commerce
10
activities, five key constituencies are identified as being in a position to monitor and assess the
processes. The three internal constituencies are e-commerce developers, IS security staff, and IT
auditors. The external constituencies are e-commerce consultants (who develop, implement and
maintain e-commerce systems) and external auditors (IT audit specialists who assess risk in
assurance and audit engagements).
Four key results are critical for risk factors identified by these various constituencies to
be generalizable to other organizations. These four key results are:
1. Consistency among internal constituencies as to the critical risk factors for each of
the three model components.
2. Agreement among internal constituencies as to the critical risk factors for each of
the three model components.
3. Consistency between internal constituencies and each of the external
constituencies as to the critical risk factors for each of the three model
components.
4. Agreement between internal constituencies and each of the external constituencies
as to the critical risk factors for each of the three model components.
A methodology consistent with the TQM philosophy is proposed for purposes of identifying key
risk factor. The proposed methodology is discussed in the following section.
STRUCTURED FOCUS GROUP METHODOLOGY
The research method used in this study has been adopted from Havelka et al. (1998). The
methodology was originally adapted from Adam et al. (1986) and subsequently refined and
validated in a series of studies examining key quality factors in auditing (Lampe and Sutton
1994a, 1994b, Sutton and Lampe 1991; Sutton 1993) and information requirements definition
11
(Havelka et al. 1998). The short form of the methodology as used by Havelka et al. is applied in
this study for risk factor identification in e-commerce environments. Given the differences
between quality factor identification and risk factor identification (albeit similar), validation
testing is also completed using the data aggregated from the focus groups in this study.
Focus Group Process
The risk factor identification methodology uses a blend of focus groups with the
structured approach of nominal group techniques (Van de Ven and Delbecq 1971; 1974). The
method is based on the TQM concept that the important factors embedded in a process that
impact success are most readily observable and identifiable by those individuals involved in the
process (Adam et al. 1986). A four step structured focus group conducted over a single threefour hour session is used in this study to identify key risk factors impacting successful ecommerce processes across the three dimensions of technical, application-user and business level
risk. The process is initially completed with internal constituency groups that include IS security
staff, e-commerce developers, and IT auditors. Follow-up sessions are conducted with external
constituency groups that include either external IT audit specialists or e-commerce consultants.
The first step in the structured focus group is an open forum by the group participants to
generally discuss their individual roles, the types of applications with which they have been
involved, their perspectives on the success of e-commerce ventures, and the impact of ecommerce on their particular organization’s business and overall industry. This open discussion
acclimates the various participants and group leaders (i.e. the researchers) to the terminology
used by the various participants and their general perspectives as they enter into the group
discussion. It also allows time to discuss the model for e-commerce and to assure there is
consensus on the meaning of each of the model’s components.
12
The second step in the structured focus group consists of a silent brainstorming session at
the individual level. The participants are provided with the component of interest (i.e. technical,
application-user or business level risk) and asked to identify all factors that they believe have an
impact on a successful e-commerce system/process. They are also instructed to consider all
factors regardless of whether they may be characteristic of implementations at the initial start-up
phase, integration phase, or mature phase (consistent with that prescribed by Khazanchi and
Sutton 2001). The participants are further instructed that at this point they should focus on
identifying all factors and not in trying to filter out any that they think might not be particularly
critical.1 The participants list out their risk factors on a sheet of paper that includes a heading
reminding them of the component of interest in the model at this stage of the process. The silent
generation period provides time for adequate reflection, social facilitation (i.e. the tension
created by watching others busily working and generating lists of risk factors), avoidance of
interruption, avoidance of prematurely focusing on the first ideas generated by the group,
sufficient time for search and recall, avoidance of competition, avoidance of status pressures,
avoidance of conformity pressures, and avoidance of choosing between ideas prematurely
(Delbecq et al. 1982).
The third step is a round-robin recording of the ideas generated. An aggregate list of the
participant’s identified risk factors is generated by taking one idea off each person’s list in a
continuous around the room pattern until all participants’ lists are exhausted. As each risk factor
is read out, one of the group leaders types the factor into a synthesizing document that is
projected to the front of the room where all participants can view the composite list of risk
factors. As each risk factor is read out, participants share in forming an understandable phrase
1
While participants are likely to focus on factors that mirror their personal experiences and, therefore, the industries
in which they participate, the subjects were not instructed to limit their listing of factors to those they have faced nor
those that would be specifically applicable to their industry.
13
representing the factor and an agreed upon definition of the factor (which is recorded off-screen
by one of the researchers). The benefits of the round-robin approach to listing risk factors
includes the following: equal participation in presentation of risk factor items, depersonalization
of ideas from personalities as the list gets combined and grows, ability to deal with a large
number of ideas, tolerance of conflicting ideas, encouragement of new risk factor generation
based on fellow participant’s ideas, and provision of a written record and glossary of the risk
factors presented (Delbecq et al. 1982). The second and third steps are briefly repeated to seek
new risk factors that are identified based on ideas that derive from other participants’ generated
lists of risk factors.
The fourth step is to evaluate the long list of risk factors and identify a subset of factors
that are considered particularly critical to the success of e-commerce processes. Each focus
group participant, at the individual level, first sorts the list of risk factors into a list of critical
factors versus those that are not critical risk factors. This process represents a Q-sorting approach
(Kerlinger 1986). Subsequently, individual participants rank each of the critical risk factors s/he
has selected based on importance to the model component being examined. This is an extreme
version of the Q-sort whereby each item is essentially placed into a classification by itself
(Sutton 1993).
The four-step procedure is repeated for each of the three components in the model. After
completion of the focus group, the researchers aggregate the rankings into a composite list of
critical risk factors; and this composite list is used in subsequent analysis. There is evidence to
support the use of this type of method for aggregating individual rankings into a composite group
rating when the intent of the research is to generate a true group preference (Huber and Delbecq
14
1972; Sutton 1993). Thus, the output of the structured focus group is a consensus set of critical
risk factors for each of the three components.
Validation of the Methodology for Risk Factor Identification
Recall from the discussion of the theoretical model that the robustness of the critical risk
factors identified is contingent in large part by whether there is consensus between different
organizations on the identified risk factors. The validity of the structured focus groups for risk
factor identification is predicated on the belief that groups from different organizations will
generate similar lists of critical risk factors for each of the model components. At the most
fundamental level, there should be agreement among individuals in similar positions across
different organizations—i.e. among IS security staff, e-commerce developers and IT auditors
from each of the three organizations participating in the study. Stated in null form, the hypothesis
relates specifically to the validation of the method and can be stated as:
H1: The risk factors selected as critical by one organization’s focus group will
be independent of those selected by another organization’s focus group.
The perceptions regarding relative importance of each of the critical risk factors is also important
as the two focus groups should not only feel similarly regarding the critical risk factors, but also
the relative importance of the critical factors (i.e. the ranking). As such, H2 relates to the ranking
of factors as a measure of the strength of agreement among the groups.
H2: The importance placed on each of the risk factors selected as critical by one
organization’s focus group will be independent of the importance placed on
each of the risk factors by other organizations’ focus groups.
From an analysis standpoint, each of the three components of the model is addressed
separately since the focus groups also rank factors in each of the model components separately.
This leads to three testable hypotheses:
H2a: The importance placed on each of the technical level risk factors selected as
critical by one organization’s focus groups will be independent of the
15
importance placed on each of the risk factors by other organizations’ focus
groups.
H2b: The importance placed on each of the application-user level risk factors
selected as critical by one organization’s focus groups will be independent
of the importance placed on each of the risk factors by other organizations’
focus groups.
H2c: The importance placed on each of the business level risk factors selected as
critical by one organization’s focus groups will be independent of the
importance placed on each of the risk factors by other organizations’ focus
groups.
While the relationship among each of the organizations is important to the validity of the
structured focus group in generating e-commerce risk factors, the breadth of the validity would
be enhanced to the degree that an external e-commerce consulting group and an external IT audit
group are also consistent in the identification of key risk factors. Similar to the examination at
the internal constituency level, the relative ranking of risk factors further adds validity to the
selected factors. Stated in null form, the hypotheses again relate specifically to extending the
validation of the method and can be stated as:
H3: The risk factors selected as critical by organizations’ focus groups will be
independent of those selected by an e-commerce consultants’ focus group.
H4: The importance placed on each of the risk factors selected as critical by
organizations’ focus groups will be independent of the importance placed on
each of the risk factors by an e-commerce consultants’ focus group.
H5: The risk factors selected as critical by organizations’ focus groups will be
independent of those selected by an external IT audit focus group.
H6: The importance placed on each of the risk factors selected as critical by
organizations’ focus groups will be independent of the importance placed on
each of the risk factors by an external IT audit focus group.
From an analysis standpoint, each of the three components of the model is addressed
separately when considering the relative importance of the factors in H4 and H6. This leads to
six testable hypotheses:
16
H4a: The importance placed on each of the technical level risk factors selected as
critical by organizations’ focus groups will be independent of the
importance placed on each of the risk factors by an e-commerce consultants’
focus group.
H4b: The importance placed on each of the application-user level risk factors
selected as critical by organizations’ focus groups will be independent of the
importance placed on each of the risk factors by an e-commerce consultants’
focus group.
H4c: The importance placed on each of the business level risk factors selected as
critical by organizations’ focus groups will be independent of the
importance placed on each of the risk factors by an e-commerce consultants’
focus group.
H6a: The importance placed on each of the technical level risk factors selected as
critical by organizations’ focus groups will be independent of the
importance placed on each of the risk factors by an external IT audit focus
group.
H6b: The importance placed on each of the application-user level risk factors
selected as critical by organizations’ focus groups will be independent of the
importance placed on each of the risk factors by an external IT audit focus
group.
H6c: The importance placed on each of the business level risk factors selected as
critical by organizations’ focus groups will be independent of the
importance placed on each of the risk factors by an external IT audit focus
group.
Focus Group Participants
Completion and validation of the methodology for identifying key e-commerce risk
factors within the framework put forth by Khazanchi and Sutton (2001) requires the involvement
of internal constituency groups involved in the development, implementation and evaluation of
corporate B2B systems and along with the relationships with external constituencies (ecommerce consultants and external IT auditors) who facilitate the development and
implementation, and/or the audit and assurance of such systems. The internal constituency
groups were designed to capture the perspectives of e-commerce developers, IS security staff,
17
and internal IT/B2B auditors. Three internal constituency groups were selected for participation
in an effort to get a diversified set of perspectives.
1. Group 1 was from a large insurance company that interacts with a variety of service
providers, medical entities, and co-insurance partners. Five individuals participated in the
group process representing the director of internal audit, an IS security officer, an audit
supervisor, and two information systems managers (one of whom was an EDI specialist).
2. Group 2 was from a large food manufacturer that interacts with a variety of suppliers as
well as a variety of customers including a number of very large chains such as Wal-Mart.
Seven individuals participated in the group process with two from web services, an
information security officer, auditor in charge of infrastructure reviews, one B2B
application developer, one tech support officer for B2B applications and the director of
internal audit.
3. Group 3 was from a large railroad and transportation company that deals with a large
number of customers and ancillary transportation providers. Five individuals participated
including three IT auditors with B2B experience, a project manager for e-commerce
implementations, and a programmer involved heavily with the development and
maintenance of the pricing exchange.
The external constituency groups were selected based on the desire for additional external risk
factor validation via e-commerce consultants and external IT auditors involved in assessing the
impact of IT risk on overall enterprise risk. Two additional groups participated:
1. Group 4 was from a regional e-commerce consulting company specializing in the design
and implementation of e-commerce systems. Four individuals participated including the
18
lead project managers in infrastructure implementation, database management,
application design, and marketing services.
2. Group 5 was from a Big Four audit firm and consisted primarily of the partners and
managers in IT audit of one region in the U.S. Six individuals participated including one
partner, four managers, and the associate CIO for global firm operations.
The five groups (three internal constituency and two external constituency) provide a rich
source of data for identifying the key e-commerce risk factors. Active involvement by all group
members yielded a long list of potential key factors that should be considered in assessing the
business risk evolving from B2B relationships with business partner organizations.
RESULTS OF THE VALIDATION TESTING
Data analysis was conducted in accordance with the two-phase validation procedure
examining first, the consistency among internal constituency groups, and second, internal
constituency groups’ consistency with external constituency groups. Specifically, Phase I
validation using internal constituency groups relates to H1 and H2, while Phase II validation with
external constituency groups relates to H3-H6. The results provide support for rejecting all six
null hypotheses, consistent with the expectations underlying the applied methodology.
Phase I Validation: Internal Constituency Groups
The structured focus group processes were completed independently with each of the
three internal constituency groups consisting of a variety of e-commerce interested participants.
The first condition necessary for reliance on the identified set of key factors is the existence of a
high level of agreement between different groups in different corporate environments as to the
factors selected as critical. To test H1, a chi-square test of independence is used. Because the chisquare test is designed to compare two groups, all pair wise combinations of the three groups are
19
tested. The chi-square test of independence uses a 2x2 contingency table that focuses on both the
commonalities (items selected/not selected by both groups) and the differences (items selected
by only one of the two groups) between the two organizations being compared. The results
provided in Table 2 indicate significant agreement between all pair wise combinations in the
selection of key e-commerce risk factors, leading to the rejection of H1.
[Please insert Table 2 about here]
While agreement on the selection of factors captures one dimension of consistency
between the groups, a second dimension should also be considered—the relative ranking placed
on each identified factor by each of the groups. To test H2, a Spearman’s rank correlation test is
used. Because the groups ranked each of the three dimensions of the risk model (i.e. technical
level, application user level, and business level) separately, the Spearman’s rank correlation test
must accordingly be used within each dimension (i.e. H2a-H2c). Using rank values provides
greater statistical power than does categorization by selected/not selected as is used in the chisquare test. Similar to the tests for H1, all pair wise combinations of the three groups are tested.
The results provided in Table 3 indicate significant agreement on rankings of the key ecommerce risk factors between all pair wise combinations of internal constituency groups,
leading to the rejection of H2a-H2c, and therefore H2.
[Please insert Table 3 about here]
In combination, the results from testing H1 and H2 indicate that there is agreement among
groups from diverse organizations (i.e. insurance, food manufacturing, and
transportation/logistics) on a set of critical factors affecting the overall enterprise risk from ecommerce relationships. This also implies that these critical factors can be used to assess the risk
inherent in conducting e-commerce relationships with other organizations. These critical factors
20
should also be important to the external auditors of such organizations in making their enterprise
risk assessments during audit planning and by e-commerce consultants who inherit related
liability during development and implementation of such systems. This leads to the Phase II
validation testing.
Phase II Validation: External Constituency Groups
Two external constituency groups were identified as major stakeholders having interest in
e-commerce risks associated with corporate systems—consultants and external IT auditors. First,
many such systems are developed and implemented by outside consultants. By assuming a
certain level of responsibility for specific applications, e-commerce consultants are expected to
be particularly alert to risks arising at the technical level and application-user level. Second and
as noted earlier, IFAC international audit statements specify the importance of the external
auditor assessing the implications of the risk from e-commerce systems on the assessment of
overall enterprise risk during a financial statement audit. External auditors are expected to be
particularly concerned with business level risks and the corresponding effect on the financial
condition of the organization.
The results of the structured focus groups conducted with e-commerce consultants and
external IT auditors are compared individually with the aggregate results of the three internal
constituency groups. Testing of H3 and H5 are the equivalent to that used in testing H1, but
examine agreement by the e-commerce consultant group with the internal constituency groups
and by the external IT auditor group with the internal constituency groups. As such, the same
chi-square test of independence using a 2x2 contingency table focusing on both the
commonalities (items selected/not selected by the internal constituency groups and the
consultants [external IT audit] group) and the differences (items selected by only one of the
21
constituencies) is used. The results displayed in Table 2 indicate significant agreement by both
the consulting group (H3) and the external IT audit group (H5) with the internal constituency
groups, leading to the rejection of both hypotheses.
While agreement on the selection of factors captures one dimension of consistency
between the groups, with both the e-commerce consultants and the external IT auditors, the
relative ranking of each identified factor remains an important dimension for assessing
agreement. As such, Spearman’s rank correlation tests are again used to test H4 and H6
(agreement between the e-commerce consultant group and external IT auditor group,
respectively, with the internal constituency groups). The tests are run separately for each level of
the risk model (i.e. technical, application-user, and business) based on the separate ranking
processes (i.e. H4a-H4c and H6a-H6c). The results provided in Table 4 indicate significant
agreement on rankings of the key B2B e-commerce risk factors between the e-commerce
consultant group and the internal constituency groups for both the technical level (H4a) and
application-user level (H4b), but not significantly related for business level (H4c) risks. The
results provided in Table 4 also indicate significant agreement on rankings of risk factors
between the external IT audit group and the internal constituency groups for technical level risks
(H6a), but only marginally significant agreement at the application-user level (H6b) and business
level (H6c). While there is mixed evidence, on an overall basis there does seem to be reasonable
support for rejecting H4 and H6.
[Please insert Table 4 about here]
In combination, the results for testing H3-H6 indicate that there is a set of risk factors that
can be agreed upon between internal constituency groups and varied external constituency
groups (i.e. e-commerce consultants and external IT auditors) that represent the critical factors
22
affecting the overall risk from B2B e-commerce relationships. This reinforces the belief that a set
of key risk factors exists that could be used in assessing the risk inherited from such
relationships. These risk assessments should be of importance to both internal and external
stakeholders.
CRITICAL B2B E-COMMERCE RISK FACTORS
While the robustness of the methodology for identifying critical e-commerce risk factors
is important to adding legitimacy to the identified factors, the primary purpose of this study is to
identify the critical set of factors. Completion of the five sets of focus groups resulted in the
identification of 49 key risk factors with 18 at the technical level (see Table 5), 16 at the
application-user level (see Table 6), and 15 at the business level (see Table 7).
[Please insert Tables 5, 6 & 7 about here]
The technical level factors are listed in Table 5 along with a marking of each of the items
identified as critical by each of the participant groups. Note that consistent with the test results
for consistency 8 of the 18 factors are identified by at least 3 of the 5 groups. A review of the
factors selected by the most groups is indicative of broad concern over security of access to
applications and networks along with the appropriate level of expertise and change management
controls to ensure continued security. Also of interests are the concerns by external
constituencies over robustness of systems over time (both in terms of systems and personnel) that
are not as prevalent among internal constituencies.
The application-user level factors along with a marking of each of the items identified as
critical by each of the participant groups are listed in Table 6. Consistent with the test results for
consistency, 8 of the 16 factors are identified by at least 3 of the 5 groups. The factors at this
level are less concentrated than for the technical level factors as a broad range of application
23
related issues are identified by multiple groups as critical, including: staffing issues and
management champions, architecture compatibility and capacity issues, partner benefits and
market sustainability, and testing for/controls over application reliability. While the external
constituencies have more unique factors at the technical level (see Table 5), they appear to be
much more in-line with the internal constituency groups in selecting critical risk factors at the
application level.
The business level factors along with the marking of each of the items identified as
critical by each of the participant groups are listed in Table 7. Consistent with the results of the
tests for consistency, the factor identification at the business level yields good consistency
among the groups for identification of key factors (9 of the 15 factors are identified by 3 or more
groups). As would be expected, broad range of issues are covered at the business level, including
regulatory, legal, cost/benefit analysis, business process integration, due diligence, risk
management, monitoring controls and management leadership in IT. Yet, despite the breadth
there is strong agreement on the critical factors affecting business level risks. On the other hand,
as might be expected, there is not necessarily strong agreement between internal constituencies
and external constituencies as to the relative importance of the individual factors identified.
DISCUSSION AND IMPLICATIONS
In the emerging Internet-driven B2B environment, the true benefits appear to come from
tight collaboration with trading partners (Lee et al. 2003), but at the same time significant
enterprise risks emerge from the corresponding increased dependence on a smaller set of trading
partners (Khazanchi and Sutton 2001; McIvor et al. 2003; Sutton and Hampton 2003). While
prior research raises many concerns and recognizes a variety of general risk factors related to the
tight coupling of interorganizational systems (e.g., Papazoglou et al. 2000; Unal 2000; McIvor et
24
al. 2003; Hempel and Kwong 2001; Westland 2002; Kumar and van Dissel 1996), the extant
research does not provide a focused examination of specific factors that can be utilized by
corporate chiefs for effective enterprise risk management, nor for auditors and other monitoring
organizations that must evaluate the riskiness of B2B activities to the viability of the
organization.
The research reported in this paper focuses on the identification of critical factors that can
be used by management, auditors and other related parties to monitor and assess the overall
enterprise risk arising from B2B interaction with a particular focus on interorganizational
systems. The study applies the Khazanchi and Sutton (2001) framework for B2B assurance,
focusing across the three levels of risk: (1) technical level, (2) application-user level, and (3)
business level risks. Based on a series of structured focus groups with internal constituency
groups representing information systems security, internal IT audit, and e-commerce
development and representing three diverse industries, a set of critical factors were identified for
each of the three levels in the framework. The set of critical factors were further refined and
validated using two external constituency focus groups (e.g. e-commerce consultants and
external IT auditors). The results of the study show strong consistency between all of the groups
in the identification of critical risk factors, strong agreement between internal constituency
groups on the relative importance of factors. The results also reflect some difference between
internal constituency groups and external constituency groups as to the relative importance of the
factors. These statistical results support the desired objective of identifying a set of critical
factors that are applicable across a broad range of organizations having concerns related to ecommerce activities. The 49 critical factors consisting of 18 technical level factors, 16
application user level factors and 15 business level factors would appear to provide broad
25
coverage of the important factors to be considered while at the same time maintaining a
relatively parsimonious set of factors at each level.
There are limitations to the research that should be considered when reviewing the output
of the factor identification process. First, application of the structured focus group methodology
necessitates the use of small groups. While attempts were made to gather data from a
comprehensive set of constituencies within each of the internal and external groups,
generalizations to other organization members and to other organizations cannot be assured.
However, the use of a diverse set of organizations from different industries along with two
unique external entities (e-commerce consultants and external IT auditors) should help minimize
this risk. Second, consensus based measures do not necessarily assure accuracy even when
highly experienced and knowledgeable participants are included. Third, many of the factors that
have been identified have been previously noted in prior research, the business press and
textbooks. However, many of the factors that were excluded as key risk factors by the
participating groups have also been identified in such sources. The research presented here
provides the additional information necessary to better understand the relative importance of
various risk factors mentioned in various publications and provides guidance to managers, ecommerce developers and auditors on the selection of a more parsimonious set of factors that
capture the key risk dimensions. The results provide insights that should be useful to managers,
developers, auditors and other researchers.
Our results should be of particular value to both corporate chief officers in addressing
enterprise risk management concerns and information systems managers concerned with secure
and effective interorganizational systems. While the factors provide a specific frame for viewing
B2B risks, measures for each of the factors will need to be developed and tailored to the specific
26
interorganizational system of interest. Consideration should also be given as to which factors
might be automated, which factors require human monitoring, and how this human monitoring
might take place. For those measures that can be automated, continuous assurance mechanisms
should be considered. This type of automated continuous assurance would seem particularly
feasible at the technical level. For those factors requiring human monitoring, consideration
should be given to whether such monitoring is possible and most desirable from internal IT
auditors examining trading partners systems and operations, external auditors or other
independent providers assure/certify trading partners, or some other alternative method. The key
is that it would appear to be critical that corporate chief officers and information systems
managers consider the risks that exude from interorganizational systems and take appropriate
steps to mitigate such risks to an acceptable level.
With the recent global spate of corporate frauds and mismanagement, there is certainly a
heightened focus on overall enterprise risk management. The focus on enterprise risk
management goes beyond just the concerns of corporate chief officers to the auditors who are
saddled with the responsibility to protect the public interest. The results of this study provide a
framework of e-commerce risk factors that should be considered under the broad guidelines of
the IFAC audit statements on e-commerce risk assessment. Further, in the U.S. where national
standards take precedence in guiding audits, the results provide guidance for further revision of
SAS 70 on assurance over service organizations and SAS 94 on the impact of IT on internal
control systems to encompass the impact of e-commerce trading partners on enterprise risk—a
step that the recent release of SAS 98 intended to update the standards to address a changing risk
environment did not encompass. Yet, clearly contemporary audit approaches that our focused on
27
business measurement and enterprise risk models should include consideration of the risks
absorbed from such interorganizational relationships.
There are also implications for researchers as further research on e-commerce risk is still
of great need. While the research presented in this paper documents specific risk factors across
each of the three levels in the assurance framework, there may be other characteristics of trading
partner relationships that provide insight into why such risks fluctuate. For instance, Hart and
Saunders (1997) found that differences in trust and power within EDI-based relationships were
related to the diversity of transactions used between trading partners and imbalances could affect
voluntary use of EDI. Such factors certainly could also influence many of the identified risk
factors as trading partners weigh the expense of maintaining secure, integrated, and wellmanaged interorganizational systems. Understanding these relationships would be highly
beneficial in attempts to control variations in risks that may affect overall enterprise risk—
particularly to the degree that such factors help assess risks associated with a potential trading
partner prior to entering into a relationship.
Design science based research examining the methods for implementing continuous
assurance systems for automated monitoring would also be beneficial. One challenge for
organizations wishing to implement measures of these risk factors through an automated
monitoring process is to determine a feasible means by which to implement monitoring across
internet linkages, embedded in trading partners’ systems, without increasing the trading partners’
risk of systems failure or reduced integrity. Instantiation of a working system could aid practical
implementation by helping to establish feasible alternatives.
28
REFERENCES
Adam, E.E., Jr., Hershauer, J. and Ruch, W. Productivity and Quality Measurement as a Basis
for Improvement. University of Missouri College of Business Research Center,
Columbia, Missouri, 1986.
Angeles, R. and Ravinder, N. “An Empirical Study of EDI Trading Partner Selection Criteria in
Customer-supplier Relationships,” Information & Management (37), 2000, pp. 241-255.
Banham, R. 2003. Fear factor: Sarbanes-Oxley offers one more reason to tackle enterprise risk
management. CFO Magazine (June 1).
Delbecq, A.L., Van de Ven, A.H. and Gustafson, D.H. “Guidelines for Conducting NGT
Meetings,” in Organizational Behavior and the Practice of Management (4th Ed.), D.R.
Hampton, C. E. Summer, and R.A. Webber (eds.), Scott, Foresman, and Company,
Glenview, IL, 1982, pp. 279-298.
Greiger, M. “Electronic Marketplaces: A Literature Review and a Call for Supply Chain
Management Research,” European Journal of Operational Research (144), 2003, pp.
280-294.
Grover, V., Teng, J.T.C. and Fiedler, K.D. “Investigating the Role of Information Technology in
Building Buyer-Supplier Relationships,” Journal of the Association for Information
Systems (3), 2002, pp. 217-245.
Hart P.J. and C.S. Saunders (1997) Power and trust: Critical factors in the adoption and use of
electronic data interchange. Organization Science 8(1): 23-42
Havelka, D., Sutton, S.G. and Arnold, V. “A Methodology for Developing Measurement Criteria
for Assurance Services: An Application in Information Systems Assurance,” Auditing, A
Journal of Practice & Theory (17), 1998, pp. 73-92.
Hempel, P.S. and Kwong, Y.K. “B2B e-Commerce in Emerging Economies: i-metal.com’s Nonferrous Metals Exchange in China,” Journal of Strategic Information Systems (10), 2001,
pp. 335-355.
Huber, G. and Delbecq, A.L. “Guidelines for Combining the Judgements of Individual Group
Members in Decision Conferences,” Academy of Management Journal (15), 1972, pp.
XXXXX.
IFAC (2001) Electronic Commerce Using the Internet or Other Public Networks - Effect on the
Audit of Financial Statements (Proposed International Auditing Standard—International
Federation of Accountants) October.
IFAC (2002a) e-Business and the Accountant, (International Federation of Accountants), March.
29
IFAC (2002b) International Audit Practice Statement 1013, (International Federation of
Accountants).
Iacovou, C.L., Benbasat, I. and Dexter, A.S. “Electronic Data Interchange and Small
Organizations: Adoption and Impact of Technology,” MIS Quarterly (December), 1995,
pp. 465-485.
Katz, D.M. 2003. What you don’t know about Sarbanes-Oxley: Snares, pitfalls, and trapdoors.
CFO.com (April 22).
Khazanchi, D. and Sutton, S.G. “Assurance Services for Business-to-Business Electronic
Commerce: A Framework and Implications,” Journal of the Association for Information
Systems (1), 2001, pp. 1-53.
Kumar, K. and van Dissel, H.G. “Sustainable Collaboration: Managing Conflict and Cooperation
in Interorganizational Systems,” MIS Quarterly (September), 1996, pp. 279-300.
Kurnia, S. and Johnston, R.B. “The Need for a Processual View of Inter-organizational Systems
Adoption,” Journal of Strategic Information Systems (9), 2000, pp. 295-319.
Lampe, J.C. and Sutton, S.G. Developing Quality Measurement Systems for Internal Auditing
Departments, Institute of Internal Auditors Research Foundation, Altamonte Springs,
Florida, 1994a.
Lampe, J.C. and Sutton, S.G. “Evaluating the Work of Internal Audit: A Comparison of
Standards and Empirical Evidence,” Accounting and Business Research (Autumn),
1994b, pp. 335-348.
Lee, S.C., Pak, B.Y. and Lee, H.G. “Business Value of B2B Electronic Commerce: The Critical
Role of Inter-firm Collaboration,” Electronic Commerce Research and Applications (1),
2003, www.ComputerScienceWeb.com.
Massetti, B. and Zmud, R. “Measuring the Extent of EDI Usage in Complex Organizations:
Strategies and Illustrative Examples,” MIS Quarterly (September), 1996, pp. 331-345.
McIvor, R., Humphreys, P. and McCurry, L. “Electronic Commerce: Supporting Collaboration
in the Supply Chain?” Journal of Materials Processing Technology (6736), 2003, pp. 1-6.
Papazoglou, M.P., Ribbers, P. and Tsalgatidou, A. “Integrated Value Chains and Their
Implications from a Business and Technology Standpoint,” Decision Support Systems
(29), 2000, pp. 323-342.
Papazoglou, M.P. and Tsalgatidou, A. “Editorial: Business to Business Electronic Commerce
Issues and Solutions,” Decision Support Systems (29), 2000, pp. 301-304.
Shin, K. and Leem, C.S. “A Reference System for Internet Based Inter-enterprise Electronic
Commerce,” The Journal of Systems and Software (60), 2002, pp. 195-209.
30
Sutton, S. G. “Toward an Understanding of the Factors Affecting Audit Quality,” Decision
Sciences (January-February), 1993, pp. 88-105.
Sutton, S.G. and C. Hampton. 2003. Risk assessment in an extended enterprise environment: redefining the audit model. International Journal of Accounting Information Systems 4(1).
Sutton, S.G. and Lampe, J.C. “A Framework for Evaluating Process Quality for Audit
Engagements,” Accounting and Business Research (Summer), 1991, pp. 275-288.
Tan, Y.H. and Thoen, W. “Formal Aspects of a Generic Model of Trust for Electronic
Commerce,” Decision Support Systems (33), 2002, pp. 233-246.
Teo, H.H., Wei, K.K. and Benbasat, I. “Predicting Intention to Adopt Interorganizational
Linkages: An Institutional Perspective,” MIS Quarterly (27:1), 2003, pp. 19-49.
Ünal, A. “Electronic Commerce and Multi-enterprise Supply/Value/Business Chains,”
Information Sciences (127), 2000, pp. 63-68.
Van de Ven, A.H. and Delbecq, A.F. “Nominal versus Interacting Group Processes for
Committee Decision Making Effectiveness,” Academy of Management Journal (14:2),
1971, pp. 203-212.
Van de Ven, A.H. and Delbecq A.F. “The Effectiveness of Nominal, Delphi, and Interacting
Group Decision Making Processes,” Academy of Management Journal (17:4), 1974, pp.
605-621.
Ware, L.C. 2002. Security and e-business will dominate 2003 IT spending. Computerworld
(December 2).
Westland, J.C. “Transaction Risk in Electronic Commerce,” Decision Support Systems (33),
2002, pp. 87-103.
Yost, P. “E-Business Web Portals and Their Role in Supply Chain Management,” Presentation at
Idea to Action: Numetrix’99, Atlanta, September 1999.
31
Figure 1: B2B Assurance Services (Khazanchi & Sutton 2001)
B2B ASSURANCE SERVICES
Application-User Level
Business Level
32
Technical Level
Table 1. B2B Assurance Services (Khazanchi and Sutton 2001)
Category of
Assurance
Purpose of Assurance
ApplicationUser Level
The services at this level will focus on assuring that trading partners trust and use EDI for
conducting B2B commerce. This may include assurance issues relating to establishing relationships
with new trading partners, developing “good business practices” and related policies.
Business
Level
The services at this level will focus on assuring that business processes, internal controls, and
policies are amenable to EDI adoption and that the processes are altered to allow for seamless
integration with the EDI application. This will include addressing legal, privacy of data, and
administrative issues for conducting reliable, secure and safe electronic commerce with trading
partners, transmission security and auditability of B2B (EDI) transactions.
Technical
Level
The services at this level will focus on assuring that all technical elements of EDI are in place and
that EDI is seamlessly integrated with internal applications. This will include issues relating to
transaction integrity, choice of applications, expansion of trading partner base and transaction
volume, system reliability, data security (risk assessment) and encryption, and transmission error.
33
TABLE 2
Results of Chi-Square Test of Independence for Key Factor Selection (H1, H3, H5)
Internal Constituency (Corporate) Groups (H1)
Organization 1 vs. Organization 2
Organization 1 vs. Organization 3
Organization 2 vs. Organization 3
External Constituency Groups (H3 and H5)
Internal Constituency Groups vs. External Audit
Group
Internal Constituency Groups vs. E-commerce
Consultant Group
34
Chi-Square
31.622
12.119
11.943
p-value
<.001
.007
.008
48.083
<.001
44.567
<.001
TABLE 3
Results of Spearman Rank Correlation Tests for Internal Constituency Groups (H2)
Technical Level (H2a)
Organization 1 vs. Organization 2
Organization 1 vs. Organization 3
Organization 2 vs. Organization 3
Application-User Level (H2b)
Organization 1 vs. Organization 2
Organization 1 vs. Organization 3
Organization 2 vs. Organization 3
Business Level (H2c)
Organization 1 vs. Organization 2
Organization 1 vs. Organization 3
Organization 2 vs. Organization 3
35
Spearman’s rho
.601
.716
.431
p-value
<.001
<.001
.003
.293
.426
.443
.033
.003
.002
.588
.562
.681
<.001
<.001
<.001
TABLE 4
Results of Spearman Rank Correlation Tests for External Constituencies (H4, H6)
Technical Level
Internal Constituency Groups vs. E-commerce Consultants
Group (H4a)
Internal Constituency Groups vs. External IT Auditors Group
(H6a)
Application-User Level
Internal Constituency Groups vs. E-commerce Consultants
Group (H4b)
Internal Constituency Groups vs. External IT Auditors Group
(H6b)
Business Level
Internal Constituency Groups vs. E-commerce Consultants
Group (H4c)
Internal Constituency Groups vs. External IT Auditors Group
(H6c)
36
Spearman’s rho
.333
p-value
.018
.324
.021
.717
<.001
.250
.060
.169
.177
.259
.076
TABLE 5
Critical Technical Level B2B E-Commerce Risk Factors
Critical Risk Factor
Change management processes in place to assure maintenance of security and integrity of
systems as technology evolves rapidly.
Trading partner’s security over all networks and network interactions ensure transmission
integrity and provide guaranteed delivery transaction to the correct trading partner.
Technology sophistication/expertise differential between trading partners and related
selection of appropriate standards and hardware/software by the right people in this trading
partner’s organization.
Trading partner’s maintenance of data accuracy during systems conversion and application
usage.
Completeness and accuracy of trading partner’s data processing activities.
Metrics related to capacity, resiliency, and monitoring in order to better predict/control
performance by trading partner.
Security of communication technology (infrastructure) --including vulnerability of ISP and/or
public internet, vulnerability to malicious code (e.g. viruses), security vendors expected
survival and the trading partner’s general security model.
Trading partner’s vulnerability to loss of availability of data, systems, applications, etc.,
whether loss is accidental, intentional, or by poor design.
Trading partner’s setting of appropriate user profiles to assure information is appropriately
compartmentalized by information types and classified by access levels.
Controls to enforce compliance with regulatory requirements and to enforce regulations
Comprehensive access management to applications/operating systems protected via
controls (e.g. firewalls) in place to assure confidentiality, availability, and integrity (e.g.
unauthorized access).
Channel security through appropriate controls (e.g. encryption implemented according to
regulations) including validation and authentication of transaction partner.
Ease of transition of information to new B2B systems, ease of integration with trading
partner's systems, consistency in methods of partner, and ability to efficiently route B2B
transactions to the right internal applications.
Flexibility and scalability of the trading partner’s system (hardware/software independence).
Redundancy and failover of trading partner’s systems (in relation to downtime tolerance).
Adequacy of trading partner’s disaster recovery plan.
Adequate staff expertise available on an as-needed basis.
Comprehensive systems documentation of trading partner’s systems.
37
Transportation
& Logistics
Company
Insurance
Company
Food
Manufacturer
Company
E-Commerce
Consultancy
Firm
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
External
IT Audit
Firm
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
TABLE 6
Critical Application-User Level B2B E-Commerce Risk Factors
Critical Risk Factor
Appropriate level of training for trading partner’s users and related cost constraints.
Will the target trading partner (TP) use a proposed B2B system (considering such issues of
whether there is a champion for the project, sufficient IT sophistication to integrate within
TP's systems environment, and ease of use of application)?
When upgrading systems based on new technologies or business partner request, the
trading partner has sufficient coordination and change control procedures in place to
maintain reliability and protect transaction validation procedures.
Trading partner’s understanding of and agreement on data structure/scope/business rules
for exchange of information.
Is there benefit of B2B ventures to the trading partner and is the e-commerce marketplace
sustainable?
Clear and sufficient contract documentation on policies, procedures, connectivity guidelines,
limitations, review plan, etc. (Service Level Agreements).
Application controls in place for completeness, accuracy, and processing integrity (i.e.
trading partner’s applications function as intended).
Trading partner’s implementation of new B2B applications include testing for assurances on
hardware/software capability to support applications, availability of supporting applications
24/7, and performance and capacity of data exchange.
Third party assurance of transaction validity.
Marketing cost to sell the trading partner on a given B2B application
Privacy of data agreements.
Alignment of trading partner’s business processes with implemented B2B e-commerce
technologies.
Adequacy of the security over access to trading partner’s business application systems.
Inaccurate, inadequate or outdated documentation on systems software/hardware provided
by trading partner.
Trading partner’s inability to have an enterprise view of the full range of trading partner
relationships.
Trust in trading partner (internal or external).
38
Transportation
& Logistics
Company
Insurance
Company
Food
Manufacturer
Company
E-Commerce
Consultancy
Firm
External
IT Audit
Firm
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
TABLE 7
Critical Business Level B2B E-Commerce Risk Factors
Critical Risk Factor
Understanding by trading partner (TP) of their business processes, where e-commerce fits
into those processes, value of business process integration with TPs, and where benefits are
derived.
Trading partner’s ability to assess the use/success of technology and the benefits of B2B
implementation/technology investment (including return on investment).
Trading partner’s costs of meeting regulatory requirements and their organization's
understanding of associated risks of non-compliance (including inter- and Intra- state
compliance issues).
Trading partner’s technical understanding at a level that facilitates creation of a
transformational vision for change and the ability to implement successful change
management strategies to achieve objectives, gain acceptance, and support sustainability of
the change.
Trading partner’s understanding of the intended functionality of a system at the
analysis/requirements stage and tying of the system to business processes that are evolved
or engineered accordingly to meet the business objective.
Trading partner’s level of adherence to contractual requirements including such things as
product volume, sales prices, time/service commitments, and settlement (including legal
agreements such as non-repudiation and the level of legal binding).
Trading partner’s due diligence in implementing B2B relationships at the business,
technology and security levels to assure users understand data
classification/ownership/security when handling partner data and the partner maintains
appropriate segregation of data to appropriate users.
Trading partner’s understanding of risks associated with their projects and accordingly
executing effective project management.
Trading partner’s understanding of the technical complexities and associated costs of B2B
development, implementation, and maintenance; and the legal ramifications, costs of
implementing vs. not implementing non-repudiation agreements, costs of new business
rules, and loss of personal marketing contacts.
Trading partner’s team expertise for guiding all aspects of B2B e-commerce projects along
with training for project teams and users.
Trading partner’s broad management involvement in IT/business planning while maintaining
independence in the selection of technology preferences.
Trading partner’s integration of applications into organizational procedures and guidelines –
including comprehensive documentation.
Auditability of trading partner’s system based on effective monitoring controls and audit trail
(history of electronic data, updates, changes).
39
Transportation
& Logistics
Company
Insurance
Company
Food
Manufacturer
Company
●
●
●
●
●
●
●
●
●
●
E-Commerce
Consultancy
Firm
External
IT Audit
Firm
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Trading partner’s ability to protect a distinguished Brand in an e-commerce environment.
Trading partner’s resilience to a business interruption.
40
●
●