Agenda
Quality in IS Research:
Theory and Validation of
Constructs for Service,
Information, and System
Yi Ding
Detmar Straub
Workshop September 2007
University of Minnesota
„
Motivation
„
Literature Review : Delone-McLean IS
Success Model and Marketing Exchange
Theories
„
Problems with Existing Quality Constructs
„
Research Questions
„
Research Methods
-2-
Motivation
Motivation
„
Importance of quality dimensions in IS Success
research (e.g. Delone and McLean, 1992)
„
Lack of validation of instruments measuring quality
„
Significant increase in quality research in recent
years to meet organizational needs (e.g. Kettinger
and Lee, 2005; Parasuraman and Zeithaml, 2005)
„
Little work exploring relational links among
information quality, system quality, and service
quality
„
Atheoretical introduction of service quality into IS
Success Model (Delone and McLean, 2003)
„
No studies finding a clear link between service
quality and satisfaction
-3-
-4-
Delone & McLean IS Success
Categories (1992)
Delone & McLean Models
-5-
-6-
Sub-Constructs of Quality To Date
„
Taxonomy based on existing information theory:
Shannon and Weaver (1949)
„
System Quality – Technological (“technical”) Level
„
„
Accuracy and efficiency of the system which produces the
information, centered around characteristics of IT artifacts that
support service relationships
Measures of information system output
order to satisfy human needs, people and
organizations are compelled to engage in social and
economic exchanges with other people and
organizations”
„
Leonard L. Berry (1980): Service marketing
A
Service Quality
No theoretical definitions articulated
Performance oriented
Exchange oriented
„
Richard P. Bagozzi (1975): Social marketing and
exchange
“In
Information Quality – Semantic Level
„
New Service Theories Needed (Marketing
Exchange Theories)
Pseudo-relationship (co-generated by information system and
customer)
-7-
„
service is a “deed, act or performance”
A. Parasuraman, Valarie A. Zeithaml, & Leonard
L. Berry (1985): SERVQUAL
Three
well-documented characteristics of services –
intangibility, heterogeneity, and inseparability
-8-
IS Role In Service Scenarios
New Service Theories Needed (Marketing
Exchange Theories)
„
Barbara A. Gutek (1995): The dynamics of service
„
Service is “a special kind of interaction between a customer and
a provider … an interaction that usually involves the exchange
of money for service or goods”
J.S. Metcalfe (2001): Modern evolutionary economic
perspectives: An overview
„
Highly cost
efficient: economic
view / utilitarian
view of marketing
exchange
Customer
Information System
Customer Service
Customer
Information System
Customer Service
An exchange between two or more parties and a transformation
(potentially intangible) received by a customer
Henry Chesbrough, Jim Spohrer (2006): A research
manifesto for services science
Information System
It involves a negotiated exchange between a provider and an
adopter (supplier and customer) for the provision of
(predominately) intangible assets
The exchange is co-generated by both parties, and the process
of adoption or consumption is an integral part of the transaction
Customer
Customer Service
Not cost efficient:
labor intensive
service task
Customer
Customer Service
-9-
-10-
Existing (1) System, (2) Information,
(3) Service Quality Constructs
Problems with Existing Quality Constructs
„
„
There have been no complete tests of the DeLoneMcLean 1992 or 2003 IS Success Model (nomological
validity).
Each new researcher tends to develop his/her own
quality measures and criteria but validation effort is often
missing.
Construct
Bailey and Pearson
1983
Yoon and
Guimaraes 1995
Rai et al. 2002
Garrity et al. 2005
Flexibility of System
Ease of Learning
Response Time
Shell Quality
Perceived Ease of
Use
Interface Satisfaction
1
Construct
Jones and McLeod
1986
Teng and Calhoun
1996
Marble 2003
Shih 2004
Perceived importance of
information items
Timeliness
Information Overload
Performance Level
Relevance
Pitt and Watson 1995
Thompson et al.
1994
Marble 2003
Parasuraman and
Zeithaml 2005
SERVQUAL
Tangibles
Reliability
Assurance
Emphathy
Facilitating Conditions
Mutual Understanding
E-S-QUAL
Efficiency
System Availability
Fulfillment
Privacy
2
„
Most research focuses on understanding how
information quality and system quality influence
satisfaction.
Construct
3
-11-
Information System
-12-
Petter and McLean Meta-analysis Working Paper - 2006
Our Research Questions
Hypothesis
Summary
# of
studies
Sample
size
effect size
(corrected
reliability)
Use (+)
Weak
14
2083
0.28
0.59
System Quality
Strong
17
3496
Info. Quality
System Quality
Use (+)
Satisfaction (+)
Moderate
5
653
0.4
Info. Quality
Satisfaction (+)
Moderate
8
1860
0.36
Insignificant
5
777
0.14
Insignificant
2
255
0.36
Strong
13
2513
0.61
Strong
6
1265
0.63
Service Quality
Service Quality
Use
Satisfaction
System Quality
Info. Quality
Perceived Usefulness (+)
Perceived Usefulness (+)
RQ1: What are the items that capture all essential
characteristics of IS quality?
RQ2: Are the constructs of System Quality,
Information Quality, Service Quality distinct from
each other?
No hypothesis on Service Quality and Perceived Usefulness
System Quality
Intention to Use (+)
Information Quality
Service Quality
Intention to Use (+)
Intention to Use
Strong
11
2271
0.51
Strong
4
773
0.51
Not tested
0
0
-13-
RQ1-Questions of Content Validity
RQ3: What is the nomology of these constructs in
the IS Success Model?
-14-
RQ2 - Relationships among Constructs
Framing
Theory /
Theories
-15-
-16-
RQ3 – Test of Delone-McLean IS
Success Model ( 2003)
RQ3 - Our Alternative IS Success (Model 1)
Not tested
-17-
-18-
RQ3 - Our Alternative IS Success (Model 2)
RQ3 - Our Alternative IS Success (Model 3)
-19-
-20-
Research Methods for Our Study
„
Three Phases (following, to some extent,
Moore
and Benbasat, 1991)
1.
2.
3.
Phase 1 - Theorizing IS Quality:
Role of IS in Service
„
IS as intermediary (e.g., Amazon)
Theorizing IS quality (Content validity)
Instrument development and validation
(N=120)
Nomological validity tests (N>120)
(Web server)
(Web server)
-21-
-22-
Phase 1 - Theorizing IS Quality:
Role of IS in Service (continued)
„
IS as co-producer (e.g., Amazon)
Phase 1 - Theorizing IS Quality:
Role of IS in Service (continued)
„
IS as support for providing service (e.g.,
Amazon)
Information System
Customer
Customer
Customer Service
Customer Service
-23-
-24-
Information System
IS Service Scenario II – Service Provided By IS
Phase 2: Factor Analysis
„
Figure 2. Human IS
Service for End-Users
Questionnaire development
Items
„
resulting from phase 1
Factorial analysis
Formative
„
and reflective construct validation
Test of reliability for reflective constructs
Figure 3. Service
Provided by an IS
-25-
Phase 2 : Other Validity Tests
„
Examination of first and second order of models
for each construct
„
Discriminant validity
„
Convergent validity
„
Test using SEM
-27-
-26-
Phase 3: Nomological Validity Tests
„
Test of DeLone-McLean IS Success
Model
„
Examine wide ranges of theoretical
linkages among IS success constructs
-28-
Validity Checks in Phases
Phase 1
Phase 2
Questions
Phase 3
Content
Validity
Factorial
Validity
Construct
Validity
Nomological
Validity
-29-
-30-