Appari and Benaroch
Pricing the IT Investment Risks
Pricing the Risks of IT Investments: A Multifactor Approach
Ajit Appari, PhD Candidate (aappari@syr.edu)
Michel Benaroch, PhD (mbenaroc@syr.edu )
1. INTRODUCTION
Financial justification of Information Technology (IT) investments has become an imperative in
the face of increasingly uncertain business dynamics and rapid pace of technological
advancements. A rational investment decision, in the context of single investment or portfolio of
investments, requires examining both risk and returns of proposed investment (Tanirverdi and
Ruefli 2004). Recent research gives credence to the assertion that IT investments by nature are
risky and increase firm’s earnings volatility (Hunter, Kobelski, and Richardson 2003). IT
investments not only yield significant gain in shareholder’s wealth (Chatterjee, Pacini, and
Sambamurthy 2002), but also results in losses (Ettredge and Richardson 2003). Economic
valuation of IT investment could be performed using various financial tools, such as internal rate
of return, return on investment, net present value, and real options. However, irrespective of the
tools, it is essential to know what is the risk of the proposed IT investment and how much should
be the discount rate to adjust the future payoffs. Our extensive review of research, appeared in
the leading IS journals, emphasizes the critical need of developing methodology to quantify the
risk and the expected return from IT investment, the fundamental steps of IT evaluation.
In the past, IT risk research has essentially examined three aspects of IT risks: definition,
measurement and management. Early research focus on definitional aspects of IT risks (e.g.
Barki et al 1993) and identifies diverse set of risk factors1. Whereas, recent research suggests
1
A risk factor is a characteristic of the investment or its contextual environment that alters the probability
distribution of an investment’s outcome.
1
Appari and Benaroch
Pricing the IT Investment Risks
different proxy measures of IT risks at firm level, as well at project level (Dewan, Gurbaxani and
Shi 2004; Erdogmus 2000; Voshmgir 2002). However, due to inherent limitations we discuss
later, these measures may not be sufficient for IT investment evaluation. Simultaneously, real
options theory is shown to account for different risks in IT investment evaluation and risk
management (Benaroch 2002; Benaroch and Kauffman 1999, 2000, Dos Santos 2003). But this
research is limited by the fact that risk-related model parameters are assumed a priori in the
absence of estimation procedures. Unfortunately, this embryonic state of IT risk measurement is
crippling the ability of organizations to manage their critical resources on IT investments vis-àvis other business initiatives (Symons 2005).
In this research, we attempt to resolve these limitations of extant research by answering:
1. How to model the risk-return relationship for IT investments and explicitly account
for the effect of various risk factors?
Broadly speaking, IT investment risk factors fall into three broad classes (which we define later):
software development risks, organizational risks, and market risks (Benaroch 2002). In this
research we adapt the multifactor model and arbitrage pricing theory (Ross 1976) to formalize
the risk-return relationship of IT investment that accounts for the effects of these risk factors.
Operationalizing this model requires estimation of the two parameters for each risk factor –
premium return per unit of, and investment-specific sensitivity, to that factor. This leads us to the
research questions of:
2. How to estimate the sensitivity of an IT investment to different risk factors?
3. How to estimate the risk premium for a unit risk of different IT risk factors?
Recognizing that different risk factors may require different methodologies, the estimation of the
parameters for all the risk factors may not be feasible within the scope of one research paper. In
2
Appari and Benaroch
Pricing the IT Investment Risks
this research we focus on a subset of risk factors and propose two distinct methodologies – one
for estimating software development risk factors and the other for market risk factors 2. For
software development risk factors, we develop a methodology that adapts a parametric cost
estimation model - COCOMO (Boehm et al 2002), which by design accounts for impact of
software development risk factors on development cost. The growing stream of IT value research
uses event study analysis as ‘the’ empirical approach to explicate the impact of IT investments
on the firm’s value and risk (Dehning, Richardson and Stratopoulos 2003). Following from this,
for market risks we develop a methodology based on the event study analysis, and illustrate its
use for ‘customer acceptance’ risk.
This research makes a fundamental contribution to IT risk management literature by proposing a
risk measurement model and estimation methodologies for a diverse set of IT risk factors. On the
one hand, our research could enable IS researchers to correlate the investment level IT risk
measures to firm-level aggregate IT risk measures, as Dewan et al (2004) have proposed is
needed to advance our understanding of IT risk. On the other hand, we anticipate that successful
quantification of all the relevant IT risk factors could lead to determination of more meaningful
risk adjusted discount rates for IT valuation purpose. This in turn, may facilitate broader
adoption and use of sophisticated analytical techniques such as real options analysis and
portfolio management to IT investments.
The rest of the paper proceeds as follows. First, we briefly review research on IT investment risk
and its shortcomings vis-à-vis risk measurement. Then, we propose formalizing the risk-return
relationship for IT investment based on arbitrage pricing theory. Subsequently we propose two
2
Estimation of organizational risk factors is beyond the scope of this research, as any methodology will have to
depend on firm’s internal, sensitive information, which is difficult to obtain.
3
Appari and Benaroch
Pricing the IT Investment Risks
different empirical methodologies for estimating the risk premium for, and sensitivity to,
different market risk factors and software development risk factors. Finally, we offer some
concluding remarks.
2. THEORETICAL FOUNDATIONS
Broadly speaking the IT risk literature, in the past three decades, has evolved into three broad
streams of research – risk definition; risk measurement; investment evaluation under risk and
risk management. Early research has primarily focused on defining the risk factors and
developing qualitative measures that enable practitioner to prioritize risks. Lately the research
has endeavored into applying financial tools, such as real options, to evaluate IT investments and
manage IT risks. In this section, we briefly review these three streams of research vis-à-vis
highlight some shortcomings that motivate our research. In addition, we briefly review the
financial view of risk-return relationship that is adapted to IT investment level, followed by
event study analysis and software cost estimation model used in this research to develop risk
parameter estimation methodologies.
2.1. IT Risk Definition
Traditionally IT risk is defined in terms of negative consequences (Barki et al. 1993, Boehm
1989); e.g., lack of skilled analyst could lead to poor system design. However, risk may also
reflect positive opportunities, such as an unexpected increase in customer response can lead to
higher market share (Benaroch 2002; Vitale 1986). Therefore, this research views risk as the
variation in the attainment of IT investment goals, consistent with rational decision theoretic
view (Bernstein 1996). This variation materializes due to the presence (or absence) of specific
characteristics in the contextual environment of IT investment, known as risk factors, e.g. lack of
4
Appari and Benaroch
Pricing the IT Investment Risks
developing similar business application may result in increased learning time, effort and money
leading to over-budget and schedule slippage.
The definitional stream of IT risk research, focusing solely on software development risks, has
produced several risk taxonomies varying from ‘top n risks’ (McFarlan 1974; Boehm 1989) to
empirically validated risk factors (Barki et al. 1993; Schmidt et al. 2001). To the best of our
knowledge, very limited research has studied other forms of risk that influence payoff from IT
investments (Benaroch 2002; Vitale 1986). Given the diverse list of risk taxonomies, we adapt a
high-level risk taxonomy (Benaroch 2002) that spans the risk factors most commonly identified
by earlier research and is amenable to financial analysis (see figure 1). In this taxonomy, IT risks
are classified into firm-specific risks (software development risks, organizational risks) and
market risks. The firm-specific risks are endogenous to the organization undertaking an IT
investment, while market risk factors are exogenous and affects all firms that consider (or have
already made) the same IT investment. This notion of classifying risks into firm-specific risks
and market risk is consistent with risk classification approach common in finance research where
risk is divided into market risk and unique risks of an asset (Brealey and Myers 2002).
Firm-Specific Risks
Firm-specific risks of IT investment are due to the uncertainty of various factors internal to the
firm investing in IT that affect it’s ability to successfully realize the payoff from IT investment.
In particular, Software Development Risks are due to uncertain endogenous capability factors
associated with software development activities that affect the ability of the investing firm to
effectively realize an investment opportunity (Benaroch 2002), e.g. requirements uncertainty,
complexity, and programmer skill. The Organizational Risks are due the misfit between the IT
investment and internal operating environment of the organization. Even if the IT system is
5
Appari and Benaroch
Pricing the IT Investment Risks
designed and implemented correctly, these risks may influence the realization of anticipated
benefits from IT investment, e.g. organizational politics, user support, and structural fit.
Figure 1: IT Risk Classification [adapted from Benaroch 2002]
Market Risks
Market risks are due to uncertain exogenous factors that affect the ability of every firm,
considering the same investment, to obtain the payoffs from a realized investment opportunity,
e.g. competition, customer acceptance of technology investment, supplier’s resistance to
technology investment, regulatory changes, and security of data.
2.2.
IT Risk Measurement
Early IT Risk research focuses on non-financial measures to prioritize risks during the software
development (e.g. Boehm 1989). This stream of research quantifies risk using the concept of risk
exposure measured as the product of probability of an unsatisfactory outcome, and the loss
potential of the outcome. On the one hand, the estimation of risk probability remains an arduous
task (Kitchenham and Linkman 1997) as it requires extensive data collection and efficient
metrics management program in place. On the other hand, the risk measures proposed by this
6
Appari and Benaroch
Pricing the IT Investment Risks
stream of research lack the capability of integrating with any of the financial techniques for
evaluating (managing) IT investments (IT risks).
Recent research proposes different proxy measures of IT risk(s): subjective measures (Voshmgir
2002), an industry level aggregate measure (Dewan et al 2004), and twin security3 based measure
(Erdogmus 2000). Voshmgir (2002) conceptualizes IT project risk as a function of three
uncorrelated IT risks – market risk, project risk, and conversion effectiveness. This notion of risk
classification is similar to Benaroch (2002) but uses a more restrictive definition for each of the
risk component. The market risk is a function of the firm’s bargaining power vis-à-vis customers,
suppliers, and its competitive position in the market. The technical risk is a function of project
management capabilities to successfully complete the project; and complexity of the project in
terms of number of transactions supported and number of business partners. The risk of
conversion effectiveness is a function of management commitment and organization’s ability to
cope with transformational changes. In this measurement framework all the component
predictors of different risks (e.g. bargaining power with customer, management commitment,
and project management capability) are measured on a nominal scale of 0 to 1, thus restricting its
usage to qualitative judgment.
More recently, Dewan et al (2004) proposed an industry level aggregate measure of IT risk that
is hypothesized to reflect market risk of IT investment, and confirmed that firms with higher IT
risk require substantially higher premium return compare to the firms with low IT risk. However,
this risk measure, estimated at industry level, is limited in its use and may bias evaluation of
individual IT investment.
3
A twin security is a security or portfolio of securities that has the same risk profile as that of the
asset under consideration.
7
Appari and Benaroch
Pricing the IT Investment Risks
In finance literature it is suggested that valuation of an asset with unknown risk can be
accomplished by using the discount rate of an equivalently risky asset (Brealey & Myers 2002).
Erdogmus (2002) used this concept, twin security or mimicking portfolio, to develop a portfolio
index of publicly traded technology firm’s stocks that are in the business of same software
technology, e.g. Java. The variance of returns on this index quantifies the risk of the underlying
technology. Even though this risk measure can be used as input to some financial tools, such as
real option; it is not a panacea of risk measurement issues due to several reasons. First,
technology risk is just one of the market risks. Second, this measure is vendor based and heavily
depends on the arbitrary weights given to each vendor’s stock in portfolio construction. Third,
this measure fails to price the risk and does not help in determining the sensitivity of investment
being considered by firms.
In summary, past IS research has proposed several IT risk measures, however from
operationalization viewpoint none of these measures is a suitable input to finance-based
techniques for decision making at the investment level, perhaps with the exception of the twin
security measure. Moreover this stream of research does not prescribe any procedure to price the
individual risk factors. This inability to quantify the impact of various risk factors, specifically in
relation to the expected returns from an IT investment, is plausibly a key deterrent to widespread
adoption of finance-based techniques like IT portfolio management and real options analysis. In
this research we attempt to address this gap by paving a way to measure IT risks in terms that
can be used in finance based evaluation models.
2.3. IT Evaluation under Risk and IT Risk Management
The intense dynamics of IT, highly uncertain business environment, and irreversibility of
implementation cost has rendered traditional investment valuation approaches – e.g., payback
8
Appari and Benaroch
Pricing the IT Investment Risks
period, discounted cash flow (DCF), and net present value (NPV) – ineffective for investment
decision-making. However, recognizing that managers do have flexibility of timing and
structuring the IT investments, Dos Santos (1991) introduced real options theory to IT evaluation
by viewing IT project as an option to exchange risky assets. Since then this growing stream of
research has seen application of real options to evaluate the IT investments in the presence of
risks (Benaroch and Kauffman 1999, 2000; Dos Santos 2003), to manage the IT risks by
structuring optimal configuration of options into the investment (Benaroch 2002), and to value
and prioritize portfolio of IT investments (Bardhan, Bagchi and Sougstad 2004).
Majority of these research focuses on justifying applicability of real options to IT investment
decisions for a single project or portfolio of projects, with the exception of Benaroch (2002) that
demonstrates how to leverage real options philosophy to manage IT risks. It is demonstrated that
managers can actively configure IT investments by creating a set of options (e.g. prototype,
defer, stage, abandon, scale up/ down, outsource, lease, and growth) into investment that
maximizes the value vis-à-vis the risks of IT investment (Benaroch 2002).
However, this stream of research faces two key difficulties: (1) it requires estimation of riskrelated parameters for which currently IS research does not prescribe any procedure, and (2) the
inclusion of multiple risk factors renders the analysis approach intractable. Even for a small set
of risk factors, testing the robustness of investment evaluation using simulation-based sensitivity
analysis requires distributional values for risk-related model parameters. In this light, our
research seeks to address some of the key IT risk measurement issues having to do with how to
model the risk-return relationship of an IT investment and how to estimate the model parameters
– sensitivity and risk premium of risk factors. In the next section we briefly elaborate on the
9
Appari and Benaroch
Pricing the IT Investment Risks
financial view of the relationship between expected return of an asset vis-à-vis its exposure to
multiple risk factors.
2.4. Financial View of Risk – Return Relationship
In corporate finance literature, risk of an investment is defined as the variation of payoff from the
investment. Economic theories ex ante predict a positive correlation between the risk of and the
return from an asset under the assertion that investor seeks higher returns for bearing high risk
(Brealey & Myers 2003). Conceptually, the risk of an asset can be divided into – unique
(idiosyncratic) risk, and market (systematic) risk. The unique risks are diversifiable by creating a
portfolio of assets which eliminates the effect of unique risks. However, systematic risks (market
risks) cannot be diversified away. Finance literature predominantly focus on pricing the market
risks in terms of beta (), that indicates sensitivity of the asset to market movements, and the
market risk premium measured as the difference between expected return on market portfolio of
unit beta and risk free rate (rm – rf). In competitive market, according to capital asset pricing
model (Sharpe 1964), the expected risk premium from an asset is proportional to the asset’s beta.
Alternatively per multifactor model, the underlying model for arbitrage pricing theory (APT),
the expected return in excess of the risk free interest rate from an asset, r, having exposure to {J}
risk factors is given by (Ross 1976):
r  b1f1  b 2f 2  ......  b jf j  .....  b J f J
(1)
where bj is sensitivity of the investment to the jth risk factor, and fj is the unexpected
change in the level of the jth risk factor.
Assuming j is the risk premium (or market price) for a unit risk of the jth factor, under the
equilibrium assumption of APT the equation (1) can be rewritten as:
10
Appari and Benaroch
Pricing the IT Investment Risks
r  1b1   2 b 2  ...... jb j  ..... J b J
(2)
These economic models assume that idiosyncratic risks are completely diversified away by
investors and hence the investors’ market prices the systematic risk only. However, recent
empirical studies in finance suggest idiosyncratic risks of asset are also priced by market and the
price of idiosyncratic risk for an individual stock depends on its correlation with aggregated
undiversified idiosyncratic return, alike market risk (Malkiel and Xu 2002; King, Sentana, and
Wadhwani 1994). This suggests that by adapting the procedures of empirical finance, it would be
possible to price the idiosyncratic risk in the context of IT investments.
Elton & Gruber (2001) suggest that if we can specify a set of factors, fj’s, (firm characteristics)
that affect expected return, then equation 2 can be used to estimate the sensitivities, bj’s, and
subsequently equation 3 can be used to estimate risk premium, j, of each factor. Recently
Vermeulen, Spronk and Wijst (1996) adopted the concept of multifactor model to analyze risk
and expected level of firm performance. In a similar vein, in this research, we adapt the multifactor approach to define the risk-return relationship of IT investments. Further, we propose
event study based estimation methodology to estimate parameters of market risk factors (i.e.
sensitivity and risk premium); and adapt COCOMO model to estimate software development risk
factors. In the next section we briefly discuss the extant IS research that has used event study
analysis in the context of IT investments. Subsequently we discuss a cost estimation model that
is adapted in this research to estimate parameters of software development risks.
2.5. Events Study in IT Value Research
In accounting and finance literature, an extensive body of research uses event study analysis to
quantify the effect of an event on firm’s value and / or risk by examining the firm’s stock price
11
Appari and Benaroch
Pricing the IT Investment Risks
movements around the event date (Binder 1998). The crux of event study analysis, according to
the semi-strong efficient market hypothesis, is that the market value of a firm fully reflects all
publicly available information (Fama et al 1969). This assertion implies that an abnormal stock
return associated with an unexpected event should be observed and measurable if the event has
information content, that forces investors to adjust the stock price.
Prior IS research suggests that public news of events associated with IT investment having
positive information generally cause an increase in firm’s stock price (Chatterjee, Pacini &
Sambamurthy 2002; Dos Santos et al. 1993; and Subramani & Walden 2001). In addition a
distinct difference (more favorable) in market reaction to infrastructure investments vis-à-vis IT
application is discovered, supporting the conjecture that infrastructure investments have growth
options (Chatterjee et al 2002). Conversely, stock market reacts negatively to public news of
events that contains negative information related to IT investments, e.g. business disruption due
to hacker’s attack (Ettredge & Richardson 2003; Hovav & D’Arcy 2003). This stream of
research, focusing on security threats, suggests that the stock market punishes the firms, across
industries, who have disclosed security breaches as well the other firms who weren’t hacked but
are vulnerable.
In summary, there is a growing interest in applying event study analysis in IT value research to
explicate effect of IT investments on firm’s value and risk by examining relevant events. In view
of the limitations observed on extant measures of IT risks in the earlier section, and recognizing
that event study analysis captures both value and risk of firm (Dehning et al 2003), we use event
study analysis to develop estimation methodology for quantifying IT market risks such as
customer acceptance. For customer acceptance risk, we use a third party performance rating
announcements, GomezPro Scorecard™ (GPSC).
12
Appari and Benaroch
Pricing the IT Investment Risks
2.6. Software Cost Estimation and Risk
Software engineering cost models could be used for budgeting, project planning and control, and
risk tradeoff analysis (Boehm et al 2000). In the past three decades several cost estimation
models have become prevalent, including COCOMO, SEER (see Boehm et al 2000 for details).
Majority of these cost models are proprietary with the exception of COCOMO. These models are
designed to account for the effect of different risk factors to estimate the expected cost of
developing a software of given size. In a comparative study, Boehm et al (2000) conclude that
none of the cost estimation models is best for all situations and all of them are continuously
challenged by rapid advances in technology. In this research, we adapt the COCOMO model as a
base model to demonstrate the feasibility of estimating risk parameters.
The COCOMO (COnstructive COst MOdel) was initially published in Boehm (1981) and
became one of the most popular parametric cost estimation models. Recent innovations in
software engineering – e.g. rapid application development process, software reuse, object
oriented design, and web based development – have rendered the estimation accuracy of
COCOMO poorer. Consequently the model is continually upgraded to account for these changes
(Boehm et al 2002). The revised model COCOMO has two submodels – Early Design Model
(EDM), and Post Architecture Model (PAM). The EDM is applicable at the investment decision
level when only the aggregate level information is available on the new project. The PAM is a
detailed model that can be used when high level design is established. This COCOMO model has
been calibrated to a database of 161 projects collected from Commercial, Aerospace,
Government and non-profit organizations using the Bayesian analysis approach (Chulani et al
1999). In addition several variants of COCOMO are under development that accounts for
specifics of rapid application development process (CORADMO), and components based
13
Appari and Benaroch
Pricing the IT Investment Risks
development (COCOTS). This suggests, a risk estimation methodology based on COCOMO
model can accommodate advances in software engineering.
According to COCOMO model, the expected cost (E), measured in person-months, to build
software of size K kilo source lines of code is -
E  AK
5
B 0.01  S j n
j1
 Ci
(3)
i 1
where Sj and Ci are weights for scale factors and cost drivers,
A, and B are calibration constants of model and
n = 7 for Early Design model and 17 for Post Architecture model.
The scale factors and cost drivers are akin to various risk factors identified in IS literature. The
scale factors for both EDM and PAM are same, while five of the cost drivers in the EDM are
aggregate of cost drivers in the PAM as shown in figure 2.
Early Design
Post Architecture
Reliability (REL)
Early Design
Product Reliability & Complexity (RCPX)
Database size (DATA)
Application Experience (APEX)
Product Complexity (CPLX)
Documentation to life-cycle needs
(DOCU)
Personnel Experience (PREX)
Language & Tool Experience (LTEX)
Platform Experience (PLEX)
Platform Difficulty (PDIF)
Execution time constraint (TIME)
Main storage constraint (STOR)
Platform volatility (PVOL)
Facility (FCIL)
Use of Software Tools (TOOL)
Multi-site Development (SITE)
Analyst Capability (ACAP)
Personnel Capability (PERS)
Programmers Capability (PCAP)
Personnel Continuity (PCON)
Developed for Reusability (PERS)*
Required Development Schedule (SCED)*
* Same as Post Architecture Model
Figure 2: Mapping of Early Design Cost Drivers to Post Architecture Cost Drivers
14
Appari and Benaroch
Pricing the IT Investment Risks
The COCOMO model provides extensive guidelines on how to rate the risk factors on a nominal
scale – from extremely low, through nominal, to extremely high. At each rating level the risk
factors are assigned a specific weight that has been determined by wideband Delphi approach
and calibrated using Bayesian analysis (Chulani et al 2002). For scale factors the weight varies
from 0 (for extremely high level) to 7.8 (for very low level). For cost drivers the weight assumes
a value of 1 for nominal level, and then increases (or decreases) depending on the direction of
effect the cost driver has on cost, e.g. Reliability changes from 0.86 (for very low level) to 1.26
(for very high level) whereas for Analyst capability the weight changes from 1.42 (for very low
level) to 0.71 (for very high level). In this research, we use a generalized version of the
COCOMO model to derive sensitivity estimates of each project to different risk factors and risk
premium of risk factor, to be discussed later in detail.
3. RISK-RETURN MODEL FOR IT INVESTMENTS
In this section, building on the past research in IS and Finance, we present a formal model to
describe the risk – return relationship for an IT investment.
Let us assume that an IT investment is exposed to {J} risk factors that includes n S software
development risks, nO organizational risks and nM market risks ( nS + nO + nM = J).
Consider a firm having portfolio of n IT investments. Each IT investment is exposed to these {J}
risk factors according to its sensitivity to each of the risk factors, defined as the risk profile (RPi)
of the investment:
RPi  (bi ,1 , bi , 2 ,....., bi , 2 ,.......bi , J )
 i = 1, ..,n
(4)
Suppose at time t, the {J} risk factors assume values as Ft  ( F1,t , F2,t ,..., F j ,t ,.......FJ ,t ) and the
unexpected change in the factor values is given by f t  ( f1,t , f 2,t ,..., f j ,t ,....... f J ,t ) with E(fj,t) = 0
15
Appari and Benaroch
Pricing the IT Investment Risks
Then by Multifactor model the expected return in excess of the risk free interest rate from an IT
investment, ri, having a risk profile RPi is (we drop the time subscript for convenience):
ri  bi ,1 f 1  bi , 2 f 2  ......  bi , j f j  .....  bi , J f J  ei
(5)
where ei is a random error with mean zero and variance equal  ei2 .
If j is the extra expected return required from the IT investment for being exposed to j th risk
factor, then by APT the equation (5) can be rewritten as:
ri   1bi ,1   2 bi , 2  ...... j bi , j  ..... J bi , J
(6)
With knowledge of the parameters (bi,j, j) involved in equations (5) and (6), organizations can
evaluate IT investments and even manage their portfolio of IT investments. Following the three
classes of IT risk factors (figure 1), the equations (5) and (6) can be rewritten as:
nS
nO
nM
j 1
k 1
l 1
nS
nO
nM
j 1
k 1
l 1
ri   bi , j f j   bi , k f k   bi , l f l
ri    i , j b j    i , k bk    l bl
(7)
(8)
Alternatively the equations (7) and (8) can be represented as:
r~
rSD  ~
rOR  ~
rM
(9)
where ~
rSD , ~
rOR and ~
rM are expected returns due to software development, organizational, and
market risks respectively.
4. RISK PARAMETER ESTIMATION
Our next goal is to develop and demonstrate estimation methodologies that can be used to
quantify parameters of IT risk factors, namely sensitivity of investment to, and premium of the
risk factor. The estimation of risk premium and sensitivity for software development risks and
16
Appari and Benaroch
Pricing the IT Investment Risks
organizational risks would normally require internal firm-specific data. By contrast, the same
parameters for market risk factors could be quantified based on publicly available market data.
With this research being the first of its kind, and having lack of access to firm’s internal data, we
focus hereafter only on those risk factors for which data is available in the public domain.
4.1. Parameters Estimation for Market Risks Factors
Past IT value research have used event study analysis to establish the economic value of IT
investment announcements (e.g. Chatterjee et al. 2002, Dos Santos et al. 1991) and hackers
attack (Hovav & D’Arcy 2003). This research offers evidence that IT investment does have an
influence on the value and risk profile of an investing firm. In empirical finance research,
scholars have devised a two-pass methodology for estimating parameters (sensitivity and risk
premium) of risk factors (see Elton & Gruber 2001). In this research, we combine the two and
suggest a methodology for estimating the parameters for market risk factors, as outlined below:
(1) Event Identification: [a] Identify the event(s) and variable(s) that capture one or more type of
market risks related to IT investments of interest, [b] Identify the publicly traded firms that
are affected by these events and have made similar IT investments.
(2) Determine the abnormal daily returns for those firms around the event dates by using a
Capital Asset Pricing Model that is estimated with 200 days daily returns prior to the event
window.
(3) Estimate Sensitivity: regress event day abnormal returns on firm characteristics and the mean
centered risk indicator variables identified in step 1. The coefficient of each indicator
variable would be an estimate of the sensitivity of the IT investment or the firm to a specific
market risk factor.
17
Appari and Benaroch
Pricing the IT Investment Risks
(4) Estimate Risk Premium: regress the event day abnormal returns on the sensitivity obtained in
step (3) for the risk factor by constructing an equally weighted and/or a value weighted
portfolio of securities.
Next, we explain how to estimate parameters of ‘customer demand’ risk using this methodology.
Estimating ‘Customer Demand’ Risk
Barua, Konana and Whinston (2005) suggest superior customer centric online capability plays a
significant role in sustaining satisfied and loyal customers leading to superior business
performance. Recent empirical research has developed and validated a multidimensional metric –
comprising ease of use, information availability, privacy & security, and quality – as a key
determinant of customer’s acceptance of or preference to B2C channel (Devaraj, Fan and Kohli
2002). Gomez, currently owned by WatchFire.com, publishes benchmark score, known as
GomezPro ScoreCard ™ (GPSC) reflecting evaluation of online services offered by financial
industries from the customer perspective on four dimensions – functionality, ease of use, privacy
& security, quality & availability. The firms are ranked by an overall weighted score normalized
to 1 – 10 scale (with weights of 40% to functionality, 35% to ease of use, 15% to privacy &
security, 10% to quality & availability). More recently several scholars have used GPSC scores,
as measure of online channel capabilities, to examine key drivers of switching cost (Chen and
Hitt 2002), and to study effect of IT investment performance on firm’s competitive advantage
(Kotha, Rajgopal and Venkatachalm 2004). In addition, several business publications indicate
GPSC as a key industry benchmark used to gauge competitor’s investment in online channel (PR
Newswire 2003). Therefore I hypothesize that GPSC announcements do convey important
information to investors leading to stock price movements, and the unexpected change in rank
18
Appari and Benaroch
Pricing the IT Investment Risks
scores can be considered as indicator variable for customer acceptance risk of investments in
online channel.
Empirical Analysis
The sample data contains a total of 63 GPSC announcements for 6 financial services industries
(banks, insurance, mortgage, brokerage and credit cards) made between April 1999 and October
2004. From November 2004, GomezPro has changed the computation of overall rank score,
hence we limit to the period Oct-1999 to Oct-2004. In the first two years (1999, 2000) Gomez
published rankings on quarterly basis, and thereafter switched to bi-annual basis (see figure 3 for
summary statistics on market to book value ratio (MBV), total assets, and GPSC scores).
* number of firms listed in each announcements are given next to announcement date
Figure 3: Summary Statistics of FSS firms
The demographic and relevant financial data of firm are collected from COMPUSTAT, CRSP
and EDGAR databases. The daily stock returns are queried by using PERMNO, a unique
identifier used by CRSP for each firm, for each event date by specifying 300 days before the
event date and 10 days after the event date. We purposely keep this longer calendar interval to
ensure that we get at least 200 trading days before each event date and 5 days after the event
19
Appari and Benaroch
Pricing the IT Investment Risks
date. Following the protocol of past event study research, I exclude all the (event date, firm)
combination for which either the event date is within + 7 days of any firm specific significant
date, including quarterly earnings announcement, merger and acquisition announcement, and
stock split or the firm is not traded for more than 60 days before the event. This elimination
procedure yielded 809 useful (event date, firm) combinations. The risk free rate (i.e. 30 days-T
bill rate) is obtained from the data library maintained by Kenneth French on his website.
The analysis is done in two stages – first stage involves determination of abnormal returns by
estimating a market model and test of significance of mean abnormal returns; stage 2 involves
estimating sensitivity of each firm to the ‘customer acceptance’ risk factor followed by the
estimation of risk premium (steps (3) and (4) respectively of the methodology ).
Stage 1: Determination of Abnormal Returns
The abnormal returns during the event window are obtained as the difference between the
observed returns and those predicted by capital asset pricing model (CAPM):
ARi ,t  ri ,t  rˆi ,t
(10)
where rˆi , t is predicted from
ri , t   i , 0  i rm , t  i , t
E[i , t ]  0,
and
V [i , t ]  i2
(11a)
where ri ,t and rm ,t are daily returns and market returns in excess of risk free rate; ˆ i and ˆ i are
firm’s estimated parameters of CAPM over an estimation period preceding the event.
Several studies have reported that daily stock returns exhibit heteroscedastic error pattern
(Brockett, Chen & Garven 1999; Schwert and Seguin 1990), and not adjusting for conditional
heteroscedasticity may seriously compromise statistical inferences (Corhay and Rad 1996). On
20
Appari and Benaroch
Pricing the IT Investment Risks
examining the market model residuals of the equation (11a), using Lagrange Multiplier
approach, a strong evidence of ARCH properties was revealed in about 25% cases 4. Therefore,
we correct for conditional heterscedasticity by using GARCH (1,1) model, as it has been shown
by past research to better fit the daily stock returns than general GARCH(p, q) models with p +q
 3 (Corhay and Rad 1996):
ri , t   i , 0  i (rm , t  rf , t )  i , t
 i , t  i , t ei , t
(11b)
i2, t    1i2, t 1  1i2, t 1
where
ei , t ~ iid
N(0,1)
Test Statistics for Abnormal Returns
The GPSC event data is a classic example of clustered events, as several firms were listed on
each of the 67 announcement dates, leading to cross-correlation of abnormal returns. The test
statistic (z0) is the ratio of the day ‘0’ mean abnormal return to its estimated standard deviation,
assumed to be unit normal under null hypothesis, as the degrees of freedom exceeds 200 (Brown
and Warner 1985).
N0
AR0
z0 
ˆ AR0

ˆ AR0 

where
AR 0 
4
  ARt  AR 
t  250


 AR i , 0
i 1
N0
(12)
4
2
and
AR t 
244
 AR i , t
t  250
247
Alternatively, we test for significance of the cumulative mean abnormal return for each event
date over the event window (–3, +3) by:
3
z   ARt /
t  3
S
3
 ˆ 2AR0
t  3
similar results were obtained through White’s test for heterscedasticity (Greene 2003)
21
(13)
Appari and Benaroch
Pricing the IT Investment Risks
The statistic (z) is assumed to follow unit normal under null hypothesis.
The analysis of abnormal returns for each industry separately indicates 6 out of 12
announcements for banks, 9 out of 11 announcements for credit cards, 8 out of 14
announcements for discount brokerages, 5 out of 9 announcements for full-service brokerages, 4
out of 8 announcements for insurance, and 5 out of 10 announcements for mortgages have
yielded significant market reaction.
Stage 2: Determination of Sensitivity and Risk Premium
The abnormal returns are regressed on unexpected deviation in GPSC score (C); and natural log
of total assets (ln[TA]), and market to book value ratio (ln[MBV]):
AR i , t  i  C,i Ci , t  LTA ln[ TA i , t ]  LMBV ,i ln[ MBVi , t ]  i , t
(14)
Due to lack of adequate time series of GPSC scores for each firm, the unexpected deviation is
computed as the percentage change of current GPSC score from its immediate past GPSC score.
And we exclude the first listing of every firm from analysis. In addition, we segregate the firms,
within an industry, based on their size into three segments – large firms, medium firms, and
small firms – corresponding to upper, inter-quartile and lower quartile range of ln[TA]. As the
dependent variable in the equation (14) is not directly observed but predicted from the equations
(10 & 11b), we use Weighted Least Square (WLS) regression with weights being the MSE
(mean square error) of the equation (11b) obtained for each (event date, firm) combination. In
addition, for each industry, firms listed on the same announcement date may fall under different
segments resulting in possible cross correlation of errors for the equation (14) across the
segments. To account for cross-correlated errors, we simultaneously estimate the equation (14)
for all the industries segments by following seemingly unrelated regression (SUR) estimation
22
Appari and Benaroch
Pricing the IT Investment Risks
procedure (Greene 2003). The coefficient of C gives the sensitivity of firms (hence of the IT
investment) within an industry segment to the customer acceptance risk.
Subsequently to estimate the risk premium of ‘customer acceptance’ risk we use WLS approach
to regress the abnormal returns on thus obtained sensitivities:
ARi ,t  i   CC ,i  i ,t
(15)
The analysis is still in progress, and preliminary summarized result is shown in Table 1.
Sensitivity Estimates
Industry Type
Firm Size
Large
Banks
Credit Cards
0.0013
-0.0005
Discount
Brokers
0.0082
Full-Service
Brokers
0.001
Insurance
Mortgages
-0.0078
0.011*
Medium
0.0002
0.0005
0.0163
0.0346*
0.0056
-0.0079*
Small
0.020*
0.0923*
0.0185*
-0.0261
-0.0141
0.0029
Table 01: Sensitivity estimates and Risk Premium estimate for ‘customer acceptance’ risk
Clearly sensitivity to ‘customer acceptance’ risk, in general, is significantly positive for smaller
firms in the banking, credit card, and discount brokerage industries. For full-service brokerage
and mortgage industries the sensitivities are significant for medium size firms with mortgage
firms having a negative exposure.
4.2. Parameter Estimation of Software Development Risk Factors
Consider a firm with history of m projects, each having risk profile based on COCOMO:
RPi  S i ,1 ,....S i ,5 , C i ,1 ,.........C i ,17 
(16)
A generalized version of the COCOMO model, based on equation (4) that accounts for effect of
risk factors, could be derived from COCOMO database as:
E  K
5
1    jS j 17
j1
 Ci  i
i 1
23

(17)
Appari and Benaroch
Pricing the IT Investment Risks
where E is cost estimated in terms of effort (man months), K is software size in kilo lines of
code, Sj and Ci are weights for scale factors and cost drivers, respectively.
A project with all the risk factors rated as nominal is analogous to a ‘risk free’ asset in financial
economics, though not free from risk. If an organization uses COCOMO as a base model to
estimate cost, then the total cost premium incurred on bearing the development risk factors due
to unexpected changes during the course of development project could be computed as a ratio:
rSD 
ERPactual  ERPestimated
ERPestimated
(18)
where ERPactual and ERPestimated are the development cost with respect to actual risk profile at the
end of the project, and estimated risk profile before the start of project. This ratio rSD is
analogous to daily return on a stock which is computed as (today’s closing price – previous day’s
closing)/ previous day’s closing price.
For each of the m projects, the sensitivity (bx) to the xth risk factor (scale factor or cost driver)
can be estimated by evaluating the partial derivative of equation (17):
bx 
rSD
x
E / x
ERPestimated

RPactual
(19)
RPactual
The estimates of sensitivity of each project to different software risk factors would yield a cross
sectional dataset for m projects as:
Total Cost
Premium
Risk Sensitivity Profile
rSD ,1
:
{b1,1 , b2,1 .........bnS ,1 }
rSD , 2
:
{b1, 2 , b2, 2 .......bnS , 2 }
......
......
......
{b1, m , b2, m .....bnS , m }
rSD , m
:
Figure 4: Cross sectional dataset of Software development risk
24
Appari and Benaroch
Pricing the IT Investment Risks
A cross sectional OLS regression of rSD on estimated risk sensitivities will yield the risk
premium for respective software development risk factors:
rSD,i    1b1,i   2b2,i  .....   n bn
S
S
,i
(20)
Event though the approach discussed here is in the context of COCOMO model, it is generic
enough to be applied to any cost model, accounting for the effects of risk factors. To demonstrate
the viability of this approach, we shall use Boehm’s (1981) dataset of 63 projects.
5. VALIDATION OF PARAMETERS ESTIMATES – BOOTSTRAPPING
The estimation of risk parameters for ‘customer acceptance’ and software development risks
discussed in the previous section involves relatively smaller samples when compared with
typical sample sizes used in empirical finance to estimate risk parameters. This arguably raises
questions of validity of the estimated values. In addition, for customer risks, the sample set of IT
investments considered are relatively similar across the financial services industries. Whereas for
software risks, the sample database considered contains projects executed in the decade of 1970
and have information on only a subset of risk factors. The risk parameter estimation would have
been improved, if a broader variety of IT investments were considered. Past IS research has used
bootstrapping procedure to ensure robustness of path coefficients in structural models (e.g.
Subramani 2004; Teo, Wei, & Benbasat 2003). Hence to improve the robustness of the
parameter estimates, in light of small sample size, we propose to Bootstrap the regression
equations (14) and (15) for ‘customer acceptance’ risk, as suggested in Mooney & Duval (1993):
(a) Generate B sub-samples of size n from the original dataset used for parameter estimation.
(b) Estimate equation (14) and (15) in sequence as discussed in earlier section on each subsample.
25
Appari and Benaroch
Pricing the IT Investment Risks
(c) Convert the bootstrapped regression coefficients (i.e. sensitivity and risk premium) into
an estimate of sampling distribution of the parameter estimates by placing a probability
of 1/B to each value of the coefficient estimate.
For software development risks, we bootstrap the equation (17) using 63 sample projects of
COCOMO database. For each run of bootstrap, the project wise sensitivities are computed and
using the equation (20) risk premium for each risk factor is obtained. Subsequently, following
the step (c), we estimate the sampling distribution of risk premiums.
6. RESEARCH CONTRIBUTION
Risk measurement is the weakest link in IT risks management research. This research makes a
fundamental contribution to the literature on the management of IT risks and IT investment
evaluation, by proposing a risk measurement model and estimation methodologies for a diverse
set of IT risk factors. The ability to quantify critical risk factors based on the proposed
methodologies could benefit IS research in several areas. First, it could lead research to develop
and demonstrate the feasibility of applying finance-style portfolio management models and
techniques to IT investment problems. Second, at the individual investment level, this research
provides estimation of risk parameters that could be used with analytical real option models to
address a larger set of risk factors simultaneously. Finally, our research could enable IS
researchers to correlate IT investment level risk measures to firm-level aggregate risk measures,
as Dewan et al (2004) have proposed is needed to advance our understanding of IT risk.
This research has significant implications for practitioners engaged in IT investment decision
making processes. A recent survey of top executives indicates a progressive growth in applying
portfolio management concepts to IT investments (Jeffery and Leliveld 2004), though the
majority of them lacks the know-how of IT risk estimation at the level of rigor used in financial
26
Appari and Benaroch
Pricing the IT Investment Risks
portfolio management. We believe that this research helps to fill this knowledge gap by
providing necessary tools and techniques to quantify IT risks. Last but not least, successful
quantification of all relevant IT risk factors could help managers to determine more accurate
risk-adjusted discount rates for investment evaluation, thereby enabling them to make more
judicious allocations of critical resources to competing investments.
7. REFERENCES
1.
Bardhan, I., Bagchi, S., and Sougstad, R. (2004), Prioritization of a Portfolio of Information Technology
Projects, Journal of Management Information Systems, 21, 2, 33 – 60
2.
Barki, H., Rivard, S., and Talbot, J., (1993) Toward an Assessment of Software Development Risk, Journal
of Management Information Systems, 10, 2, 203 – 225
3.
Barua, A., Konana, P., Whinston, A.B., and Yin, F. (2004), An empirical investigation of Net-enabled
business value, MIS Quarterly, 28, 4, 585 - 620
4.
Benaroch, M. (2002) Managing information technology investment risk: a real options perspective, Journal
of Management Information Systems, 19, 2, 43 – 84
5.
Benaroch, M. and Kauffman R.J. (1999) A case for using real option pricing analysis to evaluate
information technology project investments, Information Systems Research, 10, 1, 70 – 86
6.
Benaroch, M. and Kauffman R.J. (2000) Justifying electronic banking network expansion using real
options analysis, MIS Quarterly, 24, 2, 197 – 225
7.
Bernstein, P. L. (1996) Against the Gods: The remarkable story of risk, John Wiley, New York
8.
Binder, J. J., (1998) The Event study methodology since 1969, Review of Quantitative Finance and
Accounting, 11, 2, 111 – 137
9.
Boehm, B.W. (1989) Software risk management, IEEE Computer Society Press, Piscataway, NJ,
10. Boehm, B.W., Abts, C., and Chulani, S., (2000), Software development cost estimation approaches: A
survey, technical report, USC Center for Software Engineering, USC-CSE-2000-505
27
Appari and Benaroch
Pricing the IT Investment Risks
11. Boehm, B.W., Abts, C., Brown, A. W., Chulani, S., and Clark, B. K., (2002) Software cost estimation with
COCOMO , Prentice Hall, New Jersey
12. Brealey, R.A., and Myers, S.C., (2003), Principles of corporate finance, 7 th edition, McGraw Hill
13. Brockett, P.L., Chen, H.M., and Garven, J.R. (1999) A new stochastically flexible event methodology with
application to Proposition 103, Insurance: Mathematics and Economics, 25, 197 – 217
14. Brown, S.J., and Warner, J.B., (1985), Using daily stock returns: The case of event studies, Journal of
Financial Economics, 14, 3 – 31
15. Chatterjee, D. Pacini, C. and Sambamurthy, V. (2002) The shareholder-wealth and trading volume effects
of information technology infrastructure investments, Journal of Management Information Systems, 19, 2
16. Chen, P., and Hitt, L., (2002), Measuring switching costs and the determinants of customer retention in
internet enabled businesses: A study of online brokerage industry, Information Systems Research, 13, 3
17. Chulani, S., Boehm, B., and Steece, B. (1999) Bayesian Analysis of Empirical Software Engineering Cost
Models, IEEE Transactions on Software Engineering, 25, 4, 573-583
18. Corhay, A. and Rad, T.A. (1996), Conditional heteroskedascticity adjusted market model and an event
study, The Quarterly Review of Economics and Finance, 36, 4, 529 – 538
19. Dehning, B., Richardson, V.J., and Stratopoulos, T. (2003),Reviewing event studies in MIS: An application
of the firm value framework, Proceedings of 36th Hawa Inernational Conference on system Sciences
20. Devaraj S., Fan, M., and Kohli, R. (2002) Antecedents of B2C channel satisfaction and preference:
Validating e-commerce metric, Information Systems Research, 13, 3, 316 – 333
21. Dewan, S., Gurbaxani, V. and Shi, C. (2004) Investigating the Risk-Return relationship of Information
Technology investment: Firm-level empirical analysis, American Accounting Association Annual Meeting,
Orlando, August 8 – 11
22. Dos Santos, B.L. (2003) Information technology investments: Characteristics, choices, market risks and
value, Information Systems Frontiers, 5, 3
23. Dos Santos, B.L., (1991) Justifying investments in new information technologies, Journal of Management
Information Systems, 7, 71 – 90
28
Appari and Benaroch
Pricing the IT Investment Risks
24. Dos Santos, B.L., Peffers, K., and Mauer, D. C., (1993) The impact of information technology investment
announcements on the market value of the firm, Information Systems Research, 4, 1, 1 – 23
25. Elton and Gruber (2001) Modern Portfolio Theory and Investment Analysis,
26. Erdogmus, H. (2000) Value of commercial software development under technology risk, The Financier, 7
27. Ettredge, M. and Richardson, V.J. (2003), Information Transfer among Internet Firms: The Case of Hacker
Attacks, Journal of Information Systems
28. Fama, E., Fisher, L., Jensen, M. C., and Roll, R., (1969) The adjustment of stock prices to new information,
International Economics Review, 10, 1, 1 – 21
29. Greene, W.H. (2003), Econometric analysis, 5 th edition, Prentice Hall
30. Hovav, A. , and D'Arcy, J., (2003) The impact of denial-of-service attack announcement on the market
value of firms, Risk Management and Insurance Review, 6, 2, 97 – 121
31. Hunter, S., Kobelsky, K., and Richardson, V. J., Information technology and the volatility of firm
performance, working paper 4449-03, MIT Sloan School of Management, 2003,
32. Jeffery, M. and Leliveld I., (2004) Best Practices in IT Portfolio Management, Sloan Management Review
33. King, M., Sentana, E., and Wadhwani, S., (1994), Volatility and links between National Stock Markets,
Econometrica, 62, 4, 901 – 933
34. Kitchenham, B., and Linkman, S., (1997), Estimates, uncertainty, and risk, IEEE Software, 14, 3, 69 – 74
35. Kotha S., Rajagopal, S., and Venkatachalam, M. (2004) The role of online buying experience as a
competitive advantage: evidence from third party ratings for ecommerce firms, Journal Of Business, 77, 2
36. Malkiel, B.G. and Xu Y. (2002) Idiosyncratic risk and security returns, working paper, University of Texas
37. McFarlan, W., Portfolio approach to information systems, Harvard Business Review, 1974,
38. Mooney, C.Z., and Duval, R.D., (1993), Bootstrapping: A Nonparametric approach to statistical inference,
(SAGE University Paper series on Quantitative Applications in the Social Sciences, series no. 07-095),
Newbury Park, CA: Sage
39. PR Newswire (2003), Two InQuira customers in top 5 internet banking sites – Gomez study finds, April 16
40. Ross, S. (1976) The arbitrage theory of capital asset pricing, Journal of Economic Theory, 13, 341 – 360
29
Appari and Benaroch
Pricing the IT Investment Risks
41. Schmidt, R , Lyytinen, K., Keil, M, and Cule, P, (2001) Identifying software project risks: An International
Delphi study, Journal of Management Information Systems, 17, 4, 5 – 36
42. Schwert, W.G., and Seguin, P.J., (1990), ‘Heteroscedasticity in stock returns, Journal of Finance, 45, 1129
43. Sharpe, W. F. (1964) Capital asset prices: A theory of market equilibrium under conditions of risk, Journal
of Finance, x, 19, 425 – 442
44. Subramani, M. (2004) How do suppliers benefit from information technology use in supply chain
relationships?”, MIS Quarterly, 28, 1, 45 – 73
45. Subramani, M., and Walden, E., (2001), The Impact of e-commerce Announcements on the Market Value
of Firms, Information Systems Research, 12, 2, 135 – 154
46. Symons C. (2005) IT measurement is still immature, CIO
47. Tanirverdi H., and Ruefli T.W. (2004), The role of information technology in risk/return relations of firms,
Journal of the AIS, 5, 11-12, 421 – 447
48. Telang, R., and Wattal, S. (2005), Impact of vulnerability disclosure on market value of software vendors:
An empirical analysis, 4th Workshop on Economics and Information Security, Boston, June 1-3
49. Teo, H.H., Wei, K.K, and Benbasat, I., (2003) Predicting Predisposition Toward IT-Based Interorganizational Linkages: An Institutional Perspective, Management Information Systems Quarterly,
50. Vermeulen, E.M., Spronk, J., and Wijst N.V. (1996) Analyzing Risk and Performance Using the
Multifactor Concept, European Journal of Operational Research, 93, 1, 173 – 184
51. Vitale, M. R., (1986) The growing risks of information system success, MIS Quarterly, 10, 4,, 327 – 334.
52. Voshmgir, S. (2002) Assessing the risk of IT investment project with network externalities, Americas
Conference on Information Systems, Aug 9 – 11
30