Information & Management 38 (2000) 103±117 Does successful investment in information technology solve the productivity paradox? Theophanis Stratopoulosa,*, Bruce Dehningb a Department of Decision Sciences, University of New Hampshire, McConnell Hall, Durham, NH 03824, USA b Department of Accounting and Finance, University of New Hampshire, Durham, NH 03824, USA Received 24 March 2000; accepted 28 April 2000 Abstract Based on previous empirical research, there seems to be little relation between investment in information technology (IT) and ®nancial performance (often referred to as the `productivity paradox'). We hypothesize that this is due to the fact that many companies implement IT projects ineffectively. Like any other asset, IT must be utilized effectively to result in increased ®nancial performance. By comparing successful users of IT and less successful users of IT, we show that successful users of IT have superior ®nancial performance relative to less successful users of IT. However, any ®nancial performance advantage is short-lived, possibly due to the ability of competitors to copy IT projects. # 2000 Elsevier Science B.V. All rights reserved. Keywords: Productivity paradox; Information technology investment; Financial ratios; Financial performance; Competitive advantage 1. Introduction In the eighties, information technology (IT) was heralded as a key to competitive advantage [10,33,39]. Porter and Millar [39] concluded that IT has affected competition in three ways: it has led to changes in industry structure and competition, it was used to support the creation of new businesses, and companies using IT outperformed their competition. The belief that IT can lead to a competitive advantage has become less certain in the nineties. Nevertheless, a high percentage of top executives still consider IT as a key to a company's pro®tability and survival [34]. In spite of theoretical arguments and professional belief in favor of a positive relation between invest* Corresponding author. Tel.: 1-603-862-2346; fax: 1-603862-2383. E-mail address: theos@cisunix.unh.edu (T. Stratopoulos). ment in IT and superior ®nancial performance, empirical evidence on this relation has been inconclusive. Several empirical studies and ample anecdotal evidence indicate that companies that spend more on IT are not rewarded with superior ®nancial performance [5,16,28,46,47,51]. This study strives to ®nd a better way to study the impact of using IT successfully on a company's ®nancial performance. Our main proposition is that, ceteris paribus, successful users of IT should enjoy superior ®nancial performance relative to less successful users of IT. The performance advantage should be observable when a company's ®nancial performance is compared with that of similar companies in the same industry that have been less successful users of IT. We propose and test the impact of IT on ®nancial performance by focusing on companies that have been recognized as successful users of IT because competitive advantage (and its corollary: relatively superior ®nancial performance) rests on the ®rm's ability to 0378-7206/00/$ ± see front matter # 2000 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 8 - 7 2 0 6 ( 0 0 ) 0 0 0 5 8 - 6 104 T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 manage IT related inputs in a more productive way [38]. Using non-parametric statistics, we test to see if there is any difference in the ®nancial performance of successful users of IT and a carefully matched control group. The successful users of IT were chosen by independent experts in the ®eld of IT, and the control group was matched on SIC code, sales and total assets. Performance was assessed using various measures of pro®tability and ef®ciency over a period of 10 years. Statistical analysis indicates that companies that have been identi®ed as successful users of IT outperform their competitors in measures of pro®tability and ef®ciency. However, any ®nancial performance advantage is short-lived, possibly due to the ability of competitors to copy IT projects. 2. Literature review The seemingly obvious yet elusive relation between investment in IT and productivity, dubbed the `productivity paradox', has been the center of numerous news articles, editorials, research projects and books [1,6±8,17,22,28,30,31,40,45±47,51]. Although the spectrum of these studies ranges from the role of investment in IT on the economy, industry, and at the ®rm-speci®c level, the latter is related to this study. For a detailed review of the literature on the productivity paradox, see [6±8,22]. In a series of articles and two books, Strassman [46,47] presents the results of his ®ndings and the ®ndings of several other studies. The conclusion he draws is that there is no identi®able association between expenditures on IT and pro®tability, and this relation has not changed for more than 20 years. In his 1990 book, he presents the results of an analysis based on 292 companies. The scatter plot used to relate investment in IT and return on investment (ROI) reveals a random pattern. When he replicated his analysis using 1994 data for a group of 539 companies, the results were equally disappointing. The results do not change when he experiments with several different measures of pro®tability such as return on assets (ROA), return on net investment, and economic value added over equity. Segmenting his data set at the industry level did not improve the results. Strassman concludes that it is not how much you spend on IT, but how you manage your IT assets that makes the difference. Weill [51] used data from the valve manufacturing industry to test the relation between investment in IT and ®rm performance, sales growth, ROA, and two measures of productivity. To better understand the impact of IT on performance, Weill categorized IT investment as strategic or transactional, depending upon the management's intention. Although transactional IT investment was found to be signi®cantly associated with ®rms experiencing strong performance, strategic IT investment was found to be neutral in the long term, and associated with poorly performing companies in the short term. Yosri [52] studied the relation between IT expenditures (operational, strategic and tactical), and revenue-contributing factors in 31 major food ®rms for the period of 1987±1990. Yosri found no signi®cant correlation between IT investment and sales growth, market share gain, new market penetration, measures of quality improvement, and productivity. Dos Santos et al. [16] ®nd that an announcement of innovative IT has a positive effect on stock price. The announcement of non-innovative IT has a negative impact on stock price. Overall, announcements of investment in IT have no impact on stock price. Loveman [30] used a microeconomic production function to estimate the impact of IT on productivity. Using sales minus change in inventory as a surrogate for output, and various non-IT expenditures, labor compensation, and IT capital as inputs, Loveman found that the output elasticity of IT was negative. This was interpreted as a suggestion to businesses that they are better off investing their marginal dollar in non-IT factors of production. Barua et al. [5] used a process-oriented methodology to measure the impact of IT on strategic business units. Their main proposition is that the impact of investment in IT will be captured at low organizational levels by intermediate level variables (ex: inventory turnover). These variables in turn will impact output measures. Using data from the manufacturing sector for a period of 5 years, they found that IT indeed had a mostly favorable impact on intermediate variables. Intermediate variables were found to be signi®cant determinants of high-level economic variables such as ROA and market share. T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 Hitt and Brynjolfsson [22] used a panel of 370 companies over the period 1988±1992, and tried to replicate previous studies to the degree that it was feasible. They examined the impact of IT spending on ROA, return on equity (ROE), and total shareholder return. Even after they introduced controls (®rm-speci®c variables affecting pro®tability) to control for the possibility of spurious correlation, the results indicated no correlation between spending on IT, total shareholder return, ROA or ROE. Mahmood et al. [31] used a 3-year, cross-sectional analysis (companies from Computerworld's Premier 100 (CWP100) list for the years 1991±1993) to compare the impact of IT investment from previous years with organizational performance and productivity of the following years. They used cluster analysis to classify ®rms based on their IT investments, performance and productivity. Their results suggest `to some degree', a relation between investment in IT, performance and productivity. For the three sub-periods in their sample, they ®nd a positive relation between IT investment and change in revenue growth. Results for the relation between IT investment and other measures of productivity and performance were not as clear. 3. Methodology Investment in IT has been on an upward trend for the last 30 years. This upward trend can be seen in both relative and absolute terms; the former through the National Income and Product Accounts (NIPA) [49], and the latter from several extensive surveys by IT consulting ®rms. Computers appear as a subcategory of the major components of the NIPA. More speci®cally, computer spending by companies is captured under the heading `Producers Durable Equipment', which is a subcategory of `Business Fixed Investment'. According to Haimowitz [21], real computer spending as a share of business ®xed investment has risen from 2.12% in 1982±1986, to 13.11% in 1992±1996. Over the same two periods, the share of computer spending in terms of producers' durable goods has risen from 3.53 to 17.88%. The above percentages may underestimate the actual share, if you consider the fact that the de®nition of computers in the NIPA includes hardware but not software, unless the latter comes preloaded on 105 the computer. According to Bakos [2], recent estimates raise the investment in IT to 30% of new capital investment. According to a 1995 report of the Standish Group [43], ``In the United States we spend more than $250 billion each year on IT application development.'' Strassmann [48] reports this ®gure to be $700 billion. Cap Gemini's 1998 Millennium Index study raised the year 2000 spending for Europe and the US to $850 billion [9]. Several studies have attempted to estimate the worldwide spending on IT. A 1998 study by the market research ®rm Killen & Associates [26] estimated worldwide spending on IT to be $1.59 trillion and projected this to be $2.62 trillion in 2002. Computer Economics [11] in their `1999 Information Systems and eBusiness Spending' report estimate the worldwide spending on IT for 2001 to be $1.1 trillion. All sources, drawing evidence from interviews and survey responses from company executives worldwide, agree that spending is not expected to slow down in the near future. One would expect this investment to translate into improved ®nancial performance; otherwise, the investment would seem irrational. As we have already seen, most of the empirical studies have concluded that either there is no relation between investment in IT and enhanced ®nancial performance or at best there is a very weak positive relation. We believe that the answer to this puzzle lies in the fact that the mere acquisition of an asset does not grant its owner a competitive advantage. What adds more validity to this argument is the fact that a signi®cant percentage of the IT projects undertaken in any year have been failing to meet their objectives. The Genesys consulting group reports that over 80% of IT projects fail to deliver their anticipated bene®ts [20]. KPMG Peat Marwick [27] has coined these projects `runaway projects'. The ®gures that are reported under the heading of `Failure Record' in The CHAOS Report [43] are disturbing. According to the report, 31% of IT projects will be canceled before they are completed, and more than half will cost almost twice as much as originally estimated. Only 16% of software projects are completed on time and on budget. The problem with `runaway projects' seems to be international. In a survey by KPMG UK, they report that 83% of the companies that they interviewed had experienced a runaway project [27]. 106 T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 In a recent report based on interviews with more than 1600 executives, the Meta Group claims ``. . .that as much as $90 billion of the $700 billion it expects to be spent on Information Technology next year could be saved if users took more care in managing their relationships with vendors and services ®rms'' [44]. Given this dismal record, it is not surprising that empirical studies show little or no relation between investment in IT and ®nancial performance. If the distinction cannot be made between successful and unsuccessful users of IT, it is quite possible to observe no relation or a negative relation between investment in IT and ®nancial performance. We use this simple yet powerful conclusion in our search to capture the elusive relation between IT and ®nancial performance at the company level. Instead of trying to establish a relation between the level of investment in IT and ®nancial performance, we suggest the possibility of superior ®nancial performance for companies that have been recognized by IT experts as successful users of IT when compared to less successful users of IT. We are proposing that, like any other asset, IT can be used effectively or ineffectively. Improved ®nancial performance is the manifestation of how well the company is managing its IT assets. Effective users of IT assets should see a marked increase in ®nancial performance compared to less successful users of IT assets. Therefore, the null hypothesis that we are testing is H0. There is no difference in the distribution of financial performance measures of successful users of IT versus that of less successful users of IT. Stated in the alternative form, it is H1. The financial performance of companies that have been recognized as successful users of IT is systematically better than that of less successful users of IT. We stress the differential approach to establish the performance superiority between the two groups, because we do not want to give the false impression that IT will necessarily be associated with abnormal returns. Because computer-based information systems are easily copied by competitors, any ®nancial performance advantage may be short-lived. Competitors will respond and attempt to neutralize the competitive advantage of the successful users of IT by copying and possibly improving the IT used. In some cases, a competitor's response may be almost immediate. For example, the World Wide Web has become the battleground in the overnight delivery service industry. FedEx was the ®rst to use the Web with an on-line package tracking service in November 1994, followed by UPS 6 months later. In the March of 1996, UPS beat FedEx with complete Web shipping service. FedEx followed with a similar service in less than a year. This demonstrates how quickly competitors can respond to new IT [35,36,50]. MIT [32] predicted that, in the 1990's, IT would be a strategic necessity rather than a source of sustainable competitive advantage. This would result in short-term improvement in ®nancial performance but no long-term differential performance advantage. 4. Dataset The premise being tested is whether successful users of IT exhibit greater ®nancial performance than less successful users of IT. The CWP100 list for 1993 was chosen to identify successful users of IT. By default, any company not appearing on the Premier 100 list is de®ned here as a less successful user of IT. We are working with 1993 for the following reasons: ®rst, this will give us a minimum of 5 years to study ®nancial performance after the publication of the list. We believe that this is necessary, given the argument presented by David [13] and Mahmood et al. [31]. David [13] draws an analogy between cyberspace and the dynamo and argues that, as with the dynamo, the computer revolution will require a period of economic readjustment before the advances are re¯ected in the bottom line. Mahmood et al. [31] argue that there is a 2-year lag between the investment in IT and an improvement in ®nancial performance. One explanation for observing the productivity paradox comprises researchers who have ignored this lag effect of investment in IT [2,7]. The second reason we used the year 1993 is that it is common knowledge that, for IT to be successful, it will have to be accompanied with signi®cant re-engineering of the business process. According to a study from MIT [32], investment in new IT without parallel organizational T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 change is unlikely to yield good results. Starting in the early nineties, more and more companies realized and started practicing this new mantra. Since 1988, Computerworld has selected the 100 most successful users of IT. Each company is evaluated in terms of its performance according to four criteria: ®rst, investment in IT as a percentage of revenues is used to determine a company's commitment to the new technology and its ability to be costeffective. Second, 5-year growth rate in pro®ts is used to establish business performance. Third, management is rated by establishing how well the information system of the company is positioned to service its business needs. Fourth, each company in the target group is asked to rank the ®ve most effective users of IT in their industry. This fourth criterion, peer evaluation, carries double weight in the creation of the ®nal weighted average score that is used for the ranking of the companies [12]. Starting in 1994, Computerworld changed its way of determining the companies that qualify for its Premier 100 list. The dataset used in this study has several limitations. Because growth rate in pro®ts is used as part of the score used to select companies for the CWP100 list (the successful ®rms), any ®nancial performance ratios involving pro®tability may be biased in favor of ®nding a signi®cant difference between the two groups. Likewise, the peer evaluation component of the CWP100 list likely contains ®rms that are outperforming their competition in a number of ®nancial performance measures. This could lead to a possible bias toward rejecting the null hypothesis. These issues must be taken into account when considering the results of this study. In order to be included in our study, companies had to have complete data on the Compustat database for the 10-year period examined, 1988±1997. This requirement eliminated 22 of the CWP100 companies. The methodology chosen to test the hypothesis in question is a matched pair design. The CWP100 companies were matched with the less successful users of IT (the control group) based on SIC code, total assets, and sales, for the year 1993. The variables to match on were chosen to control for industry, size, and capital intensity. Matching on these variables rules them out as alternative explanations for any difference found in ®nancial performance between the two groups. 107 Where possible, the CWP100 companies were ®rst matched with the control group by a four-digit SIC code. After selecting potential control group companies with complete data on Compustat for the period examined, the closest match based on total assets and sales was chosen as the corresponding ®rm for the CWP100 company. If a suitable competitor could not be found that was no more than twice as large or one half as small as the CWP100 company, matches were made at the three-digit SIC code level. Again, if a suitable match could not be made that was no more than twice as large or one half as small as the CWP100 company, matches were made at the two-digit SIC code level. Out of the 78 companies with complete data on Compustat, 44 were matched by the four-digit SIC code, 11 by the three-digit SIC code, and 16 were matched at the two-digit SIC code level. There were no matches for seven companies in the CWP100 list. This left 71 pairs of companies in the ®nal study (see Table 1). Table 2 contains a list of all companies in the analysis. Table 3 provides summary statistics for the two groups in terms of total assets and net sales. By design, both groups should have mean sales and total assets that are approximately equal. Various ®nancial performance variables are used to measure the relative performance of the two groups. These can be separated into two categories, pro®tability measures and ef®ciency measures. The pro®tability measures are growth in net sales, gross pro®t margin, operating pro®t margin, net pro®t margin, ROA, return on equity (ROE), and ROI. The ef®ciency measures are ®xed assets turnover, total assets turnover, and inventory turnover. All data is from the PC Compustat database for the 10-year period Table 1 Reconciliation of the original Computerworld Premier 100 (CWP100) companies and the 71 CWP100 companies in the study CWP100 firms matched on a four-digit SIC code CWP100 firms matched on a three-digit SIC code CWP100 firms matched on a two-digit SIC code 44 11 16 Subtotal Ð total firm pairs 71 Number of unmatched CWP100 companies CWP100 companies without full data on Compustat 1988±1997 Total Ð number of CWP100 companies 7 22 100 108 T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 Table 2 The 71 Computerworld Premier 100 (CWP100) and 71 matched control group companies used in the study S. No. Experimental group (CWP100) Control group 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 Abbott Laboratories Air Products & Chemicals Inc. Airborne Freight Corp. Albertsons Inc. Alexander & Baldwin Inc. Allied Signal Inc. Amerada Hess Corp. Banc One Corp. Bandag Inc. Bankers Trust Corp. Becton Dickinson & Co. Browning-Ferris Industries Carolina Power & Light Cigna Corp. Cone Mills Corp. Crestar Financial Corp. Dillards Inc. Duke Energy Corp. Dun & Bradstreet Corp. Engelhard Corp. FDX Corp. FMC Corp. Food Lion Inc. FPL Group Inc. Gencorp Inc. General Dynamics Corp. Goodrich (BF) Co. GTE Corp. Harley-Davidson Inc. Harnischfeger Industries Inc. Hershey Foods Corp. Hon Industries Home Depot Inc. Honeywell Inc. Hormel Foods Corp. Intl Flavors & Fragrances Kellogg Co. Kellwood Co. Limited Inc. McDonalds Corp. MCI Communications Mellon Bank Corp. Merck & Co. Minnesota Mining & Mfg. Co. Navistar International Nortek Inc. Northeast Utilities Northrop Grumman Corp. Oryx Energy Co. Penney (JC) Co. Phelps Dodge Corp. Polaroid Corp. Quaker Oats Co. American Home Products Corp. Kerr-McGee Corp. America West Hldg. Corp. Great Atlantic & Pac Tea Co. Carnival Corp. United Technologies Corp. Imperial Oil Ltd. First Union Corp. Continental Can/De Morgan (JP) & Co. Medtronic Inc. Westcoast Energy Inc. GPU Inc. Aetna Inc. Guilford Mills Inc. Summit Bancorp Harcourt General Inc. DTE Energy Co. Unisys Corp. Allegheny Teledyne Inc. US Airways Group Inc. Hercules Inc. Southland Corp. Dominion Resources Inc. Cordant Technologies Inc. McDermott Intl. Inc. Sequa Corp. BellSouth Corp. Huffy Corp. Nacco Industries Brown-Forman Lowes Cos. Kimball International Emerson Electric Co. Dean Foods Co. Lubrizol Corp. Ralston Purina Co. Leslie Fay Companies Inc. Venator Group Inc. ICH Corp. Sprint Corp. Wachovia Corp. Bristol Myers Squibb Georgia-Pacific Group Dana Corp. Griffon Corp. Central & South West Corp. Litton Industries Inc. Burlington Resources Inc. May Department Stores Co. Inco Ltd. Bausch & Lomb Inc. Tyson Foods Inc. T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 109 Table 2 The 71 Computerworld Premier 100 (CWP100) and 71 matched control group companies used in the study S. No. Experimental group (CWP100) Control group 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 Raychem Corp. Raytheon Co. Readers Digest Association Rohm & Haas Co. Rubbermaid Inc. Schering-Plough Sherwin-Williams Co. Tecumseh Products Co. Textron Inc. Travelers Group Inc. US West Inc. UAL Corp. Union Pacific Corp. UNOCAL Corp. Wal-Mart Stores Westvaco Corp. Whitman Corp. Willamette Industries National Service Industries Inc. Baxter International Inc. McGraw-Hill Companies Arco Chemical Co. Armstrong World Industries Inc. Pharmacia & Upjohn Inc. Olin Corp. United Dominion Industries Tenneco Inc. Morgan Stanley Dean Witter & Co. Ameritech Corp. Delta Air Lines Inc. CSX Corp. Occidental Petroleum Corp. K Mart Corp. Union Camp Corp. Coors (Adolph) Bowater Inc. 1988±1997. For an explanation of how each ratio was calculated and interpreted as a measure of ®nancial performance, see Table 4. There is an extensive body of literature on the distributional properties of ®nancial accounting ratios. For a thorough review of the literature on ®nancial ratios, see Salmi and Martikainen [41] and Foster [18]. There is agreement among researchers that the distributions are characterized by non-normality, skewness (primarily right skewed), fat tails, and a signi®cant number of outliers [3,4,14,19,24]. Distributions with fat tails (leptokurtic or cauchy distributions) are particularly problematic when using parametric statistics [25]. Table 5 provides summary statistics of the variables considered for all 142 companies in the sample for 1993. Foster [18] explains the existence of outliers in distributions of ®nancial ratios in terms of accounting, economic and technical reasons. As expected, there are a signi®cant number of outliers. The interquartile range can be used as an aid to con®rm the existence of outliers. A simple rule of thumb states that we call an observation a possible outlier if it falls more than 1.5times the interquartile range above the third quartile or below the ®rst quartile. The results were similar for every year in the data set. Given the nature of the proposition to be tested, difference variables were created for each pair of Table 3 Comparative statistics (for 1993)a N Mean Standard deviation Min Max Total assets CWP100 Control Total 71 71 142 11560 11914 11737 20815 23510 22125 352 270 270 101360 133888 133888 Sales CWP100 Control Total 71 71 142 6263 5524 5894 8627 5681 7288 590 437 437 67345 34353 67345 a Dollar amounts are in millions. 110 Ratio Profitability measures Growth in net sales Gross profit margin Operating profit margin Net profit margin Return on assets Return on equity Return on investment Efficiency measures Fixed assets turnover Total assets turnover Inventory turnover Calculation Interpretation Net sales for the current period minus net sales from the prior period divided by net sales from the prior period Gross profit divided by net sales Income from operations divided by net sales Income from continuing operations divided by net sales Income available to common shareholders from continuing operations divided by average total assets Income available to common shareholders from continuing operations divided by common shareholder's equity Income available to common shareholders from continuing operations divided by total invested capital Measures a company's growth in net sales over the prior year Net sales divided by average property, plant and equipment Net sales divided by average total assets Cost of goods sold divided by average inventory Percentage of gross profit per dollar of sales Measures income from operating activities per dollar of sales Measures income from ongoing operations per dollar of sales Measures profitability and how efficiently assets were employed during the period Measures the profitability of the investment to the owners Measures the profitability of the investment based on total investment, both debt and equity Measures management's ability to generate sales, given an investment in fixed assets Measures how efficiently management utilized assets to generate sales Measures the liquidity of inventory and how quickly inventory is sold T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 Table 4 Financial performance variables T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 111 Table 5 Summary statistics (for 1993) Variable Min Growth in net sales Gross profit margin Operating profit margin Net profit margin Return on assets Return on equity Return on investment Fixed assets turnover Total assets turnover Inventory turnover ÿ53.20 2.16 ÿ11.30 ÿ14.39 ÿ21.90 ÿ160.68 ÿ479.93 0.32 0.06 0.08 Q1 ÿ1.20 22.33 5.33 2.10 1.28 5.62 3.33 1.55 0.62 3.96 Median Q3 Max Mean Standard deviation 4.45 30.74 9.75 4.65 3.52 13.27 6.99 3.34 1.02 6.16 8.90 44.78 16.78 8.66 7.07 19.08 12.28 5.24 1.41 9.29 95.50 81.99 67.80 27.78 19.82 142.23 46.76 19.63 3.62 131.11 4.27 34.72 11.75 5.70 4.39 11.95 4.66 3.92 1.09 11.50 13.60 17.80 9.97 6.67 5.83 25.19 42.16 3.21 0.70 18.42 matched companies. Subtracting the ®nancial performance measure of the less successful company (control group) from the successful company (CWP100) for each year created the difference variables. Positive values for the differential measures of pro®tability and ef®ciency speak in favor of our hypothesis. Table 6 reports summary statistics of the difference variables for the 71 pairs of companies in the sample for 1993. Given the nature of the underlying data set and the proposition to be tested, it may be deceiving to look at measures of central tendency. However, we can still get a glimpse at the results by looking at the means and medians of the distributions. The t-tests for testing if the population mean is different from 0 is signi®cant in most cases, but may not be reliable due to the underlying distributions of the variables. Although the sample is large enough to justify the use of the central limit theorem, and it may be possible to compensate for the existence of outliers, we believe that this type of testing will not capture the true nature of our proposition. To test whether the ®nancial performance of companies that have been recognized as successful users of IT is systematically better than that of less successful users of IT, non-parametric statistics will be used. We refrain from testing weather the median of differences is 0 because it implies the assumption of a distribution that is continuous and symmetric [23,29]. We have already seen that the distribution of ®nancial ratios tends to be skewed. The non-parametric test is the Wilkoxon signed rank test for matched pairs, the recommended test in cases of paired data where the normality of the data is questionable [29]. Table 7 reports the p-values for the Wilkoxon test. Lower pvalues indicate stronger sample evidence in favor of rejecting the null hypothesis. Table 6 Difference variables (for 1993) Ð summary statistics Difference in: Count Min Q1 Median Q3 Max Mean Standard deviation Growth in net sales Gross profit margin Operating profit margin Net profit margin Return on assets Return on equity Return on investment Fixed assets turnover Total assets turnover Inventory turnover 71 71 71 71 71 71 71 68 71 65 ÿ89.40 ÿ37.53 ÿ36.80 ÿ29.04 ÿ12.73 ÿ130.87 ÿ22.52 ÿ5.07 ÿ0.68 ÿ86.48 ÿ3.30 ÿ4.53 ÿ2.90 ÿ2.52 ÿ1.25 ÿ4.53 ÿ2.29 ÿ0.38 ÿ0.07 ÿ1.57 2.60 1.56 2.30 1.49 2.05 1.73 3.05 0.18 0.04 0.78 8.30 9.73 6.50 5.42 5.26 13.97 11.14 0.99 0.30 2.42 57.10 35.34 17.60 17.35 27.47 172.82 487.68 7.45 0.98 86.05 1.05 2.33 1.68 1.06 2.64 2.72 11.07 0.33 0.10 0.84 16.96 14.49 9.43 7.62 6.65 31.56 12.87 2.32 0.31 21.85 112 T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 Table 7 Hypothesis testing: p-values for H0a Measure 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 No. of years (p0.05)b Growth in net sales Gross profit margin Operating profit margin Net profit margin Return on assets Return on equity Return on investment Fixed assets turnover Total assets turnover Inventory turnover 0.08 0.50 0.92 0.62 0.10 0.39 0.15 0.72 0.03 0.58 0.76 0.97 0.80 0.09 0.01 0.19 0.01 0.32 0.00 0.50 0.61 0.89 0.41 0.05 0.00 0.02 0.00 0.27 0.00 0.27 0.44 0.47 0.23 0.02 0.00 0.01 0.00 0.38 0.00 0.17 0.00 0.30 0.02 0.00 0.00 0.00 0.00 0.36 0.01 0.11 0.06 0.13 0.02 0.06 0.00 0.05 0.01 0.16 0.01 0.30 0.01 0.21 0.02 0.01 0.00 0.01 0.00 0.37 0.00 0.16 0.77 0.98 0.45 0.20 0.02 0.26 0.07 0.34 0.01 0.07 0.03 0.69 0.38 0.90 0.24 0.10 0.15 0.41 0.13 0.11 0.10 1.00 0.71 0.65 0.46 0.10 0.81 0.57 0.07 0.55 3 0 3 4 7 5 6 0 8 0 a b p-values in `bold' are significant at the 5% level (two-tailed test); `italics' indicate instances where the control group performed better. The mean is 3.6 years; the median is 3.5 years. Although previous empirical studies have been unable to demonstrate the existence of a positive correlation between investment in IT and ®nancial performance, this should not be interpreted as a reason to consider investments in IT as worthless. As our results demonstrate, there is a systematic difference in the ®nancial performance of successful users of IT and less successful users of IT. Although we are using the CWP100 list for 1993, we expect that these companies have been successful users of IT for a number of years. Peer evaluation comprises 40% of the weighted average score to appear on the Premier 100 list, and it likely takes a few years to establish a reputation within a peer group. Therefore, we expect to see a difference in the ®nancial performance between the two groups before 1993. We predict that growth in net sales will be larger for the successful users of IT than for the less successful users of IT. This difference is signi®cant (p0.05) in 1992, 1994, and 1996, clearly supporting our proposition. It has been argued that successful users of IT should outperform less successful users of IT in pro®tability measures such as gross pro®t margin, operating pro®t margin and net pro®t margin. Our results support this argument. For gross pro®t margin, the pvalues decrease as we approach 1992±1993 and increase subsequently, but the sample evidence is not strong enough to lead to the rejection of the null hypothesis (in 1993, p0.13). For operating pro®t margin, the difference is signi®cant (p0.02) in the years 1992±1994, but not before or after this period. The difference in net pro®t margin tells a similar story. The difference is signi®cant as early as 1990 (p0.05), and remains signi®cant until 1994, except in 1993 (p0.06). The three return measures, ROA, ROE, and ROI, all support our hypothesis. For ROA, the difference is signi®cant in 1989, and remains signi®cant through 1995. For ROE, the difference is signi®cant in 1990 and remains signi®cant until 1994. The difference in ROI is signi®cant in 1989±1994, 1 year after the publication of the Premier 100 list. The three ef®ciency measures examined were ®xed assets turnover, total assets turnover and inventory turnover. The only ef®ciency measure that signi®cantly supports our hypothesis is total assets turnover. The difference in total assets turnover is signi®cant in the ®rst 8 years examined, but not signi®cant in 1996 and 1997. The p-values for ®xed assets turnover and inventory turnover decrease when approaching 1992± 1993 and increase subsequently, but the sample evidence is not strong enough to lead to the rejection of the null hypothesis. As expected, the relative difference in performance starts before 1993, the year of publication of the list. Companies in the list have been recognized by their peers for their superior use of IT. Naturally, the IT they have implemented and how it was implemented are scrutinized, replicated, and improved by their competitors. The recognition of superior use of IT carries the seed of its own destruction. Any differences in per- T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 formance will disappear shortly after recognition because competitors will respond by copying IT systems that have a proven record of success. The empirical evidence presented here clearly supports this argument. We observe that the p-values start rising (the differences becoming less signi®cant) shortly after the year of publication. Looking at the duration of the statistically signi®cant superior performance, we see that it usually ranges for 3±4 years (mean3.6 years, median3.5 years). 5. Conclusions According to the proponents of the productivity paradox, its manifestation becomes apparent in the lack of any statistically signi®cant correlation between investment in IT and gains in productivity. Logically, one would expect that companies that invest heavily in IT should be rewarded with superior ®nancial performance. As mentioned in our literature review, empirical evidence in favor of a positive association between investment in IT and ®nancial performance is anemic. Brynjolfsson [6] proposes four non-exclusive explanations for the productivity paradox: mismeasurement of inputs and outputs, lags due to learning and adjustment, mismanagement of information and technology, and redistribution and dissipation of profits. Brynjolfsson attributes the measurement error to the dif®culty of developing accurate, quality-adjusted price de¯ators. He argues that improvements in product quality and the introduction of new products need to be properly accounted for in the value of output. Lags as an explanation of the paradox suggest that the bene®ts associated with investments in IT may take several years before they `show (up) in the bottom line'. This is due to a period of learning associated with adjustment and possibly restructuring of the organization caused by new IT. The third proposition, `mismanagement of information and technology', suggests that IT is not productive, and managers who choose to invest in IT are not acting in the best interests of the company. Finally, redistribution as an explanation of the productivity paradox argues that `IT rearranges the shares of the pie' in favor of some companies `without making it any bigger'. Bakos [2] offers an alternative list of possible explanations for the productivity paradox, listing mis- 113 measurement, mismanagement, diffusion delay, and capital stock theory as the four `prominent hypotheses' for the explanation of the paradox. Although diffusion delay is another name for Brynjolfsson's lags, he replaced `redistribution' with Oliner and Sichel's `capital stock theory' [37]. According to Oliner and Sichel, in spite of the recent spending in IT, IT's share of capital stock is still small. This is because ®rms have only recently started investing heavily in IT, and by nature, IT tends to rapidly become obsolete. This makes it dif®cult for researchers to observe the impact of investment in IT on ®nancial performance. Brynjolfsson and Hitt [7] considered and empirically tested the possibility of the productivity paradox as `an artifact of mismeasurement'. They used the neoclassical production theory in order to determine the contribution of such inputs as computer capital and information systems staff labor to output. They measured output in in¯ation-adjusted dollar terms because, according to the authors, this partially accounts for changes in product quality and introduction of new products. The authors conclude that ``our results indicate that IS have made a substantial and statistically signi®cant contribution to ®rm output,'' and that the productivity paradox ``disappeared by 1991, at least in our sample of ®rms.'' By focusing on one of the four possible explanations (mismeasurement) for the productivity paradox, Brynjolfsson and Hitt [7] were able to show a signi®cant relation between investment in information systems and ®rm output. In all likelihood, the productivity paradox is due to a combination of factors, as suggested by Brynjolfsson [6] and Bakos [2]. Our research considers the possibility that a portion of the productivity paradox is attributable to mismanagement. Schrage [42] describes a rather colorful variation of the productivity paradox. He says that companies have wasted billions of dollars ``believing the big lie of the Information Age.'' According to Schrage, the spending spree on IT was justi®ed by a ``beautiful hypothesis'' that companies that had more and better information could improve their ®nancial performance and competitive position. The hypothesis was slew, according to Schrage, by an ``ugly fact'' that managers had acted irresponsibly in relying on technology to solve fundamental problems. Recent empirical ®ndings of Strass- 114 T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 mann [47,48] indicate that the lack of any signi®cant correlation between the investment in IT and performance points to possible irrational behavior of the management. What Schrage calls a ``beautiful hypothesis'' was developed over the years and was based on several studies (e.g. [33,39]), suggesting that IT could be a source of competitive advantage.1 It can be argued that companies responding to these propositions and ample anecdotal evidence joined the IT bandwagon and invested ever-increasing amounts in IT. As shown earlier, the latter can be seen in both growth rates and in absolute terms. Unfortunately, not all of this investment was successful. Investment in IT at the early stages spread like an epidemic (it was considered a panacea) and naturally did not deliver the unrealistically expected results. What is disconcerting is the fact that, in more recent years, in spite of the sensitivity and awareness of company CFOs regarding the amounts spent unsuccessfully on IT, which resulted in more careful justi®cation of IT projects, the situation has not improved. The majority of IT projects continue to fail. Recent developments indicate that this trend may continue because it is fed by the e-commerce revolution. As long as companies are rushing to jump indiscriminately on the e-commerce bandwagon, we are bound to see more and more of these investments fail. As mentioned earlier, according to the 1995 Chaos Report [43], well over 80% of IT projects fail, either because they cannot deliver the desired results or because they encounter signi®cant overruns in terms of spending and/or time needed for completion. Naturally, if you combine these two factors, high investment in IT and high failure rate in IT projects, you should not expect to ®nd any positive correlation between amount invested and performance. Any correlation will be obscured by the high percentage of the amount invested that fails to have any positive impact on the performance of the company. The correlation will continue to be insigni®cant even when controlling for such factors as industry, nature of investment in IT etc. [47,48]. The combination of increasing investment in IT and a high failure rate for IT projects is another way to express Brynjolfsson's [6] and Bakos' [2] concept of mismanagement. The natural corollary of this explanation is in the form of a proposition. If you could control for mismanagement, would you expect to observe a statistically signi®cant positive correlation between investment in IT and ®nancial performance? Several studies point to the fact that this will not be enough because IT is not an isolated island within the organization (e.g. [47,48]). The sheer completion of a project on time, on budget and with the required speci®cation is not enough to lead to superior ®nancial performance. Many companies might invest in the same technology, but only those who manage to successfully integrate the IT into their business processes will be able to add value to the company. According to Porter and Millar [39], IT is the conduit that links the processes within an organization and adds value to the company. Therefore, superior ®nancial performance will only be the reward of companies who have not simply completed IT projects but have successfully integrated IT into their business processes. One reason why we ®nd a ®nancial performance advantage is that the CWP100 companies are speci®cally evaluated based on how well the information system of the company is positioned to service its business needs. Our contribution is in terms of empirically testing and validating the proposition that mismanagement is another viable explanation for the productivity paradox. Using a quasi-experimental design allows us to examine to what extent companies that have been recognized by industry experts and their peers as successful users of IT will experience statistically signi®cant performance advantage relative to their competitors. The empirical results provide statistical support for our argument that successful investment in IT leads to superior ®nancial performance. 6. Limitations and suggestions for further research The issue of causality is problematic in a quasiexperimental design where the experimental and control groups are not randomly assigned. Consider the three classical requirements of causality, a signi®cant correlation between variables, a temporal ordering of events, and the ruling out of alternative explanations. In this study, we show the correlation between successful investment in IT and ®nancial performance. Using a matched pair design controls for alternative T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 explanations such as the effects of industry, size, and capital intensity. The temporal ordering of events is unclear. We argue that successful use of IT leads to improved ®nancial performance, but it is possible that improved ®nancial performance leads to successful use of IT. The logical sequence seems to be the former, but we have not ruled out the latter. The data in Table 7 show that the CWP100 companies started outperforming the control group years before appearing on the CWP100 list in 1993. We argue that this is because these companies were probably successful users of IT before 1993. We do not argue that the CWP100 companies became successful users of IT in 1993, and were previously not successful users of IT. Peer evaluation is a major component of the score used to compile the list. It likely takes a number of years to develop a reputation as a successful user of IT within your peer group, and therefore, companies appearing on the list have probably been successful users of IT for a number of years. A logical follow-up study would be to examine a small subset of ®rms on a case-by-case basis to try to identify the temporal ordering of ®nancial performance and successful use of IT. An additional area for future research is to examine other explanations for the productivity paradox. Bakos [2] gives four possible explanations for the productivity paradox viz. mismeasurement, mismanagement, diffusion delay, and the capital stock theory. In order to truly understand the productivity paradox, each explanation should be tested independently, and then together. Brynjolfsson and Hitt [7] addressed mismeasurement, and in this paper, we address mismanagement. Future research could focus on diffusion delay and the capital stock theory. 7. Summary Several empirical studies have had dif®culty relating investment in IT and ®nancial performance. This is often referred to as the productivity paradox. In this paper, we address this question using a quasi-experimental design comparing successful users of IT with less successful users of IT. This focus on successful use of IT was primarily to assess mismanagement as a possible explanation for the productivity paradox. 115 Examining 10 ®nancial performance measures over 10 years, we ®nd that, in general, successful users of IT outperform less successful users of IT for 3±4 years. It appears from our results, taken in the light of previous studies, that how you manage your IT assets is more important than how much you spend on IT. We expect that any performance advantage of the effective users of IT will tend to erode with time as competitors copy their IT investment and implementation. References [1] P. Alpar, M. Kim, Microeconomic approach to the measurement of information technology value, Journal of Management Information Systems 7 (2), 1990, pp. 29±54. [2] Y. Bakos, The productivity payoff of computers: a review of the computer revolution: an economic perspective by Daniel E. Sichel, Science 281, 1998, p. 52. [3] P. Barnes, Methodological implications of non-normally distributed financial ratios, Journal of Business Finance and Accounting 9 (1), 1982, pp. 51±62. [4] P. Barnes, Methodological implications of non-normally distributed financial ratios: a reply, Journal of Business Finance and Accounting 10 (4), 1983, pp. 691±693. [5] A. Barua, C. Kriebel, T. Mukhopadhyay, Information technology and business value: an analytic and empirical investigation, Information Systems Research 6 (1), 1995, pp. 3±23. [6] E. Brynjolfsson, The productivity paradox of information technology: review and assessment, Communications of the ACM 36 (12), 1993, pp. 67±77. [7] E. Brynjolfsson, L. Hitt, Paradox lost? Firm-level evidence on the returns to information systems, Management Science 42, 1996, pp. 541±558. [8] E. Brynjolfsson, S. Yang, Information technology and productivity: a review of the literature, in: Advances in Computers, Vol. 43, Academic Press, New York, 1996, pp. 179±214. [9] B. Caldwell, Study: Year 2000 spending up 20% to $858 billion, InformationWeek Online, 9 November 1998. [10] J. Cash, B. Konsynski, IS redraws competitive boundaries, Harvard Business Review 63 (2), 1985, pp. 134±142. [11] Computer Economics, 1999 Information Systems and eBusiness Spending, http://www.computereconomics.com, 1999. [12] Computerworld, Section 2 of 2, 13 September 1993, p. 45. [13] P. David, The dynamo and the computer: a historical perspective on the modern productivity paradox, American Economic Review 80 (2), 1990, pp. 355±361. [14] E.B. Deakin, Distributions of financial accounting ratios: some empirical evidence, Accounting Review 51 (1), 1976, pp. 90±96. [15] B. Dehning, T. Stratopoulos, Information technology and competitive advantage, Working Paper, University of New Hampshire, 2000. 116 T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 [16] B.L. Dos Santos, K.G. Peffers, D.C. Mauer, The impact of information technology investment announcements on the market value of the firm, Information Systems Research 4 (1), 1993, pp. 1±23. [17] Editor's Comment: Probing the productivity paradox, Management Information System Quarterly 18(2) (1994) 21±24. [18] G. Foster, Financial Statement Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1986. [19] T.J. Frecka, W.S. Hopwood, The effects of outliers on the cross-sectional distributional properties of financial ratios, Accounting Review 10 (4), 1983, pp. 115±128. [20] The Genesys Group, http://www.genesysgroup.demon.co.uk/, 1999. [21] J. Haimowitz, Has the surge in computer spending fundamentally changed the economy? Federal Reserve Bank of Kansas, Economic Review Second Quarter 1998, pp. 27±42. [22] L. Hitt, E. Brynjolfsson, Productivity, profit and consumer welfare: three different measures of information technology value, MIS Quarterly 20 (2), 1996, pp. 121±142. [23] R.V. Hogg, A.T. Craig, Introduction to Mathematical Statistics, 5th Edition, Prentice-Hall, Englewood Cliffs, NJ, 1995. [24] J.O. Horrigan, Methodological implications of non-normally distributed financial ratios: a comment, Journal of Business Finance and Accounting 10 (4), 1983, pp. 683±689. [25] C.M. Judd, G.H. McClelland, Data Analysis, a Model Comparison Approach, Harcourt, Brace and Jovanovich, Orlando, FL, 1989. [26] Killen & Associates, Worldwide Information Technology Spending Reached $1.6 Trillion in 1997, Will Grow at 10% to $2.6 Trillion in 2002, K Press Release, 17 September 1998. [27] KPMG Peat Marwick, Runaway Projects, http:// www.kpmg.co.uk/uk/services/manage/run.html, 1999. [28] T. Landauer, The Trouble with Computers, MIT Press, Cambridge, MA, 1995. [29] R.J. Larsen, M.L. Marx, An Introduction to Mathematical Statistics and its Applications, 2nd Edition, Prentice-Hall, Englewood Cliffs, NJ, 1986. [30] G. Loveman, Information Technology and the Corporation in the 1990's, MIT Press, Cambridge, MA, 1994. [31] M.G. Mahmood, I. Mann, M. Dubrow, J. Skidmore, Information technology investment and organization performance: a lagged data analysis, in: Proceedings of the 1998 Resources Management Association International Conference edited by M. Khosrowpour, Idea Group Publishing, Harrisburg, Pennsylvania 1998, pp. 219±225. [32] Massachusetts Institute of Technology, The Landmark MIT Study: Management in the 1990s, MIT Press (Cambridge, MA) in cooperation with Arthur Young, 1989. [33] F.W. McFarlan, Information technology changes the way you compete, Harvard Business Review 62 (3), 1984, pp. 98±103. [34] B. McNurlin (Ed.), Trends in Information Technology, Andersen Consulting, Chicago, IL, 1991. [35] K. Nash, Overnight services duke it out on-line, Computerworld, 22 April 1996. [36] E. Natalie, Keeping up with the Joneses, Computerworld, 24 February 1997. [37] S. Oliner, D. Sichel, Computers and output growth revisited: how big is the puzzle? Brookings Papers on Economic Activity 2, 1994, pp. 273±334. [38] M.E. Porter, Clusters and the economics of competition, Harvard Business Review 76 (6), 1998, pp. 77±90. [39] M.E. Porter, V.E. Millar, How information gives you competitive advantage, Harvard Business Review 63 (4), pp. 149±160. [40] S. Roach, The Economics of Competitive Advantage, Morgan Stanley International Investment Research, Morgan Stanley, New York, 1996. [41] T. Salmi, T. Martikainen, A review of the theoretical and empirical basis of financial ratio analysis, Finnish Journal of Business and Economics 43 (4), 1994, pp. 426±448. [42] M. Schrage, The real problem with computers, Harvard Business Review 75 (5), 1997, pp. 178±188. [43] The Standish Group, The CHAOS Report, http://www.standishgroup.com/chaos.html, 1995. [44] C. Stedman, Better IT spending oversight could save $90B, study claims, Computerworld, 23 March 1999. [45] B. Stewart, Enterprise Performance Through IT: Linking Financial Management to Contribution, IT Expo, Lake Buena Vista, FL, 1996. [46] P. Strassmann, The Business Value of Computers: An Executive's Guide, The Information Economic Press, New Canaan, Connecticut 1990. [47] P. Strassmann, The Squandered Computer, The Information Economic Press, New Canaan, Connecticut 1997. [48] P. Strassmann, Will big spending on computers guarantee profitability? Datamation, February 1997. [49] US Bureau of Economic Analysis, Survey of Current Business, July 1994. [50] M. Walsh, The air bill joins the 8-track, Internet World, August 1997. [51] P. Weill, The relationship between investment in information technology and firm performance: a study of the valve manufacturing sector, Information Systems Research 3 (4), 1992, pp. 307±333. [52] A. Yosri, The relationship between information technology expenditures and revenue contributing factors in large corporations, Doctoral Dissertation, Walden University, 1992. Theophanis Stratopoulos teaches MIS and Managerial Statistics at the University of New Hampshire's Whittemore School of Business and Economics. He received his BA and MA in Economics from the Athens School of Economics and Business, and his PhD from the University of New Hampshire. Professor Stratopoulos' research interests focus on applied statistical analysis in the areas of information technology and financial performance. Currently, he is working on a project investigating the sustainability of competitive advantage for companies investing in IT. His work has been published in the Journal of Post-Keynesian Economics, and International Advances in Economic Research. Bruce Dehning is an Assistant Professor of Accounting at the University of New Hampshire's Whittemore School of Business T. Stratopoulos, B. Dehning / Information & Management 38 (2000) 103±117 and Economics. He holds a BS in Finance, an MS in Accounting, and a PhD in Accounting from the University of Colorado. Professor Dehning's current research is on behavioral explanations for market phenomena and investment in information technology. He is currently working on projects investigating the individual 117 investor's reactions to earnings announcements, the market reaction to airline crashes, and the sustainability of a competitive advantage for companies that invest in information technology. Bruce has work experience in insurance and as an accounting information systems consultant for small businesses.