cso.online.com.au, Australia
09-28-06
Hoping the House Burns Down
Sue Bushell, CSO Online
Security strategy is as important as corporate strategy. Getting it right is critical.
Learn how detailed cost/benefit information can lead to more effective security
management decisions
Real-life stories abound about hapless CIOs and CSOs who breeze through the
task of convincing their ambitious but spending-averse boards and CEOs of the
value of CRM, ERP or wireless initiatives but struggle to get funding for the
infrastructure and technology to secure them.
Of course when the inevitable happens - the major system is breached, the
regulations are seriously, if accidentally, violated, the corporate reputation goes
down the toilet or revenues start to plummet - it's often tmhe hapless CSO who
wears the blame. Small wonder security executives stew when the message they
hear loud and clear is that if they want the resources to manage security risks,
they will have to somehow demonstrate a healthy return on investment (ROI).
No point telling the execs they should count reduced legal liabilities, fewer thefts
and better employee morale as ROI savings - they're just not equipped to think
that way. To many executives, ROI ain't ROI unless it delivers awesome cost
savings or new business returning more to the organization than the original
investment.
So convincing executive management of the merits of a stable and secure
network infrastructure is tough enough. Selling them on network security
initiatives can sometimes seem almost impossible.
"The business case issue around security is still a very real one in Australia, and
part of this is because of the lack of ability to translate the impact, or potential
impact, of a security issue to the business," says KPMG Australia associate
director Rob Goldberg. "And so there is, unfortunately, a number of organizations
- from the large end of town all the way down to the SMEs - who haven't been hit
who are kind of sitting there thinking: 'Well it hasn't happened to us, so why
should I make this investment?'"
In fact it currently seems like there are two types of organizations in this world:
the ones who resist spending on security because they have yet to be hit, and
those who see security investments as so inevitable they never even bother to
consider an ROI case. Still, in many organizations how to justify security
investments throughout the development cycle is becoming such a thorny issue
that a new group - the Application Security Industry Consortium - formed last
year just to tackle it.
Comprising representatives from Microsoft, SAP, Oracle, Red Hat, Gartner, the
Florida Institute of Technology and others, the group is mapping security
measures to business needs, and tracking other issues dear to the hearts of
CEOs and CIOs. Manufacturing Business Technology reports APPSIC hopes to
move the debate well beyond the use of scare tactics traditionally used to justify
security investments.
"We have two specific goals: to provide metric guidance, and deliver a
methodology for evaluating platform and application security," says APPSIC
chairman Herbert Thompson, who is chief security strategist for technology and
services provider and consortium member Security Innovation.
"Among the challenges are to show value for security activities when we're
building software, and to show customers how to determine value when they're
buying."
"We're trying to get people to understand security associated with software
applications" says IDC research director Charles Kolodgy, a fellow APPSIC
member. "Perimeter security has improved, and as people deploy more defences
in different areas, hackers have decided to attack applications much more than
they used to. We're attempting to assess the risk of applications and find
meaningful metrics for security."
Doug Jacobson, Director of the IOWA State University Information
Assurance Center, concurs. "Security is like insurance: it's hard to justify," he
says. "APPSIC is bringing that to the forefront by seeking models that people can
use to demonstrate potential ROI. This would go a long way toward making
applications more secure."
That "security-as-insurance policy" is a metaphor Cutter Consortium contributing
writer John Berry is entirely comfortable with. Any value from security measures
stems mostly from the costs and negative impacts the buyer manages to avoid
by warding off catastrophe, Berry notes in a paper called ROI Analysis of
Enterprise Risk Management and Governance.
"Attempting to quantify the economic value of security-related information
technology is a lot like understanding the value of an insurance policy. Value
stems less from what's delivered to the buyer and more from what costs and
negative impacts the buyer avoids should a catastrophic event occur. Avoiding
costs can be more powerful than saving or making money. The problem is that
quantifying avoided costs is only truly possible after disaster strikes. This reality
can lead to some twisted thinking: a desire to capture value from a security-
related technology is like hoping that your house will burn down so you can take
advantage of your homeowners' policy," he wrote.
Which begs the question: if it's "twisted thinking" to expect any monetary reward
for your security efforts, why bother doing ROI calculations at all? As Berry notes,
particularly where the threats seem huge (for example, phishing) and where
investment helps the organization meet regulatory requirements, or where the
law mandates that customer information be protected, the proposed security
technology morphs from a discretionary capital outlay to a must-have capability.
So why bother analyzing potential value? Why not move straight to a feature-set
analysis, determine if its functionality meets the law's requirements, and cut a
cheque?
How about because of the truth of the old adage that "if you can't measure it, you
can't manage it", Goldberg says. Or because security is bedevilled by the fact
there is so much any organization doesn't know that it doesn't know. Or because
while there's rarely a direct correlation to positive bottom line impact in the ROI
equation, organizations are starting to discover that the brand risk and trust risk
to organizations from a security breach is far greater than was once appreciated,
as can be the downstream negative revenue impacts.
"So the ROI then needs to be considered from a broader perspective," Goldberg
says. "Where there are links that you can draw to your balance sheet, by all
means do so that you measure that the technology investments you make are
working. But then you also need the qualitative measurements that show you that
the effectiveness of your technology investments and your procedural
investments are actually working to meet an expected level."
Revealing Exercise
The very act of undergoing an economic value analysis can help show where the
true value of the security technology rests, Berry says. It can also allow clearer
thinking about the costs the technology might help avoid and the likely economic
impact of current threats.
You can typically divide IT spending into an if-to-invest bucket and a when-toinvest bucket, Berry says. "If-to-invest technologies offer high risks and high
rewards: large cost reductions from process efficiencies, profit impacts from
revenue generation, re-engineered business models, and so on. They are also
highly strategic in nature and it takes a thorough cost-benefit analysis to
determine if they should go ahead.
"When-to-invest technologies - like databases, servers, networks, and HR
applications - on the other hand, offer both far fewer risks and lower rewards. But
since they're needed to keep the organizational 'lights on', the question around
investment becomes when, not if."
Security-related technology, Berry notes, falls into both camps. On the one hand,
security technology is a when-to-invest proposition, since managers perceive that
the risk of doing nothing in the face of growing information-asset threats is
enormous. On the other hand, security technology is also an if-to-invest
proposition with the flavour of a strategic technology since the deployment of a
specific kind of security technology can influence an organization's security
strategy.
"So since security technology exists in a kind of value assessment purgatory, is
there managerial benefit in analyzing the economic value of security
technology?" Berry's paper asks. "Certainly, but not because such an exercise
will help the organization decide to invest or not invest - we have already
conceded that most organizations will invest in security technologies regardless
of what the ROI or net present value (NPV) figure is."
Instead, he says companies should analyze the economic value of security
technology as if it were the most complex, risky, and strategic product around
because of what the effort will reveal to them.
Berry says an ROI analysis can illuminate the following ideas for managers:
»Because the value of security-related IT is commonly captured in cost
avoidance, accurate forecasts on the probability of a threat can help the
organization prioritize security technology investment.
»Any rigorous financial modelling should expose the hidden costs lurking under
the price tag for the security technology investment including the cost impact of
various types of organizational or process change introduced into the
organization by virtue of the investment. Exposing hidden costs is critical when
the investment offers no direct economic benefit because the cost side of the
ledger has a bigger influence on the ROI.
»People, process, and organizational change introduced by security technology
can have multiple impacts on the organization; economic analysis heightens an
awareness of these impacts before the investment is made.
»Managers get so caught up in forestalling real security threats to their
information and it infrastructures that project risks can easily be overlooked. An
ROI analysis reminds managers in organizations where project management
skills are limited that security technology implementation faces the same project
risks as any other IT project.
»Perhaps most importantly, economic analysis provokes a larger discussion and
review of the organization's entire security strategy.
"On a very practical level the analysis asks the organization: 'Are we equipped to
use the security related technology optimally?' In other words: 'Are we going to
use all of the functionality to the best of our ability and are the people that are
going to use this technology equipped take full advantage of the functionality of
the software?'" Berry writes.
Take the example of Fraud Detection Software (FDS), used to help organizations
discover potentially suspicious activity from online visitors and forestall an actual
security breach. FDS helps an organization pre-empt the real damage wrought
by phishing, hacking, or other security breaches by proactively flagging
suspicious Web site activity that is an early indicator of these disastrous
outcomes; and automates the investigative process of actually mining this data
for this suspicious activity.
Historically, organizations that had bothered to monitor Web server logs in
search of potentially compromising behaviour from outsiders relied upon
painstaking and time-consuming line-by-line file reviews from technical staff,
Berry notes. Under these circumstances would an ROI analysis help any
organization contemplating investment in this FDS technology?
Perhaps not, especially if the organization was experiencing the very kind of
information security breach this technology is supposed to help it avoid, Berry
says.
"The company might find itself in the throes of paranoia and panic. Economic
value analysis, meant to illuminate how value is expected to be achieved for the
money invested and the foundation of a decision to implement a particular
technology, is an extraneous data set to the beleaguered manager perceiving an
immediate need for a technological solution to a damaging security breach. If any
laws were broken as a result of the attack that would subject the organization to
fines only heightens the urgency to dispense with value analysis on the way to
procurement. Besides, some of the benefits from FDS are very difficult to
quantify.
"However, even if the organization under these circumstances does see an ROI
analysis as a distraction, the questions such an analysis asks of managers who
take security management seriously are still quite relevant. The exercise has
value as it provokes the kinds of questions that focus attention on how the
investment fits in with the current and future security strategy. Is the strategy
improved? Must it change in some way to accommodate the new security
technology?"
Often, the very act of asking such questions can be highly illuminating.
Here Be Dragons
While both Berry and Goldberg agree there can be much hidden value in
conducting ROI analyses of security technology, they say the landscape is
replete with potential stumbling blocks.
Goldberg says organizations questioning the ROI of security tend to make two
common mistakes. The first to focus on a purely quantitative analysis, which risks
failure to recognize the technology's business impact. For example you can't tell
whether your security risk profile has changed just by knowing the number of
monthly alerts generated by your intrusion detection software. If you received
500 alerts one month and only 300 the next, does that mean that the intrusion
detection is working well, or that it isn't working as it should be, since it may have
not even noticed some intrusions in month two? And an ROI analysis certainly
can't tell you if the one alert you should have responded to in month two, when
your numbers were down, was an extremely dangerous one.
"So there's some problems with trying to pull this stuff out of the technology, and
a lot of organizations go into this monthly reporting cycle trying to show that all of
this stuff is doing something, but not drawing a link between that and actual
impact on the business," he says.
But the other mistake is to not keep the metrics dynamic enough, Goldberg says.
Once you have some software working and you're getting some value add out of
the investment, you need to consider whether you should change the metric
because the risk profile or the business has changed.
And Berry warns that numbers in an ROI analysis can be extremely rubbery. "In
an ROI analysis you can come up with any number that you want to," he says.
"You know, this idea we are going to avoid all these costs, and it's going to mean
a $6 million cost avoidance if we invest in this. You extrapolate or rationalize any
kind of figures that you want; that doesn't mean that they're accurate and that's
one of the pitfalls of all of this.
"Many organizations get into the inherent difficulty in calculating cost avoidance
which is very much more art than science."
The only way to tip the balance in favour of science is to gather as much
independent empirically driven information and data as possible, Berry says. For
instance, the value of a reduction in viruses infiltrating the organization entirely
depends on the potential cost to the organization of the viruses in question. To
get a head start on doing those calculations, you might see what data is available
from the carrier companies that do risk insurance and the analysts houses. It's
true that many of these breaches are so novel that the carriers won't have deep
experience over a number of years to draw upon which would let other
organizations make reasonable extrapolations about the calculations they should
make within their own organizations. Berry concedes. Nonetheless, using
independent third-party information as the basis of your calculations whenever
possible provides you with a powerful analytic tool.
But Berry also cautions organizations against over-enthusiastic use of that tool.
Plenty of organizations suffer "analysis paralysis", he says, getting so bogged
down in such calculations that they never make the investment.
"Given the fact that a lot of companies don't even bother to do ROI analysis it is
not that much of a problem in the organizations, but it can be a risk," he says.
But it is also vital to have the expertise to do the return on the investment
analysis, he says. Without such experience, you can have the greatest third-party
information in the world, but still make a lot of calculation mistakes.
Above all, remember that when it comes to security, value stems less from what's
delivered to the buyer and more from the costs and negative impacts the buyer is
able to avoid in the event of a catastrophe.
"A thorough economic analysis is essential to achieve a complete understanding
of the entire range of costs, risks, rewards, and resource demands," Berry says.