The Disclosure and Diffusion of Security Information
Sabyasachi Mitra (saby.mitra@scheller.gatech.edu)
Sam Ransbotham (sam.ransbotham@bc.edu)
Working Paper: please do not quote or circulate
With the nearly instantaneous dissemination of information in the modern era, policies regarding the
disclosure of sensitive information have become the focus of significant discussion in several contexts.
The fundamental debate centers on tradeoffs inherent in disclosing information that society needs, but that
can also be used for nefarious purposes. Using information security as a research context, our empirical
study examines the adoption of software vulnerabilities by a population of attackers. We compare attacks
based on software vulnerabilities disclosed through full disclosure and limited disclosure mechanisms.
We find that full disclosure accelerates the diffusion of attacks, increases the penetration of attacks within
the target population, and increases the risk of first attack after the vulnerability is reported. Interestingly,
the effect of full disclosure is greater during periods when there are more overall vulnerabilities reported,
indicating that attackers may strategically focus on busy periods when the effort of security professionals
is spread across many vulnerabilities. Although the aggregate volume of attacks remains unaffected by
full disclosure, attacks occur earlier in the life cycle of the vulnerability. Building off our theoretical
insights, we discuss the implications of our findings in more general contexts.
Keywords: Information Security; Information Disclosure; Software Vulnerability; Diffusion of
Innovation; Negative Innovation;
1. INTRODUCTION
Information security remains gravely important (Anderson and Moore 2006; Arora et al. 2008). This
is highlighted by recent surveys (D'Arcy et al. 2009), government regulations (Schultz 2004), privacy
concerns (Bélanger and Crossler 2011), relentless high profile data breaches at major institutions (Ante
2010; Ramstad 2011; Tudor 2011), and event studies of companies that experience security breaches
(Cavusoglu et al. 2004). Vulnerabilities, primarily errors in software that run on computer systems, are an
important pathway for information security compromise (Arora et al. 2008; Cavusoglu et al. 2008).
Attackers can exploit these vulnerabilities to gain unauthorized access to systems, download sensitive
information, install malicious code, and disrupt normal operations. Consequently, the information security
industry focuses intensely on the discovery and disclosure of vulnerabilities by security professionals, on
the development of patches and other corrective measures by software vendors, and on the installation of
patches and countermeasures by target companies. While technical solutions are critical in developing
detection tools, patches and countermeasures, vulnerability disclosure policies implemented by security
agencies, vendors and governments have an important role in promoting a secure computing environment
(Arora et al. 2008; Cavusoglu et al. 2008).
When security professionals discover a new software vulnerability, they have two main mechanisms
for disclosing the information (Kannan and Telang 2005; Ransbotham et al. 2012). In the first
mechanism, termed limited disclosure, security professionals report the vulnerability to organizations
such as Computer Emergency Response Team (CERT) and other similar agencies (e.g., the vulnerability
markets iDefense or Tipping Point) that play a critical role in the reporting of vulnerabilities. After a
vulnerability is reported, these agencies inform the vendor immediately, but wait (usually 45-180 days)
before making the vulnerability public (Ransbotham et al. 2012). Thus, while vendors receive immediate
notification, security professionals and attackers are notified about the vulnerability at the time of public
disclosure. In the second mechanism, termed full disclosure, security professionals publicly disclose the
vulnerability immediately after discovery through public forums such as Bugtraq. In full disclosure,
vendors, security professionals and attackers receive information concurrently about the vulnerability.
A significant and on-going debate in the information security industry revolves around the benefits
and drawbacks of limited versus full disclosure (Anonymous 2010; Cavusoglu et al. 2007; Cooper 2001;
Lemos 2011; Messmer 2007). Proponents of limited disclosure argue that it ensures that vendors and
targets receive reasonable time to develop and deploy patches and countermeasures before systems are
attacked, while the alternative full disclosure creates an unsafe window of opportunity when systems can
be attacked before patches and countermeasures are deployed. Vendors typically support limited
disclosure and have been known to take action against those who violate limited disclosure norms (Lemos
2011). On the other hand, proponents of full disclosure argue that it provides incentives to vendors to
create better quality software (Arora et al. 2008). Proponents also argue that full disclosure notifies
security professionals so that they can install countermeasures immediately, while limited disclosure
creates a period when security professionals are unaware of a vulnerability that attackers may discover
and exploit independently. Unfortunately, limiting disclosure is inherently difficult and relies on obscurity
to provide advantage to defenders. But, obscurity hides information from security professionals as well,
while attackers can independently discover and then share vulnerability information (Mookerjee et al.
2011). Thus, it is unclear whether limited or full disclosure leads to a more secure computing
environment.
In this paper, we focus on this information disclosure debate. Specifically, we empirically compare
the two alternative disclosure mechanisms by analyzing a large, proprietary database of information
security alerts collected by a managed security service provider. The alert database contains over 2.4
billion information security alerts for 960 client firms of the managed security service provider and spans
two years (2006-2007). After deriving hypotheses through an analytical model and theoretical perspective
based on the diffusion of innovations literature, we assess four measures of the effect of full disclosure
versus limited disclosure of vulnerability information : (a) attack delay — does full disclosure speed the
diffusion of attacks corresponding to the vulnerability through the population of target systems, (b) risk of
first attack — does full disclosure increase the risk that a firm is attacked for the first time on any specific
day after the vulnerability is reported (given that it has not been attacked prior to that day), (c) attack
penetration – does full disclosure increase the number of target firms affected by attacks based on the
vulnerability, and (d) attack volume — does full disclosure increase the volume of attacks based on the
vulnerability? Furthermore, since multiple vulnerabilities may be disclosed at the same time, we also
consider the effect of multiple, concurrent vulnerabilities on the information security environment.
We find that full disclosure accelerates the diffusion of attacks corresponding to a vulnerability. Full
disclosure also increases the risk of first attack on any specific day after the vulnerability is reported.
Interestingly, the marginal effect of full disclosure on the risk of first attack increases when more
vulnerabilities are reported in a specific period, indicating that attackers may strategically focus on busy
periods when the effort of security professionals is spread across many vulnerabilities. Full disclosure
also increases the penetration of attacks within the population of target systems. Additionally, although
the aggregate volume of attacks remains unaffected by full disclosure, attack activity shifts earlier in the
life cycle of a vulnerability, thereby reducing its effective life span but intensifying activity while active.
These findings make important practical and research contributions. Practically, quantifying the net
effect of information disclosure on the diffusion of attacks informs the continuing debate about the
optimal disclosure of information security vulnerabilities. Empirical examination of the effect of full
versus limited disclosure of vulnerability information on the diffusion of information security attacks is
limited. Furthermore, by examining the indirect effects from multiple concurrently disclosed
vulnerabilities, we add depth to this debate and uncover a potential negative effect of full disclosure
hitherto unexamined. The short life cycle of full disclosed vulnerabilities also has policy implications
because its effect could be more pronounced for smaller organizations that might be less likely to install
countermeasures early. On February 12th, 2013, President Obama called for greater cyber security
information sharing among various public and private entities during his state of the union address. While
there are many positives of information sharing, our results quantify possible negative consequences
when information shared cannot be prevented from reaching attackers. From a research standpoint, we
also empirically evaluate “black hat” activity, complimenting prior information systems research that has
primarily relied on analytical analyses of vulnerability disclosure policies and on surveying “white hat”
perceptions (Mahmood et al. 2010).
More broadly, we also make significant contributions to the diffusion of innovation literature. The
importance of the diffusion of innovation is evident in the sizable academic and practitioner literature;
Rogers (2003, p. 477) notes more than 5,000 publications across practically every discipline, including
operations management (Ho et al. 2002; Loch and Huberman 1999), strategy (Greve 2011; Teece 1980),
marketing (Bass 1969; Van den Bulte and Joshi 2007), and information systems (Cooper and Zmud 1990;
Fichman and Kemerer 1999; Majchrzak et al. 2000; Nilakanta and Scamell 1990; Parthasarathy and
Bhattacherjee 1998). However, the overwhelming majority of innovation research implicitly promotes
diffusion of innovation, encouraging complete and rapid adoption (Rogers 2003). Unfortunately, not all
innovations are positive; there are also many innovations that society would like to discourage. In fact,
“one of the most serious shortcomings of diffusion research is its pro-innovation bias” (Rogers 2003, p.
106). Innovations that are undesirable for society continue to receive only a small fraction of the research
attention that positive innovations do; however, their diffusion is nonetheless critically important and
affects diverse areas such as terrorism (attack techniques), sports (performance enhancement drugs),
finance (value destroying schemes), and many others. Thus, we contribute to this literature by focusing on
the diffusion of a societally undesirable innovation (the exploitation of software vulnerabilities by
attackers), offsetting the almost exclusive attention to the diffusion of positive innovations. Second,
societally undesirable innovations give rise to two opposing diffusion processes – the diffusion of the
innovation itself (attacks corresponding to the software vulnerability in our context) and the diffusion of
countermeasures to protect against it. We explore through models and empirical analysis, the implications
of these two opposing diffusion processes that result from the disclosure of these innovation, an
interesting phenomenon that has not been examined in the innovation diffusion literature. Third, policies
regarding the initial disclosure of information have become the focus of significant debate in various
other contexts. For example, biologists who recently wanted to publish the methods they followed to
create a dangerous strain of the avian influenza virus stirred intense debate regarding the benefits and
drawbacks of publishing information needed by researchers and public health officials, but that could also
help bio-terrorists (Enserink 2011). Homeland security experts often promote voluntary self-restraint in
the disclosure of sensitive information, while others argue that such advice is misguided because it
withholds information from law enforcement (Boyd 2011). In the concluding section of the paper, we
discuss the implications of our analysis on information disclosure for these more general contexts, by
examining the boundary conditions that define when our findings are more likely to hold.
2. THEORY AND HYPOTHESIS DEVELOPMENT
2.1. Information Security Context
Most information security attacks exploit vulnerabilities in software that run on corporate systems
(Arora et al. 2008; August and Tunca 2008; Cavusoglu et al. 2008). The discovery of a vulnerability sets
off opposing diffusion processes, as one diffusion occurs within the population of security professionals
who protect systems and another diffusion occurs within the population of attackers who seek to exploit
the vulnerability to compromise systems. For example, when they become aware of the vulnerability,
vendors develop patches to correct the vulnerability (Arora et al. 2010). If a patch is available at the time
of public disclosure, security professionals (in target companies or in security service providers) install
the patches as expeditiously as possible without disrupting business operations, and often according to a
predefined and periodic patching schedule. If patches are not available or cannot be installed quickly, or if
the vulnerability cannot be corrected through patching, security professionals install other
countermeasures such as including vulnerability signatures in intrusion detection systems, closing certain
vulnerable ports, or disabling features in software and devices. Countermeasures (including patches) are
not installed immediately when available, but diffuse through the population of target systems over time,
initially adopted by a few target organizations (perhaps with sophisticated IT operations), and finally
reaching other organizations over time as countermeasures demonstrate relative advantage or are
observable. Meanwhile, among attackers, early adopters of the vulnerability (expert attackers) exploit the
vulnerability to gain access to a few target systems. Over time, they develop exploits and attack tools that
allow them to reach a larger number of target systems, and they eventually disseminate these exploits and
tools to novice attackers through websites and forums, making it easy to exploit the vulnerability
(Mookerjee et al. 2011; Ransbotham and Mitra 2009). Thus, akin to other technological innovations that
diffuse through a target population, both offensive (attacks) and defensive (patches and countermeasures)
actions diffuse through the population of target systems over time based on factors such as the
attractiveness of the target company, the size of its internet footprint, and the expertise of its security staff.
Computer systems at a target firm are at risk if attackers exploit the vulnerability before security
professionals deploy patches and countermeasures.
2.2. Modeling the Diffusion of Attacks and Countermeasures
Following Ransbotham et al (2012), we model the diffusion of attacks corresponding to a
vulnerability through the familiar s-curve that has been widely used in the diffusion of innovations
literature (Johansson 1979; Rogers 2003; Van den Bulte and Stremersch 2004) for several reasons. First,
attacks follow the familiar diffusion pattern with few attacks initially, followed by an intermediate peak as
the vulnerability becomes easier to exploit, and then fewer attacks as more systems are protected and
interest wanes in the attacker community (Ransbotham and Mitra 2009). Consequently, the cumulative
number of systems attacked follows the generalized s-curve similar to the diffusion of other innovations.
Second, as we discuss later, the s-curve allows us to model an intuitive element of the diffusion process
that is of interest in this study – the delay in the diffusion process associated with limited versus full
disclosure. A similar argument also applies to the heterogeneous adoption of countermeasures by firms.
Initially, a few firms with professionally managed information security environments adopt the
countermeasures, rapidly followed by an intermediate peak as adoption spreads through the relatively
defined group of security professionals, and subsequently fewer adoptions as the population of
unprotected systems decreases.
We also investigate an alternative model for the diffusion of
countermeasures explained later.
In the appendix, we derive the specific form of the s-curve shown below; an intuitive understanding is
sufficient here. For any specific vulnerability, let Na(t) be the cumulative number of attacked systems at
time t, Np(t) be the cumulative number of protected systems at time t, and N be the total number of target
systems. Let 𝑇!! be the time when half of the target systems have been attacked by exploiting the
vulnerability, and 𝑇!! be the time when half the target systems have had countermeasures installed to
protect against the vulnerability. Further, let Ra and Rp be the slopes of the s-curves for the cumulative
number of firms attacked and the cumulative number of firms protected, respectively. Note that in (1) and
(2) below, the time horizon is centered at the half-life of the attack (𝑇!! ) and countermeasure (𝑇!! )
diffusion processes, and varies from -∞ to +∞. This simply allows the curve to asymptotically approach
zero (when t = - ∞) rather than be exactly zero at t=0, and simplifies the form of (1) and (2) below. It also
takes into account the existence of zero day attacks such that the number of attacked systems at time t=0
is small but positive. Likewise, a few firms may independently discover and protect against the
vulnerability before public disclosure at time t=0. Thus, at the start of the process (t = - ∞), the cumulative
number of systems attacked (protected) is 0, and at the conclusion of the process (t = ∞), the cumulative
number of systems attacked (protected) approaches N. At time t=0 (public disclosure), the number of
systems attacked (protected) is small but positive. (As we explain later, in the empirical analysis, we
introduce a delay term that shifts the curve to the right so that 𝑁! 0 fits the observed number of attacks.) 𝑁! 𝑡 =
𝑁! 𝑡 =
!
!
!!! !!! (!!!! )
!
!!!
!!! (!!!!
!)
(1) (2) In addition to the above, we also explore an alternative scenario in the appendix where all firms adopt
countermeasures almost concurrently after a specific delay. In the modern security environment, there
may be less heterogeneity in the adoption of countermeasures, especially among larger and medium sized
enterprises. For example, vendors may download patches to all its subscribers within a short period of
time, managed security service providers can install similar protection across its client base at
approximately the same time, and security professionals in target companies usually try to expeditiously
install countermeasures as soon as possible without severely disrupting operations. To model this
scenario, we envision that all target firms adopt countermeasures almost concurrently after a delay (m)
after public disclosure. Thus, Np(t)=N if t ≥ m and 0 otherwise. In the remainder of this section, for
brevity of presentation, we derive our hypotheses based on (2). However, we describe in the appendix
how the alternative formulation leads to similar hypotheses and conclusions.
2.3. The Effects of Limited and Full Disclosure on Diffusion
Limited disclosure intends to introduce a delay in the diffusion of attacks since attackers may be less
likely to be aware of the vulnerability until it is publicly disclosed. CERT, for example, discloses the
reported vulnerabilities immediately to the vendor, while the vulnerability market mechanisms such as
iDefense and Tipping Point also include the encrypted signature of the vulnerability and other
countermeasures in the intrusion detection systems and advisories they provide to their subscribers. Thus,
limited disclosure may provide an advantage to defenders and speed the diffusion of countermeasures,
while full disclosure may provide an advantage to attackers and speed the diffusion of attacks through the
population of target systems.
The patent race literature provides support for the reasoning above. For example, analytical models
demonstrate that a firm with even an arbitrarily small head-start can preempt its rivals in a patent race
(Fudenberg et al. 1983). Harris and Vickers (1985) show that “if one player is far enough ahead of the
other….then the latter gives up completely, leaving the former to move to the finishing line at his own
pace.” Inventors who have made significant breakthroughs have been known to delay the introduction of
the first product to obtain a head start in developing subsequent products (Matutes et al. 1996) . In
summary, delay in disclosing information about an innovation is recognized as a way to preserve
advantage (Baker and Mezzetti 2005).
Hypothesis 1: Full disclosure of information about a software vulnerability accelerates
the diffusion of attacks corresponding to the vulnerability through the target population.
To model this difference, we include an additional delay term 𝑑 in the cumulative diffusion
expression for attacks, and we rewrite (1) as shown below. In (3), d = 0 for full disclosed vulnerabilities,
and d = 𝑑! for vulnerabilities disclosed through limited disclosure. A positive value of d shifts the curve to
the right and it also decreases the number of zero-day attacks at time t=0.
𝑁! 𝑡 =
!
!! !!!!
! !!
!!! !
(3) Hypothesis 1 predicts that 𝑑! > 0. However, there are at least two reasons why that may not
necessarily be the case. First, it is important to note that attackers can discover the vulnerability on their
own and share the information through the vulnerability black markets that exist (Mookerjee et al. 2011;
Radianti and Gonzalez 2007). Consequently, limited disclosure, which hides the vulnerability information
from security professionals for a period, may disadvantage security professionals and speed the diffusion
of attacks. Second, limited disclosure requires the sharing of vulnerability information with a subset of
defenders (software vendors and security organizations like CERT). Information leakage, either
accidental or malicious, may lead attackers to obtain the information while the majority of security
professionals remain unaware resulting in increased incentives to attack (Kannan and Telang 2005). Thus,
while we expect limited disclosure to provide an advantage to security professionals in their race to
protect systems before they are attacked, it is ultimately an empirical issue that we evaluate through data
analysis.
2.4. Probability of Successful Compromise
Not all attempted attacks are successful because a system may be protected before it is attacked. To
calculate the probability of successful compromise, we define the following two random variables for a
specific firm. Let 𝑇! be the time at which the firm is attacked through the exploitation of the focal
vulnerability, and let 𝑇! be the time at which the installation of countermeasures protects the firm from
the focal vulnerability. Remember that attacks and countermeasures diffuse through the population of
target systems based on the diffusion models depicted in (2) and (3). For simplicity of exposition, we
assume that countermeasures and attacks follow the same underlying diffusion rates. That is, we assume
that 𝑅! = 𝑅! = 𝑅 and 𝑇!! = 𝑇!! = 𝑇 ! in (2) and (3). (These assumptions do not change the intuition
behind our results but allows us to focus more effectively on the effect of delay d associated with limited
disclosure, derive closed form analytical expressions of the probability of compromise in terms of d, and
it increases readability. We relax these assumptions in the empirical analysis and allow R and 𝑇 ! to vary
based on vulnerability characteristics and disclosure method).
The probability of a successful compromise of any random firm is the probability that the firm is
attacked prior to countermeasures being installed (i.e. 𝑇! ≤ 𝑇! ). In the appendix, we derive an expression
for the probability of successful compromise, Pr 𝑇! ≤ 𝑇! , and describe the intuition here.
Pr 𝑇! ≤ 𝑇! =
!!(!!!)! !
(! ! !!)!
(4)
Figure 1 plots Pr 𝑇! ≤ 𝑇! as a function of d to explain the intuition behind (5). When d=0 (no
advantage to attackers or security professionals), attacks and countermeasures have identical diffusion
processes and probabilities of success (Pr 𝑇! ≤ 𝑇! = 0.5). As d increases (providing advantage to
security professionals), Pr 𝑇! ≤ 𝑇! decreases and asymptotically approaches 0. On the other hand, as d
decreases (providing advantage to attackers), Pr 𝑇! ≤ 𝑇! increases and asymptotically approaches 1. In
Figure 1, the point A (d=0) represents a vulnerability disclosed through full disclosure, while the point B
(𝑑 = 𝑑! ) represents a vulnerability disclosed through limited disclosure. Note that if the rates (R) and the
half-lives (𝑇 ! ) for the diffusion of countermeasures and attacks were not equal, Pr 𝑇! ≤ 𝑇! would
deviate from 0.5 when d=0, but the shape of the curve in Figure 1 would not be different. This is because
the probability of compromise should asymptotically approach 1 (or 0) as d approaches -∞ (or +∞), as
shown in Figure 1.
2.5. The Race between Expert Attackers and Security Professionals
In addition to the delay d = 𝑑! introduced by limited disclosure on the diffusion of attacks, two other
factors affect the diffusion of attacks and countermeasures, and consequently Pr 𝑇! ≤ 𝑇! . First, early
adopters (expert attackers) can speed the diffusion of attacks through additional effort by developing
attack methods quickly, employing social engineering methods, and widely disseminating attack tools
(Mookerjee et al. 2011; Ransbotham and Mitra 2009). The role of the expert attacker is similar to that of
social hubs in the diffusion of technological innovation who speed the diffusion process by popularizing
the innovation and making its adoption easier and more attractive to the broader population (Goldenberg
et al. 2009; Van den Bulte and Joshi 2007). Expert attackers can speed the diffusion of attacks through
additional effort in three ways. First, the existence of a vulnerability in software does not necessarily
mean that it can be exploited, since target firms usually have multiple layers of defenses in place. The
expert attacker can do additional research to discover ways in which existing defenses can be bypassed
and the vulnerability can be exploited in practice. Second, most vulnerabilities in modern software require
many steps to successfully compromise the targeted systems. Expert attackers can package the steps into
automated scripts or attack toolkits that more novice attackers can exploit. Third, expert attackers can
widely disseminate these tools through transient “black hat” websites and forums that exist for this
purpose (Mookerjee et al. 2011; Ransbotham and Mitra 2009; Swire 2004). In some cases, the motivation
to disseminate can be reputational (Ransbotham and Mitra 2009), but it can also be financial (e.g.
vulnerability information and toolkits can be sold on “black hat” websites for significant financial gain
(Ransbotham et al. 2012)). To incorporate the effort of expert attackers on Pr 𝑇! ≤ 𝑇! , we envision that
such effort reduces d by an amount 𝑑! (speeds up the attack diffusion process) and consequently
increases Pr 𝑇! ≤ 𝑇! .
Second, security professionals can also speed the diffusion of countermeasures through additional
effort. For example, software patches are typically installed according to a pre-defined schedule and
security professionals can expedite countermeasures by installing patches ahead of normal schedule. They
can also disable certain vulnerable services by creating and providing alternatives to such services.
Similarly, they can disable features in software and devices to make it more difficult for attackers to
exploit the vulnerability while managing the disruption in operations. To incorporate the effort of security
professionals, we envision that such effort increases d by an amount 𝑑! and consequently decreases
Pr 𝑇! ≤ 𝑇! . In summary, attacker effort moves d to the left in Figure 1, while security professional
(defender) effort moves d to the right. Thus, for full-disclosed vulnerabilities, 𝑑 = 𝑑! − 𝑑! , while for
limited-disclosed vulnerabilities, 𝑑 = 𝑑! + 𝑑! − 𝑑! .
2.6. Strategic Allocation of Effort by Expert Attackers
Given a set of vulnerabilities disclosed through full disclosure and limited disclosure, our primary
interest lies in determining how early adopters (expert attackers and security professionals) divide their
effort between such vulnerabilities. Consider that there is a single vulnerability (F) disclosed through full
disclosure and a single vulnerability (L) disclosed through limited disclosure (extending the analysis to
multiple vulnerabilities of each type does not change the intuition behind the analysis but reduces
tractability of the analysis). In the appendix, we show that if limited disclosure provides a sufficient and
non-trivial delay in public disclosure (i.e. 𝑑! is sufficiently large), the equilibrium is such that both the
security professional and the expert attacker expend all their additional effort on the full disclosed
vulnerability. More specifically, let 𝑑!! and 𝑑!! be the amount by which the expert attacker speeds the
diffusion of the full-disclosed (superscript F) and the limited-disclosed (superscript L) vulnerability,
respectively. Likewise, Let 𝑑!! and 𝑑!! be the amount by which security professionals accelerate the
diffusion of countermeasures for full-disclosed (superscript F) and limited-disclosed (superscript L)
vulnerability, respectively. Since there are limited number of expert attackers and security professionals,
they have limited capacity (denoted by 𝐷! and 𝐷! , respectively). In the appendix, we show that at
equilibrium, 𝑑!! = 𝐷! , 𝑑!! = 𝐷! , 𝑑!! = 0, and 𝑑!! = 0.
The intuition behind this equilibrium is important. At the start of the process, with no additional effort
from security professionals and expert attackers, full disclosed vulnerabilities are at point A (d=0) in
Figure 1, while limited disclosed vulnerabilities are at point B (𝑑 = 𝑑! ) in the figure. The slope of the
curve is negative throughout the range and is minimized (maximum downward slope) at d=0. Thus, both
security professionals and expert attackers focus on full-disclosed vulnerabilities because the marginal
effect of effort on the probability of compromise is greater for full-disclosed vulnerabilities than that for
limited disclosed vulnerabilities. The equilibrium solution does not imply that the diffusion of attacks and
countermeasures for limited disclosed vulnerabilities stop; rather, they proceed at the normal rate without
being expedited through additional effort.
Our second hypothesis follows directly from this equilibrium. Since expert attackers devote greater
effort on full disclosed vulnerabilities, the risk of first attack for a firm on any specific day after the
vulnerability is disclosed is greater for such vulnerabilities. The fundamental intuition is that expert
attackers find such vulnerabilities more attractive because additional effort on their part has a greater
effect on the probability of compromise.
This reasoning also has intuitive support in the patent race literature. In their seminal work on the
speed of R&D and innovation, Dasgupta and Stiglitz (1980) examine the effect of competition in R&D on
the amount of research. They find that competition (free entry into research) will always lead to more
research than in a pure monopoly, and under certain circumstances, it may result in excessive
expenditures on R&D relative to the social optimum. Full disclosure of vulnerabilities informs the
vendor, attacker and security professional at the same time and can be viewed as increasing the
competition between attackers and defenders when compared to limited disclosure. Increased competition
leads to greater effort by attackers and a greater risk of first attack.
Hypothesis 2: Full disclosure of information about a vulnerability increases the risk of
first attack corresponding to the vulnerability for a target firm.
It is important to note that while we expect full disclosure to increase the risk of first attack, this may
not necessarily be the case. For example, if attackers independently discover vulnerabilities disclosed
through limited disclosure, they may choose to focus on such vulnerabilities because security
professionals are unaware of such vulnerabilities, and attackers know that security professionals are likely
to focus on full disclosed vulnerabilities. Further, our arguments above require a non-trivial delay
associated with limited disclosure, and may not hold when this delay is small. Thus, the effect of full
disclosure on the risk of first attack is an empirical issue that we evaluate through data analysis.
2.7. Strategic Choice and the Workload of Defenders
Since proponents of limited disclosure often argue that full disclosure puts excessive pressure on
defenders to protect against the vulnerability, it is important to understand the effect of full disclosure
during periods of high activity when defenders (security professionals) have less time to devote to such
vulnerabilities. Our basic premise is that while attackers can strategically choose their focus, defenders
have less flexibility in exercising this choice. Thus, during periods when many vulnerabilities are
concurrently disclosed, security professionals must devote some of their time to other vulnerabilities and
may not be able to focus exclusively on full disclosed vulnerabilities. Company policy, security
guidelines and legal requirements often dictate a minimum level of effort by security professionals to
protect against each disclosed vulnerability. This implies that during periods of high workload, the
additional capacity (Dp) available to security professionals to expedite the diffusion of countermeasures
for a specific full disclosed vulnerability is effectively lower, thereby increasing the probability of
compromise based on that vulnerability. Attackers take advantage of such periods by diverting additional
resources towards full disclosed vulnerabilities from other activities, strategically increasing their focus
on such vulnerabilities and their available capacity (Da) to focus on such vulnerabilites. This leads to
more attacks and consequently a higher risk of first attack during periods when there are more active
vulnerabilities reported. Fundamentally, the logic above presumes that attackers are strategic in their
choice of vulnerabilities to exploit and defenders have less flexibility, which may not necessarily be true.
Thus, while we expect the following hypothesis to hold, we investigate it through empirical analysis.
Hypothesis 3: The effect of full disclosure of information about a vulnerability on the risk
of first attack will be greater during periods of high workload for defenders.
2.8. Novice Attackers and Volume of Attacks
While the risk of first attack is driven by expert attackers, attack volume is primarily driven by the
majority—the large number of novice attackers who utilize the exploit tools created and disseminated by
expert attackers. With the wider availability of tools to exploit a vulnerability, the marginal cost of attack
is low for the novice attacker. However, more systems are also protected against the vulnerability over
time, decreasing the likelihood of success and the expected marginal payoff from the attack. The effective
lifetime of the vulnerability is the time when the marginal payoff from the attack becomes less than the
marginal cost of the attack for the average novice attacker. Since security professionals focus their efforts
on full disclosed vulnerabilities, the diffusion of countermeasures are expedited for such vulnerabilities.
Consequently, the window of opportunity for the attacker is reduced as more firms become protected
against the vulnerability quickly, and the expected marginal payoff from attack decreases rapidly over
time for full disclosed vulnerabilities. Thus, the effective life span of a full disclosed vulnerability is
shorter and most attacks occur earlier in its lifecycle. The logic above assumes that security professionals
focus more on full disclosed vulnerabilities leading to a shorter window of opportunity for attackers, a
logic that we test empirically through the following hypothesis.
Hypothesis 4: Full disclosure of information about a vulnerability will shorten its
effective life span such that a greater proportion of the attack volume corresponding to
the vulnerability will occur earlier in its lifecycle.
3. DATA
3.1. Data Sources
Our data set for this research is created from two main sources. First, we use a proprietary database of
alerts generated from intrusion detection systems (IDS) installed in client firms of a managed security
service provider (MSSP) during 2006 and 2007. This data was made available to us by the MSSP after
removing any client identifiers. Intrusion detection systems are a valuable source of information for
investigating Internet-based attack activity (Cavusoglu et al. 2005; Ransbotham and Mitra 2009;
Ransbotham et al. 2012). The dataset contains a large volume (billions of alerts) of real alert data (as
opposed to data from a research setting) from 960 client firms with varied infrastructure across many
industries. We summarize the data into a panel dataset containing the number of alerts generated every
day during the two-year period of our analysis, for each target firm and a specific vulnerability. That is,
each data point in our dataset is for a specific target firm-vulnerability combination, and it contains a
count of the number of alerts generated for each day in the two-year period of the study (2006-2007).
Second, we combine this panel dataset with information in the National Vulnerabilities Database
(NVD 2008) to obtain detailed characteristics of the vulnerabilities we study. The NVD consolidates data
from several other public vulnerability data sources such as Computer Emergency Response Team
(CERT), Bugtraq, XForce and Secunia (NVD 2008). Vulnerabilities in the NVD are assessed by experts
using a Common Vulnerability Scoring System (CVSS) (Mell et al. 2006; Mell et al. 2007). The CVSS is
a mature, well-established metric that categorizes the fundamental characteristics of each vulnerability
using a defined list of attributes (Frei et al. 2006). The characterization of each vulnerability is openly
inspected by differing entities (such as security firms and software vendors); see Mell et al. (2007) and
Ransbotham et al. (2012) for additional details. We match the records in our panel dataset with the data in
the NVD through a CERT assigned unique ID for each vulnerability. We use the vulnerability attributes
from the NVD data as controls in our empirical analysis to ensure that the results we observe are due to
differences in the disclosure mechanisms (full versus limited disclosure), and not due to differences in
vulnerability and target firm characteristics, or changing attacker preferences over time. The control
variables are described below and shown in italics.
3.2. Full versus Limited Disclosure
Our focal variable (Full_Disclosure) is set to 1 if the initial disclosure was made through a public
forum and 0 otherwise. The NVD provides the disclosure history of a vulnerability that shows the dates
and forums where the vulnerability was disclosed. Among the disclosure forums listed in NVD for the
vulnerabilities in our sample, Bugtraq and Full Disclosure are public forums that notify all parties
simultaneously, while other forums in NVD (e.g. CERT, iDefense, Secunia, XForce) delay public
disclosure. Thus, we classify a vulnerability as full disclosed if it is first reported on a public forum, even
if it is subsequently reported through other non-public reporting agencies.
We first identify all
vulnerabilities that were ever reported through a public forum by manually examining the disclosure
history of the vulnerability. We then eliminate vulnerabilities from this list that were first reported
through the non-public forums and not broadcast first through a public forum. (While the Full Disclosure
public forum has become more popular recently, an overwhelming majority of our exploited fulldisclosed vulnerabilities in the sample were reported through Bugtraq in 2006-2007.) This list constitutes
our list of full-disclosed vulnerabilities. It is important to point out that full disclosure is possible through
other public forums (such as websites, blogs, social media etc.) that are not reported in the NVD
disclosure history. This introduces some noise in the classification of vulnerabilities and makes it more
difficult for us to find the differences between full and limited disclosed vulnerabilities that we observe in
the empirical analysis. Thus, our results will be stronger if we are able to perfectly classify vulnerabilities.
3.3. Control Variables
In addition to the focal variable (Full_Disclosure), we use several control variables in our analysis to
incorporate alternative explanations based on (a) vulnerability characteristics, (b) environmental
characteristics, and (c) firm and time fixed effects. The control variables are described below.
Vulnerability characteristics: Once the attacker has access, vulnerabilities require varying degrees of
complexity to exploit; experts categorized these as Low, Medium or High Complexity and we include
control variables for medium and high complexity, with low complexity as the base type. The Impact of a
vulnerability is categorized by experts into one or more categories (Confidentiality Impact, Integrity
Impact and Availability Impact), and we use an indicator variable for each impact category that is set to 1
if the potential for the specific impact is present, 0 otherwise. The NVD classifies vulnerabilities into
several different Defect Types based on the software defect that the vulnerability represents (Input
Validation, Design, Exception), and we used indicator variables to control for each defect type. We also
include the Age of the vulnerability (log transformed) at the time of our analysis (measured by the number
of days since the vulnerability was reported) to control for any age related effects. We include an
indicator (Market) if the vulnerability was disclosed through a market that pays security professionals for
reporting vulnerabilities.
Prior research has indicated differences in the diffusion of attacks for
vulnerabilities reported through market based mechanisms (Kannan and Telang 2005; Ransbotham et al.
2012). Expectations of success also influence attacker behavior. Therefore, the countermeasures available
to defenders may influence attack activity. Some vulnerabilities have an associated signature that can be
used by defenders to detect attacks based on that vulnerability.
We include an indicator variable
(Signature) that is set to 1 if a signature was available at the time that the vulnerability was disclosed, 0
otherwise. Similarly, we also include an indicator (Patch) if the software vendor had a corrective patch
available to eliminate the vulnerability on the focal day of analysis. Software products primarily run on
either individual desktop computers or servers. An attacker may expect larger reward from exploiting a
vulnerability in server based software because of the potential for more valuable content, or may be more
interested in desktop computers because of less sophisticated countermeasures. An additional variable
(Server) indicates whether the software corresponding to the vulnerability is desktop (0) or server (1)
based.
Environmental Characteristics: Vulnerabilities do not exist in isolation. Instead, at any time, there
are many vulnerabilities that attackers can choose to exploit. There are two ways this may affect an
attacker’s response – through the alternatives available to the attacker and the workload of the defender.
First, the presence of other vulnerabilities offers a greater number of alternatives for attackers to exploit.
We reflect the presence of these alternatives through an index developed by the NVD. The index is
calculated by totaling the number of vulnerabilities disclosed in the last 30 days, and then weighting
vulnerabilities by their severity (based on their aggregate CVSS score.) We calculated the Alternatives
variable using this formula for every day in our focal period. Second, we develop a more immediate form
of this index (Workload) that uses the same formula, but is restricted to the two days surrounding the
vulnerability disclosure. Through the narrow window, this index captures the workload on security
professionals to respond to vulnerability disclosures and incorporate countermeasures. (There is no
specific theoretical guidance for the sizes of the time windows; however, our results are robust to
alternative sizes as long as the Workload window is relatively short (less than 5 days) and the Alternatives
window is substantially larger than the Workload window, as would be expected.).
4.
METHODS AND RESULTS
Table 1 shows the descriptive statistics for selected control variables in our sample of 1201
vulnerabilities included in our alert database that we could match with NVD to obtain data on
vulnerability characteristics. Table 2 shows the correlations between selected focal and control variables.
Of the 1201 variables in the alert database, only 333 were exploited through attacks on 960 clients of the
managed security service provider during 2006-2007. We excluded firms from the sample that had more
than one intrusion detection system to avoid bias in the analysis.
4.1. Delays in Attack Diffusion
To evaluate Hypothesis 1, we construct a panel dataset with each exploited vulnerability as the panel
variable. We aligned each of the 333 exploited vulnerabilities with day 0 representing the date that the
vulnerability was disclosed. For each vulnerability and for each date after day 0, we calculated the
cumulative number of firms that had experienced exploitation attempts based on the vulnerability until
that date, to build a panel dataset of 132,768 observations. Since day 0 is not the same for all
vulnerabilities, the panel was unbalanced with some vulnerabilities having more observations than others
in the 2 year period.
We utilize this panel data set to estimate the following equation derived from (3) through non-linear
least squares estimation, with Pa (penetration), Ra and da as linear functions of the focal (Full_Disclosure)
and other control variables (see Table 3). Pa is the penetration of attacks in the population of target
systems and the empirical models below allow us to evaluate if Pa, Ra, and da varies based on the type of
disclosure. Since the unit of analysis is a specific vulnerability, all control variables corresponding to
vulnerability characteristics are included in the analysis. Note that in (8), we allow the penetration of
attacks (Pa) to be different from the total population of target systems (N).
𝑁! 𝑡 =
!!
!!! !!! !! !!
(5)
𝑑! = 𝛽!! + 𝛽!! 𝐹𝑢𝑙𝑙 𝐷𝑖𝑠𝑐𝑙𝑜𝑠𝑢𝑟𝑒 +
!
! 𝛽! 𝑐𝑜𝑛𝑡𝑟𝑜𝑙!
𝑅! = 𝛽!! + 𝛽!! 𝐹𝑢𝑙𝑙 𝐷𝑖𝑠𝑐𝑙𝑜𝑠𝑢𝑟𝑒 +
!
! 𝛽! 𝑐𝑜𝑛𝑡𝑟𝑜𝑙!
𝑃! = 𝛽!! + 𝛽!! 𝐹𝑢𝑙𝑙 𝐷𝑖𝑠𝑐𝑙𝑜𝑠𝑢𝑟𝑒 +
!
! 𝛽! 𝑐𝑜𝑛𝑡𝑟𝑜𝑙!
(6) (7) (8)
Note that the constant term in (6) incorporates the term (𝑅! ∗ 𝑇!! ) in (1). The non-linear least squares
estimation works as follows. Using initial values of the parameters (β), it calculates the value of da, Ra and
Pa based on (6)-(8) for each vulnerability and for each day, and then calculates the value of the dependent
variable 𝑁! 𝑡 using (5). Using a hill-climbing procedure, it then adjusts the parameter values to
minimize the least square deviations from the observed values of the dependent variable. Table 3 shows
the results of the estimation and the parameter estimates. The coefficient for the Full Disclosure variable
(𝛽!! ) in column da is negative, indicating that full disclosure reduces the delay associated with the
diffusion of attacks. Full disclosure also increases penetration of attacks in the population of target
systems (the coefficient for the Full Disclosure variable is positive in Column Pa).
To ease the interpretation of the estimated parameters and evaluate economic significance, Figure 2
plots the resulting diffusion curves for full disclosed and limited disclosed vulnerabilities based on the
estimated coefficients and all control variables set to their median values. The graphs illustrate that full
disclosure reduces diffusion delay by approximately 4 days. Further, it is interesting to note that the
diffusion curve corresponding to full disclosure intersects the y-axis at a positive and non-zero value,
indicating that attacks (known as zero day attacks) may occur on the day of disclosure or before. Thus,
our results support Hypothesis 1.
It is also interesting to note that full disclosure increases the penetration of attacks corresponding to
the vulnerability from 7% to 11% of target firms, an increase in penetration of 57%. This large increase in
penetration is important for two reasons. First, it shows greater activity by attackers for full-disclosed
vulnerabilities. Second, as greater number of target systems are attacked, the probability of successful
compromise increases for the attacker.
4.2. Risk of First Attack
We examine the risk of first attack through a Cox proportional hazard model with the first
exploitation attempt of a vulnerability for a specific firm as the event being explained. All vulnerabilities
were aligned with day 0 representing the date the vulnerability was disclosed. We construct a data set that
has for every vulnerability-firm combination, the specific day after day 0 of the first attempt to exploit the
vulnerability for that firm. In this analysis, it is also possible to utilize data on vulnerabilities that were
never exploited for a specific firm or for any firm in the sample. Thus, with 1201 vulnerabilities and 960
firms, we have 1,152,406 client-firm combinations. For some vulnerability-firm combinations, we have
the day when the specific vulnerability was first exploited at the specific firm, while for others there was
no exploitation attempt in the study period.
The Cox proportional hazard model estimates the likelihood (hazard rate) that a specific vulnerability
is exploited (failure event) at a specific firm on the focal day, given that it has not been exploited at that
firm prior to that day. In the Cox proportional hazard model, this hazard rate consists of (a) a baseline
hazard function that captures how the risk of first attack generally changes over time, and (b) a set of
parameters for each control and focal variable that describes how each variable affects the baseline hazard
function. All control variables corresponding to vulnerability and environmental characteristics are
included in the analysis. To incorporate unobserved differences across client firms, we stratify the
analysis so that the baseline hazard function can vary by firm and incorporate any unobserved firm
specific attributes. This is similar to a firm fixed effects analysis in ordinary regression models.
Table 4 shows the results from the Cox proportional hazard model. Model 0 introduces only the
control variables, while Model 1 introduces our focal variable (Full Disclosure). The coefficient for the
Full Disclosure variable is positive (β = 0.19, p<0.001), indicating that full disclosure increases the risk of
first attack on any day. Based on the estimated parameters, full disclosure increases the risk of first attack
by 20% (e0.19 = 1.2), an economically significant increase. We conclude that our results support
Hypothesis 2.
Table 4 Model 2 introduces the interaction term (Full Disclosure*Workload) to the hazard model.
Workload measures the number of vulnerabilities disclosed during the two-day period surrounding day 0
of the focal vulnerability (weighted by the severity of the vulnerability). It is a measure of the workload of
security professionals during the time of disclosure of the focal vulnerability. The coefficient of the
interaction term is positive and significant, indicating that the effect of full disclosure is greater during
periods of higher workload for security professionals. Based on the estimated coefficient (β = 0.25,
p<0.001), full disclosure increases the risk of first attack by 28% (e0.25 = 1.28), and for every additional
vulnerability disclosed during the two-day period surrounding day 0 of the focal vulnerability, the effect
of full disclosure increases by an additional 20% (e0.19 = 1.2). We conclude that our results support
Hypothesis 3. Interestingly, we also find that the direct effect of the Workload and Alternatives variables
are negative, indicating that when there are more vulnerabilities disclosed around day 0 of the focal
vulnerability, the risk of first attack decreases for the focal vulnerability, perhaps because the attacker has
more options to choose from1.
4.3
Full Disclosure and the Volume of Attacks
To evaluate Hypothesis 4, we construct a data set that has for each of the 960 firms and each of the
1201 vulnerabilities, the number of alerts generated on each day of our research period. The number of
alerts generated is a measure of the volume of activity by novice attackers, since activity volume is
primarily driven by novice attackers. As before, we align all vulnerabilities so that day 0 represents the
day the vulnerability was first disclosed. We use a Poisson model conditioned on exploitation (since only
333 of the 1201 vulnerabilities are exploited during the two year research period) to estimate the effect of
full disclosure on the volume of attacks. In Table 5, Model 0 introduces the control variables, while
Model 1 introduces the Full Disclosure variable. Interestingly, the coefficient of Full Disclosure in Model
1 is slightly negative and marginally significant (β = -0.125, p<0.05) indicating that full disclosure
reduces the volume of attacks by 12% (e-0.125 = 0.88). Model 2 introduces the interaction term (Full
Disclosure*Age) where Age is the number of days since day 0 for the focal vulnerability. The coefficient
estimate for the Full Disclosure in Model 3 is positive and significant (β = 2.33, p<0.001) indicating that
at the beginning of the lifecycle of the vulnerability (Age = 0), full disclosure increases the volume of
attacks by tenfold (e-2.33 = 10.3). The coefficient for the interaction term (Full Disclosure*Age) in Model 3
is negative and significant (β = -0.56), p<0.001) indicating that for each day after day 0, the effect of full
disclosure on attack volume decreases by 43% (e-0.56 = 0.57). The results indicate that the volume of
attacks for full disclosed vulnerabilities occur during the early stages of their lifecycle. We conclude that
our results support Hypothesis 4.
4.4
Robustness Checks Using Matched Samples
There is a possibility that the higher risk of first attack of a vulnerability could affect the choice of its
disclosure method. For example, if a security professionals suspect that a vulnerability has a very high
risk of first attack, they may choose full disclosure to quickly inform other security professionals so that
they can immediately install countermeasures for protection. If so, the higher risk of first attack that we
observe for full disclosed vulnerabilities could result from inherent differences between full versus
limited disclosed vulnerabilities and not from the choice of the disclosure method. A similar argument can
1
Since defenders may increase effort when a vulnerability is first observed, we also used an alternative measure
of workload on the day the attack is first seen in the firm or in the sample (if it is not exploited for a firm). For
vulnerabilities that were never exploited, we used the workload based on the publication date of the vulnerability as
before. The results with this alternative measure of workload are consistent with those in Table 4. We are grateful to
an anonymous reviewer for suggesting this additional analysis.
also be made regarding our analysis of attack volume. If attacks corresponding to a vulnerability are
likely to occur early in its life-cycle, the security professional who discovers the vulnerability may choose
full disclosure to inform other security professionals expeditiously.
To partially correct for this potential bias in our analysis of the risk of first attack and attack volume,
we match vulnerabilities into groups using the Coarsened Exact Matching Software (CEM) described in
(Blackwell et al. 2009). We matched vulnerabilities using all time-invariant, indicator variables available
to us (Table 6). Thus, vulnerabilities in each group had exactly the same value for all the variables listed
in Table 6 (as evidenced by the post-matching zero mean imbalance reported in the table). We dropped
groups that did not have at least one full disclosed and one limited disclosed vulnerability. Thus, this
matching process ensures that vulnerabilities in each group are the same as each other based on a variety
of factors and any residual differences observed between full and limited disclosed vulnerabilities are
likely due to the disclosure method. The procedure identified 27 groups in our dataset.
In the Cox proportional hazard model analysis reported in Table 7, we allow the baseline hazard to
vary based on these 27 groups and based on the target firm. Thus, any firm level or group level
unobserved factors are incorporated in the baseline hazard, while the parameters for the other focal and
control variables indicate how they affect the baseline hazard. The results reported in Table 7 are similar
to those in Table 3, indicating support for our hypotheses. Similarly, in the Poisson regression analysis of
the volume of attacks in Table 8, we introduced fixed effects for the vulnerability groups (in addition to
the fixed effects based on the firm and the month of attack). The results are similar to those in Table 4 and
support our hypotheses.
5. SUMMARY AND CONCLUSIONS
In this paper, we examine the effects of full versus limited disclosure of vulnerability information on
the diffusion of attacks corresponding to the vulnerability. Specifically, we analyze intrusion detection
system data for 960 clients of a managed security service provider corresponding to 1201 software
vulnerabilities during 2006-2007. We find evidence that when compared to limited disclosure, full
disclosure of a vulnerability leads to (a) less delay in the attack diffusion process for the vulnerability, (b)
greater penetration of attacks corresponding to the vulnerability among target systems, and (c) higher risk
of first attack on any specific day after the vulnerability is reported. Further, we find that the effect of full
disclosure on the risk of first attack is greater during periods when more vulnerabilities are reported,
indicating that attackers may strategically take advantage of busy periods when the efforts of security
professionals are diffused across many vulnerabilities. In addition, while the total volume of attacks
corresponding to a vulnerability (indicative of the level of participation by novice attackers) is marginally
affected by full-disclosure, attacks occur earlier during its lifecycle for vulnerabilities reported through
full versus limited disclosure.
5.1. Implications for Information Security
A key debate in the security industry is whether full or limited disclosure of vulnerabilities leads to a
more secure computing environment. Proponents of limited disclosure argue that it provides software
vendors time to develop patches and to security professionals to deploy patches before vulnerability
information is widely distributed to attackers. Thus, proponents argue that limited disclosure provides an
advantage to security professionals in their race to protect systems before they are attacked. On the other
hand, proponents of full disclosure argue that limited disclosure hides information from security
professionals, while attackers may independently discover and share vulnerability information through the
Internet, thereby disadvantaging security professionals in their race to protect systems before they are
attacked. We shed light on this debate through systematic analysis of intrusion detection system data.
Overall, we find that full disclosure expedites the onset of attacks corresponding to a vulnerability,
increases the penetration of attacks among target systems, increases the risk of first attack, and shifts the
volume of attacks corresponding to a vulnerability to earlier in its lifecycle. This is significant because
attacks that occur early and with less delay may be more likely to be effective since fewer systems are
likely to be protected against the vulnerability when attacked. Even though limited disclosure hides
information both from attackers and security professionals, our empirical results indicate that, despite
legitimate concerns about reliance on obscurity, limited disclosure may be currently providing some
practical advantages to security professionals in their race to protect systems. However, as we note in the
limitations section, this is only one aspect of the overall information disclosure debate.
Another important policy question centers on the effect on the security environment if more
vulnerabilities are disclosed through full disclosure. Many security experts encourage full disclosure to
promote openness and immediate sharing of vulnerability information. If security professionals do indeed
heed such advice, more vulnerabilities will be disclosed through full disclosure in the future. We find that
the effect of full disclosure on the risk of first attack is greater during busy periods when more
vulnerabilities are reported. Our analytical models predict that security professionals expend greater effort
on full disclosed vulnerabilities. Thus, if more vulnerabilities are indeed reported through full disclosure,
thereby increasing the workload of security professionals, we contend that the effect of full disclosure on
the risk of first attack for each reported vulnerability will be greater. Overall, we conclude that wide scale
usage of full disclosure mechanisms may strain security professionals as they will need to expend greater
effort on each reported vulnerability.
There are two primary contributions of this research to the information security management
literature. First, while several analytical models have examined optimal vulnerability disclosure and
patching policies (Arora et al. 2008; Kannan and Telang 2005; Mookerjee et al. 2011), ours is one of a
few that empirically evaluates disclosure policies through an analysis of intrusion detection system data.
This interplay between analytical and empirical research is critical in advancing our understanding of the
information security environment (Willison and Warkentin 2013). Second, empirical research on
information security in the information systems literature has primarily focused on surveys of users and
security professionals. A recent editorial comments that the “information systems field is heavily
overemphasizing research on white hats to the detriment of studies on black hats” (Mahmood et al. 2010
pp. 431). The intrusion detection data that we utilize in this research allows us to observe “black hat”
behavior and fill a gap in existing research.
5.2. Implications for the Diffusion of Innovation
The overwhelming majority of innovation research implicitly promotes diffusion of innovation,
encouraging complete and rapid adoption. However, there are many negative innovations that society
would like to discourage. An interesting feature of negative innovations is that there are two distinct
diffusion processes – that of the negative innovation itself and of countermeasures to prevent its diffusion.
The disclosure of information about the negative innovation engenders an inherent conflict – defenders
need the information to install countermeasures, but attackers can also exploit the information to do harm.
While our empirical context is focused on information security, we make three important
contributions to the diffusion of innovation literature. First, we focus on the diffusion of a negative
innovation, filling a gap in the literature that has focused almost exclusively on the diffusion of positive
innovations (Bessen 2005; Enkel et al. 2009; Kultti et al. 2006; Owen-Smith and Powell 2004; Rogers
2003). Second, we explore through models and empirical analysis, the implications of two opposing
diffusion processes that result from the disclosure of a negative innovation, an interesting phenomenon
that has not been examined in the literature. The two opposing diffusion processes capture the race
between attackers and defenders, and can provide insights on how best to benefit those who seek to
defend societal priorities. Third, we examine the effect of alternative methods of information disclosure,
an analysis that provides insight on how to best manage information about a negative innovation.
Debates on information disclosure abound in many different innovation diffusion contexts. Should
scientists who discover a dangerous strain of a virus publicly disclose the information so that others can
develop protective measures, or should they withhold information lest it be misused by those who seek to
cause harm? Should a company disclose information about its performance, strategy and weaknesses so
that employees are more effective decision-makers, or will the information be misused by competitors to
the detriment of the company? Is it better to disclose the flaws in the master key locking system so that
consumers are better informed, or will that information cause a greater number of break-ins as more
criminals become aware of the flaw? Our analysis points to two fundamental ways that limited rather than
full disclosure of information may be beneficial. First, full disclosure accelerates the race between those
who protect and those who seek to cause harm. While this may accelerate the development of protective
measures, it also accelerates the use of information for harmful purposes and causes such activity to occur
early in the lifecycle. This affects the weaker and less sophisticated segments of a population who are less
likely to adopt protective measures expeditiously. For example, disclosure about a potentially dangerous
virus may expedite the development of a vaccine, but bio-terrorism based on the virus may
disproportionately affect the poor who may be less likely to be vaccinated in time. Second, full disclosure
and the ensuing race increase the burden on those who protect, making them less effective by straining the
resources available to them. For example, full disclosure of several dangerous viruses concurrently will
burden the few public health officials in the country because they have to divide their effort among all
potential threats. At the same time, a potentially larger number of criminals can strategically choose a few
viruses to exploit. In a race, flexibility provides advantage, and criminals may have greater flexibility in
strategically choosing where to focus, while defenders have to provide some level of protection against all
possible avenues of attack.
However, we believe that there are four conditions in the information security environment that
exacerbate the effects of full disclosure and illustrate the boundary conditions of our findings. First,
vulnerability information can be exploited rapidly by attackers and disseminated through the Internet as
illustrated by the zero-day attacks common in the environment. In other contexts (such as those with long
production times), the time required to exploit may dwarf the speed of information dissemination and
offset the effects of full disclosure. Second, there are few barriers to entry (such as specialized resource or
infrastructure requirements), enabling many potential attackers to develop exploits. In other contexts,
other complementary assets may be required to take advantage of the information disclosed. Third,
vulnerability information is often causally unambiguous, precise and definite so that attackers are able to
develop exploits given vulnerability details. In other contexts (such as corporate reporting), there may be
many confounding effects that reduce the ability to evaluate deterministic causality. Fourth, with a
multitude of software running on corporate and personal systems, there are numerous vulnerabilities
reported regularly, a situation without parallel in other contexts. In summary, speed of potential
exploitation, low barriers to entry in the development of exploits, the lack of ambiguity in exploiting
vulnerability information, and the numerous vulnerabilities in software regularly discovered and reported
contribute to the disadvantages of full disclosure in the information security environment and define the
boundary conditions of our findings. Our results can best be extended to those environments that exhibit
similar characteristics.
5.3. Limitations
There are several limitations of this research. First, while we evaluate the effect of full disclosure on
attack delay, risk of first attack and attack volume, we do not evaluate its effect on vendor behavior and
software quality (Arora et al. 2010). Proponents of full disclosure argue that it exerts pressure on software
vendors to produce better quality software with fewer vulnerabilities, an important aspect of the debate
that we did not evaluate. Second, a key concern in security is the damage possible from a single
vulnerability; limited disclosure may increase the value of vulnerabilities that remain hidden and are not
yet exploited by expert attackers, an issue that we did not examine. Further, while the intrusion detection
system data allows us to observe “black hat” behavior, it is inherently noisy with a high percentage of
false positives. Also, it is important to point out that full disclosure is possible through several other
public forums (such as websites, blogs, social media etc.) that are not reported in the NVD disclosure
history that we utilize. Also, vendors may disclose a vulnerability simultaneously on multiple forums
(including public forums such as Bugtraq) when they have a patch available. Thus, there is opportunity
for mis-classification of vulnerabilities. This introduces noise in the analysis and our results would be
stronger if we could perfectly classify vulnerabilities or eliminate the false positives in the IDS data
(benign activity mis-identified as attacks). Our analysis may also be affected by the endogeneity in the
choice of the disclosure mechanism. That is, security professionals may choose full disclosure if they see
that the vulnerability is already being exploited at the time of discovery. However, since it is unlikely that
security professionals who discover vulnerabilities have access to current attack data from a large number
of companies, this concern is mitigated. Also, there was no evidence in the security provider logs that the
vulnerabilities in our data set were being exploited prior to disclosure. Nonetheless, some concerns
around endogeneity in disclosure choice remains in the analysis.
In spite of these limitations, the analysis of IDS data can be used to validate the results from
analytical models that examine policy related questions (Arora et al. 2008; August and Tunca 2008;
Cavusoglu et al. 2008; Kannan and Telang 2005). The ensuing tension between analytical models and
empirical research will lead to better theory and practical insights. It is important to note that while we
observe a negative consequence of full disclosure, the total consequences of disclosure mechanisms are
also important to evaluate. Concluding that limited disclosure is better than full disclosure based entirely
on this analysis would not be correct. Instead, the community of defenders must be cognizant of the
inherent tradeoffs in disclosure policies to make informed decisions about vulnerability management.
6. REFERENCES
Anderson, R., and Moore, T. 2006. "The Economics of Information Security," Science (314:5799), pp
610-613.
Anonymous. 2010. "Windows Flaw Disclosure Causes Fierce Debate," in: Network Security. Netherlands,
Kidlington: Elsevier BV, pp. 2-2.
Ante, S.E.2010."At&T Discloses Breach of Ipad Owner Data," Wall Street Journal, June 10, 2010, p.
Arora, A., Krishnan, R., Telang, R., and Yang, Y. 2010. "An Empirical Analysis of Software Vendors'
Patch Release Behavior: Impact of Vulnerability Disclosure," Information Systems Research
(21:1), pp 115-132.
Arora, A., Telang, R., and Hao, X. 2008. "Optimal Policy for Software Vulnerability Disclosure,"
Management Science (54:4), pp 642-656.
August, T., and Tunca, T.I. 2008. "Let the Pirates Patch? An Economic Analysis of Software Security
Patch Restrictions," Information Systems Research (19:1), pp 48-70.
Baker, S., and Mezzetti, C. 2005. "Disclosure as a Strategy in the Patent Race," Journal of Law and
Economics (XLVIII), pp 173-194.
Bass, F.M. 1969. "A New Product Growth for Model Consumer Durables," Management Science (15:5),
pp 215-227.
Bélanger, F., and Crossler, R.E. 2011. "Privacy in the Digital Age: A Review of Information Privacy
Research in Information Systems," MIS Quarterly (35:4), pp 1017-1042.
Bessen, J. 2005. "Patents and the Diffusion of Technical Information," Economics Letters (86:1), pp 121128.
Blackwell, M., Iacus, S.M., King, G., and Porro, G. 2009. "Cem: Coarsened Exact Matching in Stata,"
Stata Journal (9:4), pp 524-546.
Boyd, D. 2011. "Protecting Sensitive Information: The Virtue of Self-Restraint," Homeland Security
Affairs (7), pp 1-24.
Cavusoglu, H., Cavusoglu, H., and Raghunathan, S. 2007. "Efficiency of Vulnerability Disclosure
Mechanisms to Disseminate Vulnerability Knowledge," IEEE Transactions on Software
Engineering (33:3), pp 171-185.
Cavusoglu, H., Cavusoglu, H., and Zhang, J. 2008. "Security Patch Management: Share the Burden or
Share the Damage?," Management Science (54:4), pp 657-670.
Cavusoglu, H., Mishra, B., and Raghunathan, S. 2004. "The Impact of Internet Security Breach
Announcements on Market Value of Breached Firms and Internet Security Developers,"
International Journal of Electronic Commerce (9:1), pp 69-104.
Cavusoglu, H., Mishra, B., and Raghunathan, S. 2005. "The Value of Intrusion Detection Systems in
Information Technology Security Architecture," Information Systems Research (16:1), pp 28-46.
Cooper, R. 2001. "A Call for Responsible Disclosure in Internet Security," Network World (18:33), pp 3737.
Cooper, R.B., and Zmud, R.W. 1990. "Information Technology Implementation Research: A
Technological Diffusion Approach," Management Science (36:2), pp 123-139.
D'Arcy, J., Hovav, A., and Galletta, D. 2009. "User Awareness of Security Countermeasures and Its
Impact on Information Systems Misuse: A Deterrence Approach " Information Systems Research
(20:1),
Dasgupta, P., and Stiglitz, J. 1980. "Uncertainty, Industrial Structure, and the Speed of R&D," The Bell
Journal of Economics (11:1), pp 1-28.
Enkel, E., Gassmann, O., and Chesbrough, H. 2009. "Open R&D and Open Innovation: Exploring the
Phenomenon," R&D Management (39:4), pp 311-316.
Enserink, M. 2011. "Controversial Studies Give a Deadly Flu Virus Wings," Science (334:6060), pp
1192-1193.
Fichman, R.G., and Kemerer, C.F. 1999. "The Illusory Diffusion of Innovation: An Examination of
Assimilation Gaps," Information Systems Research (10:3), pp 255-275.
Frei, S., May, M., Fiedler, U., and Plattner, B. 2006. "Large-Scale Vulnerability Analysis," Proceedings
of the 2006 SIGCOMM workshop on Large-scale attack defense, Pisa, Italy: ACM, pp. 131-138.
Fudenberg, D., Gilbert, R., Stiglitz, J., and Tirole, J. 1983. "Preemption, Leapfrogging and Competition in
Patent Races," European Economic Review (22:1), pp 3-31.
Goldenberg, J., Han, S., Lemann, D.R., and Hong, J.W. 2009. "The Role of Hubs in the Adoption
Process," Journal of Marketing (73), pp 1-13.
Greve, H.R. 2011. "Fast and Expensive: The Diffusion of a Disappointing Innovation," Strategic
Management Journal (32:9), pp 949-968.
Ho, T.-H., Savin, S., and Terwiesch, C. 2002. "Managing Demand and Sales Dynamics in New Product
Diffusion under Supply Constraint," Management Science (48:2), pp 187-206.
Johansson, J.K. 1979. "Advertising and the S-Curve: A New Approach," Journal of Marketing Research
(16:3), pp 346-354.
Kannan, K., and Telang, R. 2005. "Market for Software Vulnerabilities? Think Again," Management
Science (51:5), pp 726-740.
Kultti, K., Takalo, T., and Toikka, J. 2006. "Simultaneous Model of Innovation, Secrecy, and Patent
Policy," The American Economic Review (96:2), pp 82-86.
Lemos, R. 2011. "More Vendors Reacting Poorly to Disclosure," in: Security Dark Reading. Information
Week.
Loch, C.H., and Huberman, B.A. 1999. "A Punctuated-Equilibrium Model of Technology Diffusion,"
Management Science (45:2), pp 160-177.
Mahmood, M.A., Siponen, M., Straub, D., Rao, H.R., and Raghu, T.S. 2010. "Moving toward Black Hat
Research in Information Systems Security: An Editorial Introduction " MIS Quarterly (34:3), pp
431-433.
Majchrzak, A., Rice, R.E., Malhotra, A., King, N., and Ba, S. 2000. "Technology Adaption: The Case of a
Computer-Supported Inter-Organizational Virtual Team," MIS Quarterly (24:4), pp 569-600.
Matutes, C., Regibeau, P., and Rockett, K. 1996. "Optimal Patent Design and the Diffusion of
Innovations," The RAND Journal of Economics (27:1), pp 60-83.
Mell, P., Scarfone, K., and Romanosky, S. 2006. "Common Vulnerability Scoring System," IEEE
Security & Privacy (4:6), pp 85-89.
Mell, P., Scarfone, K., and Romanosky, S. 2007. "A Complete Guide to the Common Vulnerability
Scoring System Version 2.0."
Messmer, E. 2007. "Debating Security Flaw Disclosures," in: Network World. United States,
Southborough: Network World Inc., pp. 1-1, 18.
Mookerjee, V., Mookerjee, R., Bensoussan, A., and Yue, W.T. 2011. "When Hackers Talk: Managing
Information Security under Variable Attack Rates and Knowledge Dissemination," Information
Systems Research (22:3), pp 606-623.
Nilakanta, S., and Scamell, R.W. 1990. "The Effect of Information Sources and Communication Channels
on the Diffusion of Innovation in a Data Base Development Environment," Management Science
(36:1), pp 24-40.
NVD. 2008. "National Vulnerability Database."
Owen-Smith, J., and Powell, W.W. 2004. "Knowledge Networks as Channels and Conduits: The Effects
of Spillovers in the Boston Biotechnology Community," Organization Science (15:1), pp 5-21.
Parthasarathy, M., and Bhattacherjee, A. 1998. "Understanding Post-Adoption Behavior in the Context of
Online Services," Information Systems Research (9:4), pp 362-379.
Radianti, J., and Gonzalez, J.J. 2007. "Understanding Hidden Information Security Threats: The
Vulnerability Black Market," 40th Annual Hawaii International Conference on System Sciences:
IEEE Computer Society.
Ramstad, E.2011."Executive Learns from Hack," Wall Street Journal, June 21, 2011, p.
Ransbotham, S., and Mitra, S. 2009. "Choice and Chance: A Conceptual Model of Paths to Information
Security Compromise," Information Systems Research (20:1), pp 121-139.
Ransbotham, S., Mitra, S., and Ramsey, J. 2012. "Are Markets for Vulnerabilities Effective?," MIS
Quarterly (36:1), pp 43-64.
Rogers, E.M. 2003. Diffusion of Innovations, (Fifth ed.). New York, NY: The Free Press.
Schultz, E. 2004. "Sarbanes-Oxley: A Huge Boon to Information Security in the Us," Computers &
Security (23:5), pp 353-354.
Swire, P.P. 2004. "A Model for When Disclosure Helps Security: What Is Different About Computer and
Network Security?," Journal on Telecommunications and High Technology Law (2:1), pp 1-38.
Teece, D.J. 1980. "The Diffusion of an Administrative Innovation," Management Science (26:5), pp 464470.
Tudor, A.2011."Citigroup Confirms Data Breach," Wall Street Journal, June 9, 2011, p.
Van den Bulte, C., and Joshi, Y.V. 2007. "New Product Diffusion with Influentials and Imitators,"
Marketing Science (26:3), pp 400-421.
Van den Bulte, C., and Stremersch, S. 2004. "Social Contagion and Income Heterogeneity in New
Product Diffusion: A Meta-Analytic Test," Marketing Science (41:4), pp 530-544.
Willison, R., and Warkentin, M. 2013. "Beyond Deterrence: An Expanded View of Employee Computer
Abuse " MIS Quarterly (37:1), pp 1-20.
Pr(Ta<Tp) 1.0 dl
A d=0 B d
d=dl Figure 1: Probability of Compromise as a Function of Delay (d)
Cumulative Percentage of Firms
Affected
12.0 10.0 8.0 6.0 4.0 Full Disclosure
2.0 Limited Disclosure
0.0 0 2 4 6 8 Elapsed Days Since Publication
Figure 2: Diffusion Graph of Full and Limited Disclosed Vulnerabilities
10 Table 1: Descriptive Statistics for Focal and Control Variables
Variable
Impact: Confidentiality
Impact: Integrity
Impact: Availability
Defect: Input
Defect: Design
Defect: Exception
Complexity: Medium
Complexity: High
Market
Server
Signature
Patch
Alternatives
Workload
Full Disclosure
All Vulnerabilities
Min
Max
Mean
0
1
0.769
0
1
0.784
0
1
0.831
0
1
0.325
0
1
0.156
0
1
0.097
0
1
0.381
0
1
0.106
0
1
0.133
0
1
0.031
0
1
0.132
0
1
0.547
210
511
345
1
115
39
0
1
0.32
St. Dev.
0.422
0.412
0.375
0.468
0.363
0.296
0.486
0.308
0.34
0.173
0.339
0.498
78
18
0.47
Full Disclosed Vulnerabilities
Min
Max
Mean
0
1
0.791
0
1
0.827
0
1
0.767
0
1
0.357
0
1
0.134
0
1
0.083
0
1
0.318
0
1
0.129
0
1
0.168
0
1
0.044
0
1
0.121
0
1
0.566
211
511
333
1
88
38
1
1
1
St. Dev.
0.407
0.379
0.423
0.48
0.341
0.276
0.466
0.336
0.374
0.205
0.327
0.496
78
17
0
Limited Disclosed Vulnerabilities
Min
Max
Mean
St. Dev.
0
1
0.758
0.429
0
1
0.763
0.426
0
1
0.861
0.346
0
1
0.31
0.463
0
1
0.166
0.372
0
1
0.103
0.304
0
1
0.41
0.492
0
1
0.095
0.293
0
1
0.117
0.321
0
1
0.025
0.155
0
1
0.138
0.345
0
1
0.538
0.499
210
511
351
77
1
115
39
19
0
0
0
0
Table 2: Correlation between Model Variables
1
2
3
4
5
6
7
8
9
10
1. Impact: Confidentiality
1.00
2. Impact: Integrity
0.51
1.00
3. Impact: Availability
4. Defect: Input
0.21
0.21
1.00
-0.10
0.01
-0.18
1.00
5. Defect: Design
-0.05
-0.15
-0.16
-0.19
1.00
6. Defect: Exception
-0.26
-0.27
0.08
-0.12
-0.07
1.00
7. Complexity: Medium
-0.05
0.14
-0.05
0.06
0.02
-0.01
1.00
8. Complexity: High
0.12
0.10
0.00
0.06
-0.05
-0.01
-0.27
1.00
9. Market
0.09
0.10
0.09
-0.09
-0.09
-0.03
0.05
-0.02
1.00
10 Server
-0.13
-0.12
0.00
-0.03
-0.02
0.07
-0.03
-0.05
0.00
1.00
11.Signature
0.11
0.10
0.10
0.00
-0.04
0.08
0.06
0.13
0.07
-0.06
11
12
13
14
1.00
12.Patch
0.00
0.04
0.04
-0.07
-0.02
0.04
0.03
0.01
0.15
0.03
0.13
1.00
13.Alternatives
-0.10
-0.09
0.06
-0.02
0.07
0.11
0.14
-0.12
0.05
0.05
-0.01
0.04
1.00
14.Workload
-0.01
-0.01
0.04
-0.03
-0.03
0.04
0.07
-0.06
0.04
0.01
0.01
-0.02
0.48
1.00
15
Table 3: Diffusion of Exploitation Attempts through the Target Population
Variable
Pa
Ra
da
Dependent Variable: Na(t)
Constant
58.711***
(2.038)
Confidentiality Impact
Integrity Impact
Availability Impact
Defect: Input
Defect: Design
-32.475***
(0.045)
11.739***
-0.394***
(1.66)
(0.089)
-11.125***
Complexity: High
Market
-0.504***
121.676***
(0.115)
(28.107)
-24.477***
-43.074***
174.273***
0.339***
9.165***
(0.078)
(2.507)
1.567***
27.602***
(0.359)
(6.871)
-0.573***
136.684***
(4.497)
(0.132)
(31.015)
42.092***
-0.090***
20.652***
(1.456)
(0.022)
(4.683)
-57.462***
-3.054*
-19.941***
123.242***
3.686***
(1.04)
N
-156.507***
(1.169)
(2.126)
Full Disclosure
0.776***
51.834***
(0.936)
Signature
91.899***
-21.953
(36.045)
(1.345)
Patch
(31.256)
(0.178)
(1.683)
Server
35.880***
(1.43)
(2.425)
Complexity: Medium
76.100***
(17.587)
-0.191***
(1.526)
(1.714)
Defect: Exception
1.122***
(0.258)
1.151***
278.744***
(0.263)
(63.943)
0.104***
27.297***
(0.024)
(6.349)
0.597***
-140.865***
(0.136)
(32.694)
-1.415***
-141.577***
(0.324)
(32.944)
0.094***
-5.765***
(0.021)
(1.83)
132,768
2
R
29.5%
Results based on 132,768 daily observations of vulnerabilities exploited in at least one of 960 client fir
!!
Non-linear regression of the cumulative number of affected firms Na 𝑡 =
(!!! !!!! ) , with Pa , Ra an
(!!!
as linear functions of the variables shown. Robust standard errors in parenthesis; two-tailed significa
levels * p<0.05, ** p<0.01, *** p<0.001
Table 4: Cox Proportional Hazard Analysis of the Risk of First Attack
Variable
Model 0
Model 1
Model 2
Dependent Variable: Likelihood of first attack
Confidentiality Impact
Integrity Impact
-0.233***
-0.246***
-0.250***
(0.025)
(0.025)
(0.025)
0.216***
(0.027)
Availability Impact
0.476***
(0.024)
Defect: Input
Defect: Design
Defect: Exception
0.327***
Complexity: High
Server
Market
Patch
Alternatives
Workload
Full Disclosure
0.501***
(0.025)
0.314***
0.499***
(0.025)
0.308***
(0.018)
-0.330***
-0.313***
-0.315***
(0.028)
(0.028)
(0.028)
0.160***
0.170***
0.143***
(0.032)
0.181***
0.148***
(0.032)
0.182***
(-0.020)
(0.021)
(0.021)
0.047*
0.044*
0.049*
(0.021)
(0.021)
(0.021)
-0.546***
-0.573***
-0.537***
(0.073)
(0.074)
(0.074)
-1.434***
-1.449***
-1.459***
(0.043)
(0.043)
(0.043)
0.073***
1.062***
0.078***
(0.019)
1.074***
0.074***
(0.019)
1.102***
(0.018)
(0.018)
(0.019)
-0.737***
-0.725***
-0.719***
(0.013)
(0.013)
(0.013)
-0.021
-0.022
-0.102***
(0.013)
(0.013)
(0.018)
0.186***
(0.016)
Full Disclosure*Workload
(0.027)
(0.018)
(0.019)
Signature
(0.027)
0.212***
(0.018)
(0.032)
Complexity: Medium
0.215***
0.253***
(0.018)
0.188***
(0.021)
Log pseudo-likelihood
-108680
-108612
-108564
Proportional hazard model on the risk of a firm experiencing an exploitation attempt using
the focal vulnerability; n = 1,152,406; 1201 vulnerabilities; stratified by firm, 960 firms; robust
standard errors in parentheses; two-tailed significance levels: *p<0.05; **p<0.01; ***
p<0.001
Table 5: Poisson Regression Analysis of Volume of Attacks
Variable
Model 0
Model 1
Model 2
Dependent Variable: Number of alerts
Confidentiality Impact
Integrity Impact
Availability Impact
0.071*
Defect: Design
Defect: Exception
Age (ln) at Event
Complexity: Medium
(0.028)
(0.028)
-0.812***
-0.897***
-0.922***
(0.032)
(0.032)
(0.033)
0.143***
0.705***
Market
Server
Patch
Alternatives
0.792***
(0.039)
0.836***
(0.016)
-1.298***
-1.187***
-1.148***
(0.038)
(0.037)
(0.037)
-0.551***
-0.583***
-0.598***
(0.037)
(0.036)
(0.035)
-0.108***
-0.079***
(0.009)
(0.009)
0.737***
0.451***
0.010
(0.010)
0.429***
(0.030)
(0.031)
-0.091***
-0.090***
(0.020)
(0.021)
(0.022)
-1.439***
-1.475***
-1.645***
(0.049)
(0.050)
(0.055)
0.123***
0.729***
0.874***
0.875***
(0.048)
0.889***
0.707***
(0.049)
0.883***
(0.019)
(0.019)
(0.019)
-0.410***
-0.304***
-0.296***
(0.019)
(0.019)
(0.019)
-0.062***
(0.014)
Workload
(0.037)
0.359***
(0.016)
(0.048)
Signature
0.362***
(0.015)
(0.028)
Complexity: High
0.201***
(0.028)
(0.037)
Defect: Input
0.166***
0.066***
(0.009)
Full Disclosure
0.108***
0.102***
(0.014)
(0.014)
-0.002
0.015
(0.009)
(0.009)
0.496***
(0.017)
Full Disclosure * Age
1.266***
(0.053)
-0.168***
(0.010)
R
2
43.9
44.6
44.8
Log Likelihood
-6747726.2
-6668926.9
-6644824.7
Poisson Regression results conditional on vulnerability exploitation; dependent
variable is a count of the number of exploitation attempts recorded; n=141233.
Robust standard errors in parentheses; two-tailed significance levels: *p<0.05;
**p<0.01; *** p<0.001
Table 6: Matching Vulnerabilities on Time Invariant Indicator Variables
Mean Imbalance
Variable
Before
After
Impact: Confidentiality
0.0327
0.0000
Impact: Integrity
0.0639
0.0000
-0.0938
0.0000
0.0465
0.0000
Defect: Design
-0.0315
0.0000
Defect: Exception
-0.0205
0.0000
Complexity: Medium
Impact: Availability
Defect: Input Validation
-0.0921
0.0000
Complexity: High
0.0341
0.0000
Server
0.0194
0.0000
Market
0.0507
0.0000
Signature
Observations
Multivariate L1 distance
-0.0167
1,152,406
0.3433
0.0000
1,013,211
0.0000
Table 7: Cox Proportional Hazard Analysis of the Risk of First Attack (Matched Sample)
Variable
Model 0
Model 1
Model 2
Dependent Variable: Likelihood of first attack
Confidentiality Impact
matched
matched
matched
Integrity Impact
matched
matched
matched
Availability Impact
matched
matched
matched
Defect: Input
matched
matched
matched
Defect: Design
matched
matched
matched
Defect: Exception
matched
matched
matched
Complexity: Medium
matched
matched
matched
Complexity: High
matched
matched
matched
Server
matched
matched
matched
Market
matched
matched
matched
Signature
matched
matched
matched
Patch
-0.014
-0.019
-0.019
(0.018)
(0.018)
(0.018)
-0.523***
-0.514***
-0.510***
(0.012)
(0.012)
(0.012)
-0.165***
-0.170***
-0.200***
(0.011)
(0.012)
(0.014)
Alternatives
Workload
Full Disclosure
0.081***
(0.017)
Full Disclosure*Workload
0.104***
(0.018)
0.074***
(0.021)
Log pseudo-likelihood
-50180.98
-50169.16
-50162.95
Proportional hazard model on the risk of a firm experiencing an exploitation attempt using
the focal vulnerability; n = 1,013,211; 1056 vulnerabilities; stratified by firm and matched
vulnerability, 960 firms; standard errors in parentheses weighted by number of matches;
two-tailed significance levels: *p<0.05; **p<0.01; *** p<0.001
Table 8: Poisson Regression Analysis of Volume of Attacks (Matched Sample)
Variable
Model 0
Model 1
Model 2
Dependent Variable: Number of alerts
matched
Confidentiality Impact
matched
matched
Integrity Impact
matched
matched
matched
Availability Impact
matched
matched
matched
Defect: Input
matched
matched
matched
Defect: Design
matched
matched
matched
Defect: Exception
matched
matched
matched
Complexity: Medium
matched
matched
matched
Complexity: High
matched
matched
matched
Market
matched
matched
matched
Server
matched
matched
matched
Signature
matched
matched
matched
Age (ln) at Event
-0.205***
-0.206***
-0.093*
(0.029)
(0.028)
(0.043)
Patch
Alternatives
Workload
0.183***
0.136***
0.095***
(0.031)
(0.028)
(0.028)
-0.186***
-0.167***
-0.175***
(0.028)
(0.030)
(0.030)
-0.009
-0.032
-0.009
(0.015)
(0.018)
(0.017)
Full Disclosure
0.123***
(0.033)
Full Disclosure * Age
0.987***
(0.189)
-0.159***
(0.032)
R
2
52.6
52.6
52.7
Log Likelihood
-2665146.4
-2670885.5
-2673017.7
Poisson Regression results conditional on vulnerability exploitation; 91,469
observations; dependent variable is a count of the number of exploitation
attempts recorded. Fixed effects for targeted firm, event month, and matched
vulnerability. Standard errors in parentheses weighted by number of matches;
two-tailed significance levels: *p<0.05; **p<0.01; *** p<0.001
APPENDIX
Cumulative Number of Target Systems Attacked and Protected
In this section, we derive equations (1) and (2) in the text. Let Fa(t) be the cumulative fraction of
target systems that have been attacked at time t. Let fa(t) be the instantaneous change in the fraction of
target systems attacked at time t. That is, 𝑓! 𝑡 =
!"(!)
!"
. We calculate the likelihood, L(t), that a target
system is attacked at time t, given that it has not been attacked until that time (see Bass (1969)).
𝐿 𝑡 =
𝑓(𝑡)
1 − 𝐹(𝑡)
How does this likelihood evolve over time? Initially, when there are few systems that have been
attacked (i.e. F(t) ≅ 0), only a few expert attackers have the awareness and know-how to exploit the
vulnerability, and the likelihood of attack is also low. As F(t) increases over time, it indicates that there
are more attackers who are able to exploit the vulnerability, there is more word-of-mouth and sharing of
exploit tools within the attacker community, and the likelihood of attack (given that the target has not
been attacked until that time) increases. Further, for certain type of attacks that spread through
compromised systems, as F(t) increases, the number of compromised systems increases, and L(t)
increases. Thus, we envision that L(t) is proportional to F(t), and we obtain the following differential
equation where 𝑅! is a constant.
𝐿 𝑡 =
!"(!)
!"
!!!(!)
= 𝑅! ∗ 𝐹(𝑡)
(A1)
The formulation above is identical to the Bass Model (Bass 1969) with the coefficient of innovation
set to 0, and it assumes that attacks spread primarily through imitation among attackers. The solution to
(A1) is given by the following (where K is a constant):
𝐹 𝑡 =
!
(A2)
!!!" !!! !
Note that in (A2), there is no value of K such that F(0) = 0 (in other words, a few initial attackers are
needed to seed the diffusion). Thus, we let F(t) asymptotically approach 0 as t approaches - ∞ (In the
empirical analysis, we apply a delay term that shifts the curve to the right such that F(0) is reasonably
close to but not equal to zero.) We define the time horizon as [-∞ to +∞] and apply the following
!
boundary conditions: F( 𝑇!! ) = 0.5 and F(+∞)=1. We obtain K = 𝑒 !! !! . Thus, from (A2),
𝐹 𝑡 =
!
(A3)
!
!!! !!! (!!!! )
𝑁! 𝑡 = 𝑁 ∗ 𝐹 𝑡 =
!
!
!!! !!! (!!!! )
(A4)
In (A4), 𝑇!! denotes the time when half of the target systems have been attacked. (A4) is identical to
Equation (1) in the text. Equation (2) in the text can be derived in exactly the same way. Later, we also
provide an alternative formulation for 𝑁! 𝑡 .
Probability of Successful Compromise
In this section, we derive equation (4) in the text. Let 𝑇! be the time at which the target firm is
attacked through the exploitation of the focal vulnerability, and let 𝑇! be the time at which the target firm
is protected from the focal vulnerability through the installation of countermeasures. Let 𝐺! 𝑥 and 𝐺! 𝑥
be the cumulative distribution functions of 𝑇! and 𝑇! , respectively, and 𝑔! 𝑥
and 𝑔! 𝑥
be the
corresponding probability distribution functions. At time 𝑡 = 𝑥, the number of firms attacked is 𝑁! 𝑥
from a target set of N firms (see equation (4) in the text). Thus, for a random firm, Pr 𝑇! > 𝑥 =
and Pr 𝑇! ≤ 𝑥 =
!! (!)
!
. Likewise, Pr 𝑇! ≤ 𝑥 =
𝐺! 𝑥 = Pr 𝑇! ≤ 𝑥 =
!
!!!
𝐺! 𝑥 = Pr 𝑇! ≤ 𝑥 =
!!! !!!!
! !!
!
!!!
!!! !!!!
!
!! (!)
!
!!!! (!)
!
. Thus,
and 𝑔! 𝑥 =
and 𝑔! 𝑥 =
!!! (!)
!"
!!! (!)
!"
= = !! !
(!!!
!!! !!!!
! !! )!
!! !
(!!!
!!! !!!!
! !!
!!! !!!!
!
!!! !!!!
!
)!
(A5) (A6) For simplicity, we assume that countermeasures and attacks follow the same underlying diffusion rates, as
explained in the paper. That is, we assume that 𝑅! = 𝑅! = 𝑅 and 𝑇!! = 𝑇!! = 𝑇 ! . This allows us to focus
on the effect of delay d, derive a closed form expression for the probability of successful compromise in
terms of d, and explain the intuition behind the hypotheses. The probability of a successful compromise
of any random firm is the probability that the firm is attacked prior to countermeasures being installed
(𝑖. 𝑒. 𝑇! ≤ 𝑇! ). Equation (A7) below is the same as (4) in the paper.
Pr 𝑇! ≤ 𝑇! =
!
Pr
!!
𝑇! ≤ 𝑥 𝑔! 𝑥 𝑑𝑥 =
!
𝐺
!! !
𝑥 𝑔! 𝑥 𝑑𝑥 =
!!(!!!)! !
(! ! !!)!
(A7) Nash Equilibrium of Effort by Early Adopters
Given a single vulnerability disclosed through full disclosure and a single vulnerability disclosed
through limited disclosure, this section derives the Nash equilibrium that describes how early adopters
(expert attackers and security professionals) divide their effort between two such vulnerabilities. Let
𝑑!! and 𝑑!! be the amount by which the expert attacker speeds the diffusion of the full-disclosed
(superscript F) and the limited-disclosed (superscript L) vulnerability, respectively. The expert attacker
achieves this acceleration through additional effort. Likewise, Let 𝑑!! and 𝑑!! be the amount by which
security professionals accelerate the diffusion of countermeasures for full-disclosed (superscript F) and
limited-disclosed (superscript L) vulnerability, respectively. Since there are limited number of expert
attackers and security professionals, they have limited capacity (denoted by 𝐷! and 𝐷! , respectively), and
it follows that 𝑑!! + 𝑑!! = 𝐷! and 𝑑!! + 𝑑!! = 𝐷! . The following Lemma describes the Nash equilibrium
level of effort by the expert attacker and security professional.
Lemma 1: If (a) the profit (loss) from successful compromise is the same for fulldisclosed and limited-disclosed vulnerabilities for the attacker (security professional), and
(b) 𝑑! > 𝐷! and 𝑑! > 𝐷! , then the following is a pure-strategy Nash equilibrium: 𝑑!! = 𝐷! ,
𝑑!! = 𝐷! , 𝑑!! = 0, and 𝑑!! = 0.
Proof: Consider Figure 2 below which is based on Figure 1 in the paper. First consider that the
expert attacker has chosen 𝑑!! = 0 and 𝑑!! = 𝐷! . Thus, limited disclosed vulnerabilities remain at
point B, while full-disclosed vulnerabilities move to point D. Given this choice by the expert
attacker, the security professional’s best response is to choose 𝑑!! = 0 and 𝑑!! = 𝐷! since the
slope of the curve at point D is greater than the slope of the curve at point B (since 𝑑! > 𝐷! ). Next
consider that the security professional has chosen 𝑑!! = 0 and 𝑑!! = 𝐷! . Thus, limited disclosed
vulnerabilities remain at point B, while full disclosed vulnerabilities move to point E. Given this
choice by the security professional, the expert attacker’s best response is to choose 𝑑!! = 0 and
𝑑!! = 𝐷! since the slope of the curve at point E is greater than the slope of the curve at point B
(since 𝑑! > 𝐷! ). ∎
Pr(Ta<Tp) 1.0 dl
Dp
Da Dp Da D A C E d=0 B d
d=dl Figure A1: Derivation of the Nash Equilibrium Level of Effort
Alternative Formulation of the Diffusion of Countermeasures
In this section, we explore an alternative scenario where all firms adopt countermeasures almost
concurrently after a specific delay. It can be argued that in the modern security environment, there may be
less heterogeneity in the adoption of countermeasures, especially among larger and medium sized
enterprises. For example, vendors may download patches to all its subscribers within a short period of
time, managed security service providers can install similar protection across its client base at
approximately the same time, and security professionals in target companies usually try to expeditiously
install countermeasures as soon as possible without severely disrupting operations. To model this
scenario, we envision that all target firms adopt countermeasures almost concurrently after a small delay
(m) after public disclosure. Consequently, a target firm is compromised if it is attacked prior to time t =
m. Thus, the probability of successful compromise is:
Pr 𝑇! < 𝑇! = Pr 𝑇! < 𝑚 = 𝐺! 𝑚 =
!
!!!
!!! !!!!
! !!
=
!
where D = 𝑅! 𝑇!! − 𝑚 + 𝑑
!!! !
(A8)
The figure below plots Pr(Ta< Tp) as a function of D. It is reasonble to assume that 𝑇!! ≫ 𝑚 (that is,
countermeasures are adopted much prior to the half-life of the attack diffusion process. So, we will
concern ourselves only with the portion of the graph to the right of the vertical axis (D>0).
Pr(Ta<Tp) 1.0 dl
D = 𝑅! 𝑇!! − 𝑚 + 𝑑
D=0 A
B
A
d
Figure A2: Probability of Compromise with Alternative Diffusion Model
In the figure above, point A corresponds to a full-disclosed vulnerability with d=0 and D=𝑅! 𝑇!! −
𝑚 , and point B corresponds to a limited disclosed vulnerability with d= dl and 𝐷 = 𝑅! 𝑇!! − 𝑚 + 𝑑! . It
can be shown through differentiation that the slope of the curve at point A is steeper (more negative) than
at point B. Thus, using exactly the same logic as in Lemma 1, the Nash equilibrium is such that both
security professionals and attackers focus all their effort on the full-disclosed vulnerability (A).