The Ever-Expanding 10-K:
Why Are 10-Ks Getting So Much Longer (and Does It Matter)?
Travis Dyer†
Mark Lang†*
Lorien Stice-Lawrence†
April 2016
Revision in Process, Comments Welcome
We document striking increases in length, complexity, and redundancy in 10-K disclosure over
the period 1996-2013. Results are consistent after controlling for sample composition, economic
determinants, litigation risk, SEC oversight, and other potential factors. All of the major sections
of the 10-K have increased in length, suggesting that the increase is not directly the result of
economic factors. Using Latent Dirichlet Allocation (LDA), we examine specific topics that
firms discuss and find that the bulk of the increase in length is due to new requirements from the
FASB and SEC, and that 3 of the 150 topics we identify account for virtually all of the increase
in disclosure length. These topics relate to fair value, internal controls, and risk factor
disclosures, all of which were subject to mandatory changes during our period. We document
that increases in these three disclosures also play a major role in explaining upward trends in Fog
and redundancy. Of the three, risk factor disclosures are most strongly associated with the shortterm market response and future outcomes. Our results are relevant to concerns from regulators
about increasingly onerous accounting disclosures and suggest that the increases in length,
complexity, and redundancy are driven by regulation itself.
† The University of North Carolina at Chapel Hill * Corresponding author: Mark Lang, KenanFlagler Business School, the University of North Carolina, Chapel Hill, NC 27516-3490,
Mark_Lang@UNC. We thank Tim Loughran and Bill McDonald for use of their business words
dictionary to compute our tone measures. We thank workshop participants at University of
Bristol, Cornell University, University of Exeter, Georgetown University, Indiana University,
University of North Carolina, and University of Utah for helpful comments and suggestions. The
Internet Appendix for this paper can be accessed at http://tinyurl.com/z369b6x.
I.
Introduction
There is a general concern among investors, preparers, regulators, and standard setters
that corporate disclosure has become longer, more redundant, more complex, and more onerous
over time (Li, 2008; KPMG, 2011; SEC, 2013). However, the magnitude, economic
determinants, specific content, and attributes of the additional disclosure have received less
attention in the academic literature. In December 2013, the SEC began a comprehensive review
of current disclosure regulation with the intent of identifying the extent of excessive, unduly
complex, and redundant disclosure (SEC, 2013). Similarly, the FASB has an ongoing agenda
project, the Disclosure Framework, evaluating the effectiveness of textual disclosure (FASB,
2012). A variety of explanations have been offered for the apparent increase in the quantity and
complexity of disclosure including the effects of litigation, increases in business complexity,
globalization, financial instruments, regulation, and new mandatory disclosure (KPMG, 2011;
SEC, 2013; Monga and Chasan, 2015). In conjunction with their reviews, the SEC and FASB
have encouraged academics and others to provide insight into the causes and consequences of the
increased length, complexity, and redundancy of disclosure.
Our objective in this study is to investigate several related questions. First, to what extent
has there been a systematic increase in the quantity of 10-K disclosure over time? Second, to
what extent can that increase be explained by economic factors such as changes in the
composition of publicly-traded firms, economic complexity, litigation exposure, SEC oversight,
and other characteristics of firms and their operating environments? Third, for changes in
disclosure that cannot be explained by economic factors, to what extent is the incremental
disclosure explained by increased disclosure requirements and what is the content of the
increased disclosure? Fourth, to what extent does the additional disclosure add complexity
1
and/or redundancy to the 10-K? Fifth, does it appear that the disclosure is informative and
timely to market participants?
Our sample includes the text of 10-Ks for 10,452 firms and 75,991 firm-years over the
period 1996 to 2013. In terms of our first question, we document a striking increase in the length
of annual report text over the sample period with the median text length more than doubling from
a little more than 23,000 words in 1996 to over 49,000 in 2013. While there is evidence of a
sharper increase following Sarbanes-Oxley, the increase in median length has been nearly
monotonic over our 18-year sample period. This increase in length has been pervasive, with the
length of every section of the 10-K increasing substantially. There is also evidence of concurrent
increases in the redundancy and complexity of disclosure.
To assess the extent to which our results reflect changes in economic attributes of the
sample firms over time, we examine length after controlling for company-level variables,
redundancy, and for a constant sample of firms, but similar patterns persist. We examine
potential demand-driven determinants such as litigation risk, institutional ownership, and SEC
comment letters for evidence on whether changes in the legal environment, ownership structure,
or oversight appear to explain the changes. While those variables are significant cross-sectional
determinants of length, changes in those variables do not appear to explain the additional
disclosure over time.
Because economic determinants do not appear to explain changing textual characteristics,
we more directly examine the content of the additional disclosure and its effect on complexity
and redundancy using Latent Dirichlet Allocation (LDA). LDA is a Bayesian computational
linguistic technique that identifies the latent topics in a corpus of documents. It is then possible
to estimate, for example, the proportion of the text in a document that relates to a specific topic
2
(e.g., fair value accounting) and to track changes in the quantity of disclosure over time (e.g.,
around SFAS 157 adoption). This approach has particular relevance in our setting because
discussion of a topic is frequently spread throughout the sections of the 10-K (e.g., in
Management’s Discussion and Analysis, Risk Factors, and the Notes to the Financial
Statements). LDA permits us to aggregate discussion of a given topic across sections, evaluate
trends in the disclosure over time, and assess the extent to which increases in the topic explain
increases in characteristics such as complexity and redundancy, as well as the relation between
the additional length and the informativeness of the resulting disclosure.1
In our LDA analysis we evaluate textual disclosure across 150 subtopics, which we
aggregate into 13 broader topic categories. The five broad topic categories which account for the
bulk of 10-K length are Performance, Compliance, Industry-Specific disclosure, and EmployeeRelated disclosure. Turning to trends in the broader topic categories, the vast majority of the
increased disclosure over time relates to compliance with new accounting and disclosure
standards. Within the Compliance category, the three topics that explain the bulk of the increase
in text over our sample period are fair value and impairment disclosure associated with SFAS
157, discussion of internal control weaknesses associated with Sarbanes-Oxley, and forwardlooking disclosure associated with risk factors which was made mandatory as part of Item 1A.
We document stark increases in these three topics in the years in which their associated standards
were implemented, consistent with the LDA topics effectively capturing disclosure in response
to these standards and validating our description of the content of these topics. Disclosure
An example used to illustrate LDA is Chen (2011)’s analysis of Sarah Palin’s emails. Although there is a large
number of emails, LDA is able to accurately categorize them by topic including: family, presidential campaign,
wildlife, education policy, and oil/gas. Trends in topics can then be tracked over time (e.g., email about family
spikes around Trig’s birth, email about the presidential campaign spikes during campaign season, etc.).
1
3
associated with these topics is not limited to a single section of the annual report but extends
across all of the major sections.
In our third set of analyses, we provide evidence on the attributes of the incremental
textual disclosure in the annual report. Because it is difficult to directly assess the usefulness of
10-K disclosure, we take two approaches. The first, which is more circumstantial, is to examine
textual characteristics of the additional disclosure. An advantage of LDA is that we can identify
the topic content of each paragraph in the 10-K and can then measure the associated textual
attributes, in particular Fog and redundancy, of paragraphs relating to individual topics thus
allowing us to identify the extent to which specific topics explain the overall trends we document
in complexity and redundancy. We focus on complexity and redundancy because regulators,
investors, and preparers, as well as prior research such as Loughran and McDonald (2014) and
Cazier and Pfeiffer (2015b), suggest that disclosure that is characterized by high levels of
complexity and redundancy is less useful to investors. We demonstrate that the trends in
complexity and redundancy that we observe for the entire 10-K largely reflect the increase in the
amount of compliance-related disclosure, particularly fair value, internal control, and risk factor
disclosure. Fair value, risk factor, and internal control disclosures tend to be unusually complex
and redundant relative to other disclosure in the 10-K, and increases in the proportion of these
disclosures in the 10-K over time largely explain the overall patterns of Fog and redundancy.
The preceding analysis relies on prior evidence to suggest that disclosure that is
redundant and complex may be less useful. In our final analysis we provide more direct
evidence on the informativeness of these additional disclosures. Admittedly, this is the most
challenging part of the analysis because it is difficult to link specific components of the 10-K to
information content because prior research suggests that the market response to the 10-K release
4
tends to be limited and because it is difficult to separate out the effects of disclosure from the
effects of underlying economics. As a result, we view this analysis as exploratory. We take two
approaches. First, we examine market responses (liquidity, turnover, and returns) around 10-K
filing dates for evidence on whether greater discussion of specific topics is associated with lower
levels of information asymmetry, more trading, and greater stock price movement. Second, we
investigate the ability of risk factor, fair value, and internal control disclosure to predict related
firm fundamentals, in particular firm risk, special items, and material weaknesses. Of the three
topics only risk factor disclosure is consistently correlated with current market outcomes and
able to predict future fundamentals, suggesting that risk factor disclosure is informative to
market participants. On the other hand, disclosures related to internal controls and fair values do
not appear to be consistently correlated with market outcomes or future fundamentals.
The combined results from our textual attribute and informativeness tests suggest that
enhanced regulation and disclosure requirements largely explain increases in the quantity,
complexity, and redundancy of disclosure over time. However, it is less clear that they have
increased the informativeness of disclosure.
Our research makes several contributions to the literature. First, we provide systematic
evidence across a number of dimensions on the time series trends in textual characteristics that
are likely to be of interest to regulators, standard setters, and investors, along with evidence on
the sources of these changes. A variety of commentators have argued that increases in length,
complexity, and redundancy of 10-Ks indicate that disclosure may have become excessive and
less useful, and a variety of potential explanations have been suggested. We provide evidence
that the largest driver of 10-K length over time is increased regulatory requirements, and we are
able to isolate the specific regulations to which the disclosure pertains. While it is difficult to
5
draw normative conclusions based on our analysis, we believe that understanding the drivers of
10-K length, redundancy, and complexity, particularly at the topic level, is important in
interpreting recent trends in financial reporting and assessing the effects on the informativeness
of the resulting disclosure.
Second, we develop and demonstrate the value of techniques such as LDA to understand
the underlying content of textual disclosure. We believe that LDA has the potential to be a
powerful tool for understanding the content of annual reports and other financial disclosures
because it provides researchers with an approach for evaluating topical coverage for large
samples of lengthy documents on a consistent and relatively objective basis. While summary
quantitative measures such as length, redundancy, and Fog are useful in providing aggregate
characterizations of documents, it is increasingly important to develop techniques that permit
insight into the underlying content of disclosure. LDA permits the researcher to identify specific
topics in disclosure, highlight trends, isolate causes, and evaluate potential economic outcomes.
Third, we associate disclosure relating to specific topics with textual characteristics of
disclosure (namely length, complexity, and redundancy) and evaluate those topics based on their
information content. An important challenge to regulators and academics is in assessing the
relative effectiveness of various disclosure requirements. While we cannot draw normative
conclusions on the topics we consider, our results suggest varying degrees of informativeness
across topics. Further, by splitting the text into interpretable topics, LDA provides a mechanism
for assessing the source of constructs such as Fog and redundancy. This is especially important
given that prior literature focuses on these textual outcomes at an aggregate level without
incorporating the fact that discussions of different topics will have different levels of Fog and
6
redundancy. LDA provides the opportunity to reinterpret the existing literature on outcomes of
these attributes factoring in the actual content of the discussion to which they relate.
This research is admittedly subject to important caveats. First, topics from LDA (much
like factors in factor analysis) require interpretation by the researcher. As discussed in the
research design section, we follow the prior computational linguistics literature in identifying the
appropriate number of topics and require agreement between multiple individuals with financial
backgrounds in categorizing topics. Additionally, we identify representative paragraphs for each
topic and measure the topic’s textual attributes such as Fog, redundancy, tone, and uncertainty.
These additional measures provide confidence that our interpretations, appropriately caveated,
are reasonable and consistent with the way the topics behave in our corpus.
Second, our results are largely descriptive, and we cannot draw normative conclusions.
We believe that LDA (along with other textual analysis techniques) has the potential to structure
the broader discussion on topics such as disclosure effectiveness and information overload. We
interpret Fog and redundancy as negative outcomes in light of the prior literature and use market
outcomes and the ability of specific disclosures to predict fundamentals as measures of
informativeness. However, assessing the usefulness of textual disclosure is clearly nuanced and
requires tradeoffs across the interests of various stakeholders.
In the next section, we discuss the related literature. Then we turn to the research design,
sample, results and, finally, conclusions and directions for future research.
II.
Background and Related Research
The issue of lengthy, repetitive, and complex disclosure has been an ongoing concern for
regulators for many years. From the initiation of Regulation S-K in 1982, the goal of integrated
7
disclosure has been to eliminate overlapping and duplicative disclosure (SEC, 2013).2 In 1994,
the chairman of Ernst and Young, Ray Groves, observed that, “In financial disclosure we have
reached a point where more is not better.” (Groves, 1994). In 2001, Chief Executive of Arthur
Andersen, Joe Berardino, wrote, “Like the tax code, our accounting rules and literature have
grown in volume and complexity…” (Berardino, 2001). In an attempt to address these types of
concerns, the SEC has initiated a comprehensive review of S-K and S-X disclosure practices,
which encompasses 10-K, 10-Q, and 8-K filings (SEC, 2013). The last such review of disclosure
requirements was performed in 1996, after which practices such as the Plain English Initiative
were implemented. The purpose of the current initiative is to understand the trajectory of
disclosure practices and make recommendations for future revisions in disclosure policies.
Chairwoman of the SEC Mary Jo White said, “We should consider all sources that may be
contributing to the length and complexity of disclosure” (White, 2013).
Despite the concern on the part of the SEC and others about annual report text, financial
reporting research has traditionally focused on quantitative data, particularly summary statistics
such as net income and shareholders’ equity, reflecting in large part the relative ease of assessing
associations between quantitative data, coupled with an inherent assumption of unlimited
information-processing capacity on the part of investors. More recently, researchers have begun
to assess textual disclosure. Research investigating determinants of textual characteristics
includes Li (2008) linking Fog to poor performance, Cazier and Pfeiffer (2015a) linking length
to complexity, and Cazier and Pfeiffer (2015b) linking redundancy to obfuscation.
Most of the research to date has focused on explaining cross-sectional variation in
textual attributes of the 10-K. However, one of the frequently referenced aspects of textual
2
Regulation S-K is the disclosure regulation for all sections of the 10-K except the financial statements and their
respective notes, which are dictated by regulation S-X.
8
disclosure is the perception of trends in quantity, complexity, and redundancy over time, which
is our focus. While prior studies suggest cross-sectional factors associated with textual
characteristics, our empirical analysis indicates that those factors have limited ability to explain
trends in reporting. Using LDA, we identify the source and content of the incremental disclosure.
Our results highlight the importance of new FASB and SEC requirements, particularly those
related to fair values, internal controls, and risk factors, in explaining disclosure trends over time,
LDA has traditionally been used in other disciplines for tasks such as documenting
historical trends in topics in, for example, newspapers, academic journals, and literature.
However, LDA has only recently been used in accounting and finance. Huang et al. (2014)
employ LDA to examine differences between the topics discussed in conference calls and analyst
reports. Hoberg & Lewis (2015) use LDA to examine changes in the content of a firm’s MD&A
in the years surrounding fraudulent behaviors that result in an AAER. Ball et al. (2014) use
LDA to identify topics within the MD&A and suggest that MD&A disclosure is especially value
relevant when the accounting numbers are less informative. While the prior literature suggests
that LDA has the potential to organize textual disclosure for numerical analysis, it has not, to our
knowledge, been applied to understanding the overall content and trends in 10-K disclosure or to
identify the topical sources of constructs such as Fog or redundancy.
III.
Data
We generate a database of text using SEC 10-K filings spanning the years 1996 to 2013.3
For our determinants and control variables, we use data from Audit Analytics, Thompson 13F,
I/B/E/S, Stanford Class Action Clearinghouse, CRSP, and Compustat. Following Loughran and
3
We include only those filings in 1996 that were issued after electronic filing on EDGAR became mandatory.
9
McDonald (2014) we remove firms with negative market-to-book ratios. The intersection of our
data constraints results in a sample of 10,452 firms and 75,991 firm-years. Specific definitions
for all of our variables are included in the Appendix.
Table 1 provides descriptive sample statistics. The median firm included 37,370 words in
their annual report. Based on the Fog index, reading and comprehending the median annual
report requires approximately 21.25 years of formal education. Approximately 77% of the
sample firms are audited by a Big “N” auditor, and institutional investors own approximately
38% of the median firm.
Figure 1 provides initial evidence on the trends in reporting over our sample period,
including length, redundancy, and Fog. Perhaps most striking and relevant for our purposes is
the increase in length over the sample period depicted in Figure 1 Panel A and the near
monotonicity of this increase. While there is some evidence of larger increases around SarbanesOxley in the early 2000s and the financial crisis, especially for firms in the 75th percentile, the
increase for the median firm has been remarkably continuous. The number of words for the
median firm increased from about 23,000 words in 1996 to almost 50,000 in 2013. The pattern
for redundancy in Figure 1 Panel C is also striking, with the median firm increasing from 800
words in redundant sentences in 1996 to almost 3,300 in 2013.4 Similarly, Fog for the median
firm has increased monotonically over the twelve years since 2000, following a decrease
between 1998 and 2000 that was likely a result of the SEC’s plain English requirements in
4
Our measure of redundancy almost certainly understates the true level of redundancy because we err on the side of
being conservative by requiring verbatim repetition of sentences. Conclusions are consistent if we relax our criteria
by not requiring that all words in a sentence be repeated verbatim.
10
1998.5 The trend since 2000 has been clearly increasing with Fog for the median firm in 2013
exceeding that prior to enactment of the plain English requirements.
The preceding analysis is descriptive, but it provides compelling evidence that the
asserted trends in the quantity, complexity, and redundancy over time are supported in the data
and that our constructs appear to capture trends in textual characteristics over time. Further, the
consistency in trends between length, redundancy, and Fog (post-2000) suggests the possibility
that the same underlying factors may be driving all three series. For parsimony going forward,
we focus our initial analysis on length since that permits us to identify incremental disclosure. In
later analysis we use the characteristics associated with the incremental disclosure (Fog and
redundancy) to assess the extent to which the sources of the additional length may also explain
the concurrent increases in Fog and redundancy.
IV.
Why has Length Increased?
There are several potential explanations for the increase in length. To provide descriptive
evidence on potential causes of the increase in 10-K length over time, Figure 1 Panel B presents
a graph comparing the trend in overall 10-K length when various economic factors are taken into
account. Median length for each measure is plotted over time with each year depicted as a
proportion of the first year’s length so that all of the trends can be interpreted on a consistent
basis based on the proportional change in disclosure over time.
First, the increase in length could reflect a change in sample composition over time. For
example, Srivastava (2014) suggests that trends in value relevance of accounting data can be
5
Fog is defined as the average number of words per sentence plus the percent of words containing more than two
syllables, multiplied by 0.4, and can be interpreted as the number of years of formal education an individual would
need to read and understand a given document.
11
explained by changes in the sample composition of publicly-traded firms. A similar effect could
be at work with 10-K disclosure if, for example, more firms with intangible assets (and
potentially more corresponding disclosure) became publicly traded during the sample period.
Figure 1 Panel B reports trends in disclosure length for a constant sample. The overall trends in
median length for the constant sample are very similar to those for the unadjusted median length
(and, if anything, slightly larger), with the 2013 value of 2.12 indicating that, holding the sample
constant, overall length more than doubled over the sample period.
Second, the increase in length could be a mechanical result of the increase in redundancy.
While Figure 1 Panel C documents that redundancy has been increasing over time, redundancy
(at least in terms of verbatim sentences) represents a relatively small proportion of the total
disclosure. The median non-redundant disclosure plotted in Figure 1 Panel B illustrates that,
excluding redundant disclosure, overall disclosure still increased by more than 100% over the
sample period.6
Third, the increase in length could reflect changes in economic conditions, SEC oversight
or litigation risk over time. Papers such as Li (2008) and Pfeiffer and Cazier (2015a and 2015b)
suggest that firm characteristics affect disclosure attributes in the cross-section. It is possible
that firm characteristics such as business complexity, leverage, size, auditor, or profitability
could have changed systematically over the sample period.
To control for economic variables that might explain the increase in length, we begin by
estimating a first stage regression including variables suggested by prior research as well as by
commentators and regulators. Results are reported in Table 2. In Column 1, we investigate the
link between length and economic characteristics of the firm. For example, disclosure might be
6
As noted earlier, our requirement of a verbatim match likely understates redundancy. Conclusions are consistent
using less stringent definitions of redundant disclosure.
12
increasing in length because the average firm became larger over our sample period or because
of greater oversight in the form of Big “N” auditors or NYSE membership. In addition, firms
might be increasing in complexity in terms of numbers of business segments or operating
segments, or carrying more debt which necessitates additional disclosure. Finally, firms
reporting losses might require more disclosure to mitigate market uncertainty.
As illustrated in Table 2 Column 1, 10-K length is significantly greater for larger firms,
likely reflecting the fact that more disclosure is required for large firms given their range of
economic activity, and for firms audited by Big “N” auditors, perhaps reflecting the greater
oversight by larger audit firms. Length is greater for younger firms, those with higher market-tobook ratios, as well as those with more business segments and foreign segments, suggesting a
greater need to explain results for firms characterized by greater uncertainty and complexity.
Length is also greater for firms reporting losses, potentially reflecting either obfuscation (Li
2008) or the need for more discussion associated with negative news (Bloomfield 2008).
Finally, 10-Ks are longer for firms with greater leverage, potentially reflecting greater demand
for information for levered firms.
Table 2 Column 2 adds variables related to ownership, analyst following, litigation risk,
SEC oversight, and other measures of demand for information. Because of the additional data
requirements, the sample size drops by nearly half. One possibility is that increased disclosure is
a consequence of increased oversight by the SEC. We consider three measures of SEC
oversight—the number of comment letters filed for the firm, the number of comment letters for
peer firms in the industry, and the number of Accounting, Auditing and Enforcement Releases
(AAERs) for peer firms in the industry. We include peer firms because we are interested in
measures that capture the overall level of oversight by the SEC rather than simply the increase in
13
disclosure triggered by SEC intervention with respect to a specific firm. Similarly, we include a
firm-specific measure of litigation risk to reflect the possibility that firms might be responding to
perceived increases in legal exposure by increasing disclosure. We include institutional
ownership and analyst following to capture the possibility that other forms of oversight during
the period might have increased disclosure. We also include unexpected earnings, mergers, and
market-wide returns to reflect the possibility that period-specific events might affect disclosure.
Based on the results in Table 2 Column 2, disclosure tends to be lengthier if there are
more SEC comment letters for the firm or for peer firms, or more AAERs in the industry.
Similarly, disclosure is longer when there are more institutional owners, and in periods of
mergers or economic downturns. The only potentially surprising result is that, conditional on the
other factors, analyst following is negatively associated with length, reflecting collinearity
between analyst following and other variables such as size and institutional ownership (the
unconditional correlation between analyst following and length is positive).
The preceding suggests that disclosure length is influenced by economic factors in
predictable ways. However, our primary interest is in establishing the extent to which those
factors explain the increase in disclosure over time. To capture that effect, we include a Trend
variable in Table 2, which measures the remaining trend in the disclosure length after controlling
for our economic variables as well as industry fixed effects. The coefficient on Trend remains
strikingly significant, suggesting that the bulk of the increase in length over time does not reflect
changing economic circumstances. To graphically illustrate the remaining trend in disclosure,
Figure 1 Panel B plots the residual from the regression of 10-K document length on firm
14
characteristics based on Table 2 Column 1.7 There is still clear evidence of an increase in length
which is not explained by firm, industry, or general economic characteristics. 8
V.
Using LDA to Explain the Change in 10-K Length
The preceding analysis suggests that while there is a clear increase in the quantity of
disclosure over time the increase does not appear to be primarily a result of changes in
redundancy, sample composition, or other economic incentives and characteristics. As a result, a
likely explanation would seem to be changes in required disclosure. However, the source and
content of the increased disclosure is unclear.
To provide an initial sense for whether the increase in disclosure might simply reflect
changes in disclosure requirements that affect a single component of the 10-K, we examine
disclosure trends over time by section. The securities laws specify the required 10-K sections
and the general requirements for content in each section.
Figure 2 plots the median length for sections of the 10-K. Of the eleven sections, three
make up about 90% of the total text in the most recent year: Sections 1 & 2 (Business and
Property Descriptions) account for approximately 35% of the total length, Section 7
(Management’s Discussion and Analysis) accounts for 25%, and Section 8 (Financial Statements
and Footnotes) accounts for 30%. Most noticeable from Figure 2 is the fact that the length of all
of the major sections has increased substantially and at roughly the same rate over time. As a
consequence, the proportion of the 10-K in each section has remained similar over time, with
Sections 1 and 2 comprising 36% of the total in 1996 as compared with 35% in 2013, Section 7
7
We plot residuals from Table 2 Column 1 so that the sample composition is comparable with the other graphs. The
pattern is similar if we use residuals from Table 2 Column 2.
8
Figure 2 Panel B almost certainly understates the remaining trend because several of the control variables tend to
trend as well and it is difficult to disentangle the source of the shared trend.
15
comprising 21% in 1996 vs. 25% in 2013, and Section 8 comprising 30% in both 1996 and 2013.
These results suggest that overall changes in disclosure for the 10-K as a whole do not reflect
requirements that are unique to a specific section but rather reflect content that spans multiple
sections.
Because the sections of the 10-K include a mix of disclosure topics that are likely shared,
it is difficult to infer the source of the added length simply by the length of the sections. In order
to more directly investigate the topics being discussed, we use a textual methodology developed
in the natural language processing literature called LDA. LDA allows us to identify the specific
topics of disclosure contained in each annual report, and trends in their proportions. By
identifying the amount of disclosure related to specific topics, we can investigate the proportion
of the 10-K allocated to each topic as well as trends in the discussion of specific topics over time.
LDA is an unsupervised Bayesian machine-learning approach developed by Blei et al.
(2003) to identify the topics contained in a large corpus of text. LDA uses the probability of
words co-occurring within documents to identify sets of topics and their associated words and is
conceptually similar to factor analysis, where the model produces topics instead of factors.9
Analogous to factor analysis, the computer identifies the words associated with a topic and the
researchers assign a label to the topic based on their assessment of the likely content given the
set of words and their probabilities.
We use the MALLET software developed by Andrew McCallum, available at
http://people.cs.umass.edu/~mccallum/code.html, to apply LDA to our sample and generate
9
The LDA procedure models probabilities as follows. First, it assumes that each document has a unique distribution
of topic proportions. Each word (e.g., Word #13) within the document is assumed to be randomly generated by first
being assigned to a topic (e.g., “Pensions”) based on the document-specific probability of that topic. Then, the
observed word is randomly chosen from the topic based on the probability distribution of words in that topic (e.g.,
“benefit”). We refer readers to the discussion in Huang et al. (2014) for a detailed explanation of the mechanics of
LDA.
16
topics for our document collection. Each topic is a set of relative probabilities for each word,
where MALLET estimates the probability that word X appears in topic Z, for all X and Z, using
collapsed Gibbs sampling. The LDA procedure uses this set of probabilities combined with the
actual words observed in each document to estimate how prominent each topic is in that
document. This is called the topic loading and can be interpreted as the proportion of the
document comprising that topic. 10
Because LDA is an unsupervised method, it is replicable and free of researcher bias.
However, the topics can sometimes be difficult to interpret, and the researcher must choose the
number of topics that the model generates. Following prior literature, we use a variety of
quantitative criteria to ensure that we evaluate the appropriate number of interpretable topics.
First, as originally proposed in Blei et al. (2003), we measure the “perplexity” of the topic model
for a variety of candidate topic numbers. Perplexity (defined more formally in the Appendix) is a
measure of how well the model matches the observed data; a low perplexity score indicates a
better match. We estimate perplexity for topic models with between 10 and 400 topics and
observe that perplexity begins leveling off at 150 topics, meaning that the model performance
gains relatively little from increasing the number (and specificity) of topics after that point.
Although perplexity is a good general guide, and lower perplexity will always lead to
models with at least marginally better fit relative to held-out data, the increase in fit is sometimes
at the expense of interpretability due to overfitting. Chang et al. (2009) discuss how increasing
the number of topics in an LDA model to produce ever finer partitions can make the model less
10
Topic loadings for a single document sum to one. Our model allows the prominence of topics (the alpha
hyperparameter) to vary across the entire corpus of all 10-Ks so that topics that appear in relatively few documents
(e.g., industry-specific topics such as healthcare) are given less prominence while topics that are used in more
documents (e.g., accounting policies) are given more prominence. This essentially means that common topics are
allowed to be “bigger” than others so that they have a consistently higher topic loading on average.
17
useful because it becomes almost impossible for humans to differentiate between many of the
topics. They propose that perplexity be balanced with interpretability as operationalized through
humans’ ability to decipher the topics implied by the model. In particular, they propose a task in
which the interpretability of a specific LDA model is measured by how often a human coder
agrees with the topics chosen using the model. We perform this “word intrusion” task by
providing the human coder with sets of six words, five of which the computer suggests belong in
the same topic and a sixth which is a word which appears commonly in 10-Ks but which the
model did not assign to that topic (an “intruder” word). The extent to which the human coder
disagrees with the computer on the assignment of words to a topic is a measure of the
effectiveness of the technique in capturing meaningful topics. We perform this word intrusion
task for 100, 150, and 200 topics (more details in the Appendix) and find that the 150-topic
model has the best interpretability (i.e., the fewest disagreements between the computer
technique and human coders). As a consequence, in our tests we use LDA topics from the 150topic model.
For parsimony going forward we group each of the LDA topics into one of thirteen
broader categories. To form the broader categories, two individuals with financial backgrounds
(one MBA student with work experience in banking and one of the authors) went through all 150
topic word lists to develop broader categories and then evaluated each of the 150 topics word
lists independently to determine the best fit of each into the broader categories. In most cases,
the two coders agreed on categorization but in cases in which the coders disagreed one of the
remaining authors judged the best fit. Additionally, we also performed a procedure similar to that
described in Hoberg and Lewis (2015) to identify “representative paragraphs” for each topic.
This process, described in detail in the Appendix, allows us to identify paragraphs that are
18
representative of each topic; that is, they have a high loading of that topic and are similar to other
paragraphs which focus on that topic. Unlike the raw word lists generated by the LDA
procedure, representative paragraphs provide coherent content that allows the researchers to
better understand and communicate the content of each topic. The Internet Appendix includes a
list with the representative paragraph for all topics, as well as the top 20 words most frequently
associated with each topic, and the topic label and category created by the researchers. 11 For all
other details relating to the specifics of our LDA procedure, please see the Appendix.
Table 3 lists the broad categories into which we group the topics in our analysis, along
with brief descriptions. For example, “Business Operations and Strategy” refers to discussions of
day-to-day business operations such as products, advertising, and information systems;
“Business Structure and M&A” refers to discussion of subsidiaries and partnerships, as well as
mergers, acquisitions and other corporate transactions; and “Loans, Debt and Banking” refers to
discussion of the firm’s financing. Of the categories, the five that constitute the largest portion
of 10-K text, especially in the early part of the sample period, are: “Performance, Revenues, and
Customers,” which is primarily discussion explaining the performance and revenue generation of
the firm; “Industry Specific Disclosure,” which includes topics that are unique to specific
industries (e.g., healthcare or transportation); “Employees and Executives” which includes
descriptions of executives and executive compensation plans; “Compliance with SEC and
Accounting Standards,” which is text associated with specific reporting requirements; and
“Investments, Securities and Derivatives,” which includes descriptions of financial instruments.
The identification of topics and the grouping into categories allows us to begin to
disaggregate the overall trend in length into the portions attributable to individual types of
11
Category labels are for parsimony and ease of interpretation and do not affect the statistical analysis.
19
disclosure. Because the LDA loading for each topic in a document can be interpreted as the
proportion of the document related to that topic, we can construct a pseudo topic “length” by
multiplying the topic loading of Topic X by the length of the total document. This provides an
estimate of the number of words used to discuss each topic.
Figure 3 Panel A plots the median number of words in each of the broader categories
over time. In general, the pattern is striking. Most topics have remained relatively constant over
the sample period and therefore do not explain the overall increase in 10-K length. The notable
exception is “Compliance with SEC and Accounting Standards” which increased markedly
during the sample period. Essentially all of the increase in the length of disclosure over the
sample period appears to be associated with specific SEC and FASB requirements as opposed to
more general discussion. Figure 3 Panel B provides a similar trend analysis but expressed as the
median proportion of total disclosure (i.e., we scale the proportion of disclosure on each topic so
that the total adds up to 1).12 Again, we see that the proportion of disclosure related to the
Compliance category has increased markedly as a proportion of the total length over the sample
period, while the proportions of the other categories (by construction) have decreased.
While the preceding discussion provides some potential insight into the source of the
increased length in 10-Ks and suggests that the bulk of the increase can be explained by new and
enhanced disclosure requirements, the categorization of individual topics into broader categories
is admittedly somewhat ad hoc. A primary strength of the LDA approach, however, is that we
are able to disentangle the specific subtopics that drive the length in the broader categories.
While the preceding analysis of broader categories is somewhat subjective, reflecting the
12
Although the proportions of all topics and topic categories within individual documents sum to one, the sum of
median proportions within a given year may not. For expositional clarity, we scale the sum of all median proportions
by year to sum to one; the inferences from the unscaled graph are identical.
20
judgment of the two coders independently assigning the 150 topics to 13 categories, this next
level of analysis is more objective because the computer algorithm determines the 150 individual
topics and assigns specific text to them.
Table 4 reports the top increasing topics by topic length. The first three of the top five
relate to topics which we categorized as “Compliance with SEC and Accounting Standards,”
including fair value/impairment, internal control, and risk factor disclosures. Notably, these top
three increasing topics alone make up the bulk of the increase in overall length with increases of
4,300, 2,200, and 2,100 words, respectively, compared to an increase of less than 600 words for
the next most increasing topic, customer accounts. Because of the large magnitude of the
increases in the lengths of these three topics compared to all other topics, we focus on them in
our remaining analyses. Examining them individually by year allows us to establish when (and,
indirectly, why) these topics increased so substantially. 13
The first of these three topics relates to fair value and impairment disclosure. Its top
words according to the LDA procedure are: “fair,” “reporting,” “consolidated,” “impairment,”
and “control.”14 Others of the top 20 words also support interpretation of this topic as relating to
fair value including “future,” “recognized,” “estimated,” “expected” and “asset.” Because SFAS
157 is the most important standard to affect fair value accounting, we expect that much of the
disclosure categorized under this topic will be related to that standard. In Table 5 we list the
representative paragraphs for each of our Top 3 increasing topics, including the Fair
Value/Impairment topic. The representative paragraph for this topic relates to the effect of fair
13
These three subtopics taken together account for almost 9,000 of the 10,000 total median word increase in the
Compliance topic over the sample period. Because these topics may substitute for other topics which decreased in
prominence, we also evaluated whether any related topics decreased in length. Consistent with the notion that
disclosure is seldom eliminated, none of the related topics substantially decreased in length over our sample period.
14
Note that “value” was excluded from the LDA procedure because it is extremely common; therefore, it cannot
appear as a keyword for any topic.
21
values for evaluating goodwill impairment; in addition to establishing a framework for
measuring fair value accounting, SFAS 157 specifically amended SFAS 142 relating to goodwill
impairment. Careful examination of many of the paragraphs with a high loading of the fair value
topic indicates that the fair value grouping reflects fair value discussion on a range of topics
including derivatives, investment securities, and other investments. We also provide an
additional example paragraph (fair value topic loading greater than 0.5) to provide a more
comprehensive view of the range of discussions that fall within this topic. This paragraph
discusses the use of fair values in yearly evaluations of debt and equity securities, also related to
SFAS 157.
The next topic relates to internal control disclosure. This disclosure is easy to identify,
with its top five words consisting of “control,” “internal,” “reporting,” “registrant,” and
“material.” The representative paragraph is the auditor’s opinion on management’s assessment of
internal control, as originally required under SOX Section 404. Because of the standardized or
“boilerplate” nature of internal controls disclosure, the representative paragraph is almost
identical in composition (cosine similarity of 0.99) to other paragraphs with high loadings of the
same topic and a similar length.
Our last main topic of interest is risk factor disclosure. The top five words in this topic
are “results,” “future,” “ability,” “result,” and “adversely.” This type of language is consistent
with risk factor disclosures that are intended to provide information on future events that might
adversely affect firm performance. This interpretation is consistent with the representative
paragraph, which describes the loss of key talent and personnel as a risk factor for the firm.
Similar to the fair value topic, the risk factor disclosure topic also includes a wide range of
discussions because firms face a wide variety of risks. We also include another example
22
paragraph for this topic, relating to risk associated with possible security breaches, to provide an
idea of the breadth of the topic. Although some firms disclosed risk factors voluntarily
throughout our sample period, the SEC mandated this disclosure in Item 1A of the 10-K in 2005.
As further support that these three top increasing topics are capturing the type of
disclosure that we have attributed to them, we identify specific firm attributes that should be
associated with each of the three topics and link them with the length of these topics. In the case
of Internal Controls, we expect significant additional text for firms with internal control
weaknesses; for Fair Value/Impairments we expect additional text for firms with substantial onetime items; and for Risk Factors we expect additional text for firms with substantial market risk.
In Table 6 we establish that our textual topics do in fact correlate with the factors that we expect
by conducting a regression of topic length on special items, internal control weaknesses, and the
firm’s market beta.15
Results in Table 6 indicate that, overall, the three quantitative characteristics, special
items, internal control weaknesses, and market risk, all have significant and predictable
associations with 10-K length.16 More importantly, each measure is significantly and predictably
correlated with the associated textual characteristic. In particular, Fair Value disclosure tends to
be longer in cases in which there are more (negative) special items, Internal Control disclosure
tends to be longer in cases in which there are more internal control weaknesses, and Risk Factor
disclosure tends to be longer in cases of higher market risk for the firm. 17 The fact that the LDA
15
We use beta as a measure of risk following research such as Campbell et al. (2014). Admittedly, risk factors
might reflect both systematic and unsystematic risk. Results are consistent if risk is measured based on overall
return volatility or firm-level litigation risk.
16
The coefficient on special items is negative, consistent with the notion that special items are generally negative
(e.g., losses) and that larger negative special items are associated with lengthier text. Results are consistent (with a
significantly positive coefficient) if we replace signed special items with the absolute value.
17
Table 6 includes our standard controls, industry fixed effects and a time trend. Similar inferences are obtained
when (1) using firm and year fixed effects, (2) using a Tobit model (excluding industry fixed effects), (3) estimating
the models over only the sub-period after SOX was effective and Material Weaknesses could take on a value greater
23
topic lengths correlate as expected with the underlying attributes provides additional reassurance
that our LDA topics reflect the constructs they are designed to capture. Finally, the regressions
include the Trend variable which remains very strong and positive for each of the text topics,
suggesting that, controlling for the underlying fundamental factors, there is still a very strong
increase in each of the disclosure topics over time.
Next, we investigate more closely the trends in fair value, risk factor, and internal control
disclosure to establish whether increases in these topics are linked with the regulatory events
mentioned above. Figure 4 plots the trends for the top three increasing topics over time and
provides evidence consistent with our predictions. Panel A plots the length of the Fair Value
topic. The graph is interesting for several reasons. First, recall that SFAS 157, “Fair Value
Measurements,” was passed in 2006 and required for fiscal years beginning after November 15,
2007 (i.e., generally in fiscal 2008), with early adoption encouraged. That timeline is very
consistent with the path of disclosure around 2006-2008, with virtually no disclosure for that
topic pre-2007, an initial substantial increase during 2007 likely reflecting early adopters, and the
bulk of the increase during 2008. The fact that the pattern is consistent with expectations is
reassuring because it suggests that, while LDA is a naïve Bayesian approach to categorizing text,
it can identify discussion associated with a specific topic quite crisply irrespective of where it
appears within a document. This is important because, although LDA has been applied in other
contexts, it has not previously been used to identify text associated with a specific accounting
rule.
Second, and more importantly, the figure indicates that disclosure around SFAS 157 was
a major source of additional length in the typical 10-K. Recall that the length of the Compliance
than 0, (4) using the presence of a material weakness instead of the number of material weaknesses, (5) dropping
topic length less than or equal to 5 words, or (6) using topic loadings in place of topic length.
24
category in Figure 3 increased by about 10,000 words; in comparison, the increase in disclosure
pertaining to SFAS 157 alone was nearly 4,300 words. This helps to explain the substantial
overall increase in the length of 10-Ks in 2007 and beyond. It is also interesting to note that this
increase does not appear to have been temporary. In fact, the text associated with this topic
leveled out to some extent after the 2008 mandatory adoption date but continued to rise, albeit
more gradually, through 2013, suggesting that additional disclosure was necessitated with
application of the standard (and related clarifications) over time.
The second largest increase is due to disclosures concerning internal controls. Recall that
SOX internal control certifications were required for fiscal years starting in 2004 and 2005.
Panel B shows a distinct increase in disclosure for the LDA topic we label Internal Controls
between 2004 and 2005, leveling off in 2006, suggesting that LDA correctly identified internal
control disclosure. More importantly, Figure 4 suggests that internal control discussion is an
important determinant of the increase in 10-K length, especially between 2004 and 2006. Unlike
fair value disclosure which has continued to increase in length, the text associated with internal
controls dropped somewhat between 2007 and 2008 before leveling off at about 2,100 words,
down from a high of 3,900 words in 2006. This drop coincides with the introduction of Auditing
Standard 5 (AS5) by the PCAOB for fiscal years ending on or after November 15 th 2007. Among
other changes to auditing procedures, AS 5 allows the auditor to issue a combined report of its
opinion on both the financial statements and the internal controls over financial reporting
whereas previously auditors were required to issue two separate reports.18
18
We observe this same decrease when examining only the subset of firms that never reported an internal control
weakness, suggesting that a higher incidence of firms with internal control weaknesses around initial
implementation of Sarbanes-Oxley is not the sole driver of this peak and that firms without internal control
weaknesses also experienced an initial increase, and subsequent decrease, in their discussion of the topic.
25
The third major source of the increased length is forward-looking disclosure associated
with risk factors, depicted in Panel C. While not specifically required in the 10-K prior to 2005
(although required in prospectuses for debt and equity offerings), firms often provided risk factor
disclosures voluntarily when they made forward-looking statements (Campbell et al. 2014).
Beginning in 2005, the SEC emphasized the importance of adequate risk factor disclosures and
required that they be discussed in a separate section of the 10-K (Item 1A). As a result, we
expect an increase in the discussion of risk factors throughout our sample period as SEC interest
increased, but with a substantial increase around 2005 when the new rules became effective. The
graph for the risk factor topic displays the predicted pattern, with a gradual increase through
2004 followed by a substantial jump in 2005 and a more gradual increase subsequent to 2005.
Unlike internal control disclosure, but as with fair values, the increase in disclosure around the
effective date does not appear to have been temporary, with an increasing subsequent trend likely
reflecting the SEC’s continuing focus on implementation following the initial requirements in
2005. By 2013, median risk factor disclosure had increased by almost 2,300 words.
Figure 4 Panel D displays the sum of the three specific topics over time. Overall, as
noted earlier, the topics combine to explain an increase of almost 10,000 words in 10-K length.
Further, there is a marked similarity between the increase in Compliance disclosure from Figure
3 and the sum of the three components in Figure 4 Panel D suggesting that those three factors
explain most of the increase in Compliance disclosure (which, in turn, explains most of the
increase in total 10-K length). In particular, both Figure 3 and Figure 4 Panel D indicate a
gradual increase in text length from about 1996 to 2003, associated with an increase in risk factor
disclosure. The increase in length accelerates around 2003, reflecting required internal control
disclosure, reinforced in 2005 by mandated risk factor disclosure. The gradual decline in internal
26
control disclosure in 2007 and 2008 is offset by large increases in fair value disclosure, resulting
in a steady climb in combined text length over the post-2007 period.
Overall, the preceding analysis provides consistent insight into the causes and content of
the additional length in 10-K disclosure. It seems clear that the increase in length was not the
result of changing underlying economics but, rather, reflects compliance with disclosure and
regulatory requirements, particularly fair values/impairments, internal controls, and risk factors.
V.
Does the Additional Text Increase Fog and Redundancy?
Having documented that much of the increase in 10-K length appears to be a result of
increases in specific topics related to Compliance disclosure associated with accounting and
other regulatory action during the mid-2000s, we next investigate the textual characteristics of
this additional disclosure. In particular, to what extent do the same topics that explain the
increase in length also explain the increase in average redundancy and Fog over the sample
period documented in Figure 1?
A primary advantage of LDA is that it identifies incremental text by topic so that we can
evaluate which specific topics explain increases in characteristics such as Fog and redundancy
over time. We focus on Fog and redundancy because, as noted earlier, standard setters and
regulators have singled out complexity and redundancy of disclosure as potential barriers to the
efficient use of financial reports by investors and other stakeholders (KPMG, 2011; FASB, 2012;
SEC, 2013). Furthermore, prior literature suggests that redundancy and Fog can affect the
usefulness of disclosures and the ease with which information can be extracted. For example,
Loughran and McDonald (2014) and Miller (2010) suggest that complexity of disclosure is
associated with higher uncertainty around the 10-K filing date and lower trading activity by
27
small investors, respectively. In addition, Rennekamp (2012) provides evidence that Fog can
influence individuals to overreact to positive and negative information when estimating value in
a laboratory setting. Redundancy has also been linked with the usefulness of disclosure; Cazier
and Pfeiffer (2015b) show that a higher amount of disclosure that is redundant with other text
within the same document leads to less efficient price discovery, and Lang and Stice-Lawrence
(2015) provide evidence that disclosure that is redundant across firms is associated with lower
levels of liquidity, institutional ownership, and analyst following.
Given the potential importance of Fog and redundancy to effective communication, we
investigate the extent to which Compliance-related disclosures explain the increases in these two
attributes documented in Figure 1 Panels C and D. An advantage of LDA is that we can apply it
at the paragraph level to evaluate textual characteristics within subsets of text. Although the
initial output of the LDA procedure does not identify where specific topics are discussed within
each document, we can use our trained LDA model to re-analyze each paragraph and estimate
paragraph-level topic loadings (essentially the probability that the paragraph belongs to a specific
topic) in a process called “inferencing.” We can then assign the paragraph to the topic which has
the highest loading. 19 The benefit of this procedure is that we are able to assign all paragraphs to
individual topics and measure the textual characteristics of disclosure specifically relating to that
topic, in particular Fog and redundancy.
Table 7 provides descriptive statistics for the paragraphs relating to aggregate disclosure
for each of the topic categories, including Compliance, as well as for our Top 3 increasing topics.
19
This classification procedure introduces noise because a given paragraph may discuss multiple topics, which
would cause the textual characteristics for paragraphs assigned to a given topic to revert to the mean because some
disclosure has been misclassified. We chose not to exclude paragraphs that might include multiple topics so that we
can aggregate all of our statistics up to the document level, but we find similar (if not stronger) results when we
instead impose a cutoff loading (probability) of 0.5 in order to categorize a paragraph as a specific topic.
28
We measure the average amount of redundancy, Fog, and tone-related words in paragraphs
relating to each topic, where redundancy is expressed as the percent of the 10-K that is
redundant, and our three measures of tone are the proportion of words that are negative,
uncertain, and litigious, based on the Loughran and McDonald business words dictionary. 20
These average statistics match up well with our interpretation of the topics based on the
top 20 words and representative paragraphs per topic, and provide assurance that our paragraphlevel approach is relatively effective at identifying paragraphs relating to particular topics. For
example, risk factor disclosures have the largest median level of negative and uncertain tone
words of any of the topics or categories provided, which is to be expected because risk factor
disclosure relates to uncertainty about potential negative outcomes.
In terms of primary characteristics of interest, Fog and redundancy, the broad category
statistics in the first panel of Table 7 indicate that Compliance disclosure tends to be
substantially more complex and redundant than aggregate non-Compliance disclosure in the 10K. Based on the third panel in Table 7, Compliance disclosure has the highest level of Fog of
any category in the 10-K measured either based on the mean or median. Compliance disclosure
also has the highest median level of redundancy of any category in the 10-K (although the mean
level of redundancy is higher for Contracts & Legal). In untabulated tests we find these
differences to be statistically significant, with mean Compliance disclosure significantly more
complex and redundant relative to the disclosure of all other categories combined (p < 0.001).
The fact that the Compliance category has disproportionally high levels of Fog and redundancy,
coupled with the earlier finding that the proportion of the 10-K devoted to Compliance has
increased substantially, suggests that the overall increase in average Fog and redundancy over
20
To reduce noise, descriptive statistics for each topic (category) are only calculated for documents which have at
least 100 words in paragraphs assigned to that topic (category).
29
the sample period could be the result of increases in the proportion of the 10-K representing
Compliance disclosure.
In terms of the Top 3 increasing topics, which fall within Compliance, the descriptive
statistics suggest that internal control disclosures have particularly high levels of Fog and
redundancy, above the level for the typical Compliance disclosure and well above the level for
non-Compliance disclosure. Both fair value and risk factor disclosure also have higher levels of
Fog than typical non-Compliance disclosure and fair value disclosure has a higher mean level of
redundancy than typical non-Compliance disclosure. Only risk factor disclosure has relatively
low redundancy, consistent with research such as Campbell et al. (2014) that risk factor
disclosure tends to be informative. Overall, however, the fact that the Top 3 increasing topics
have relatively high levels of Fog and redundancy suggests that the rising prevalence of
Compliance disclosure in general, and the Top 3 topics in particular, could explain the overall
increase in Fog and redundant disclosure over our sample period.
In Figure 5 Panel A, we investigate the extent to which Compliance disclosure helps
explain the trends in Fog. Recall from Figure 1 Panel D that Fog has increased nearly
monotonically since 2001. Panel A graphs the median level of Fog for each of our disclosure
categories. Two points are worth noting. First, consistent with Table 7, the level of Fog
associated with Compliance is consistently above that of the other categories over the entire
sample period suggesting that Compliance-related text tends to be complex in general. Second,
all of the categories exhibit a fairly constant level of textual complexity over time with the
exception of Compliance which experiences an increase beginning in 2001.
To provide further evidence on the extent to which the Fog in Compliance disclosure
explains the overall increase in Fog over our sample period, we categorize each paragraph in the
30
10-K in terms of Fog and identify the high-Fog (or “foggy”) paragraphs as those with abovemedian levels of Fog.21 Panel B plots the increase in high-Fog paragraphs over the sample
period. Because the graph is scaled by the number of high-Fog paragraphs in 1996, the ending
value of nearly 2.5 indicates that there were 2.5 times as many high-Fog paragraphs in 10-Ks in
2013 relative to 1996. The graph also reports the number of high-Fog paragraphs by major
category in the 10-K. Clearly, the bulk of the increase in Fog during our sample period is due to
the increasing number of high-Fog paragraphs in the Compliance section, while increases in the
other sections are relatively modest. Because increases in the number of high-Fog paragraphs in
Panel B are driven to some extent by increases in the total number of paragraphs over time, Panel
C plots the median proportion of high-Fog paragraphs related to Compliance relative to the total
number of high-Fog paragraphs in the document. The proportion of high-Fog paragraphs related
to Compliance has increased substantially over the sample period.
In Panel D we evaluate the potential for the Top 3 topics to explain the increase in highFog paragraphs over time. The comparison is analogous to Panel C and is expressed as a
proportion of total high-Fog paragraphs. From Panel C we see that the Compliance category’s
share of complex paragraphs more than doubled (i.e., increased from 0.20 to 0.41) over our
sample period. Of that 0.21 increase, 0.18 (almost 90% of the total) was attributable to risk
factors, fair values, and internal controls. The primary source of the increased Fog was internal
control disclosure which increased sharply concurrent with Sarbanes-Oxley internal control
disclosure requirements in 2004. Risk factors also contributed to the overall increase in Fog,
with a jump in 2007 associated with required risk factor disclosure, and, to a lesser extent, so did
21
We cannot simply add up Fog across sections because Fog is not additive. However, once we identify high-Fog
paragraphs we can add the number of high-Fog paragraphs across sections. We use the median level of Fog in 1996
(21.2) to categorize high-Fog paragraphs in our primary analysis, but conclusions are similar using other cut-offs.
31
fair values. Overall, it appears that the bulk of the increase in Fog since 2001 results from the
increasing prevalence of Compliance disclosure (and internal control, risk factor, and fair value
disclosure in particular) in the 10-K.
In Figure 6, we conduct a similar analysis for redundancy. Given that we have identified
the topic for every paragraph using LDA, we can link each redundant sentence with an individual
topics and aggregate redundant words at the topic level (i.e., redundancy is additive). Panel A
plots the median number of redundant words per category over time and provides very clear
evidence that the increase in overall redundancy in the 10-K is driven by increases in redundancy
in Compliance disclosure. While redundant words in other disclosure categories increased only
slightly, their effect is swamped by the increase in redundant words related to Compliance.
Panel B graphs the increase in redundant words for the Top 3 increasing subtopics. There
is strong evidence of an increase over time for the redundant content of the Top 3 topics taken
together which mirrors the increase for the Compliance category in general. Again, the most
pronounced increase is in redundancy around the imposition of internal control requirements in
2004. Fair value contributes to the increase in redundancy beginning around 2007, coincident
with the adoption of SFAS 157. In contrast, redundancy associated with risk factor disclosure
contributed very little to the overall increase in redundant disclosure over the sample period,
consistent with the notion that, although risk factor disclosures have been increasing in quantity,
they tend to have low redundancy (as documented in Table 7).
The results to this point tell a clear and consistent story. Overall length of the 10-K has
increased dramatically over time, as has complexity and redundancy, justifying concerns
expressed by regulators and investors about the quality of corporate disclosure. Results based on
LDA indicate that the increase in length, complexity, and redundancy is predominately the result
32
of increased Compliance-related disclosure requirements from the FASB and SEC. Further,
within Compliance the bulk of the increase in length, complexity, and redundancy is the result of
disclosure associated with three disclosure requirements: fair value disclosure resulting from
SFAS 157, SEC-mandated risk factor disclosure, and SEC-mandated internal control disclosure.
This is particularly noteworthy because, although regulators and standard setters have urged
companies to avoid making their reports more complex and redundant, it appears that disclosure
mandates from the FASB and SEC have been primary contributors to the increase in length, Fog,
and redundancy of 10-Ks.22
VI.
Is the Additional Disclosure Useful?
To this point we have focused on documenting the trends in textual attributes over time
and taken at face value the notion that understanding the source of lengthy, complex, and
redundant disclosure is important in improving financial reporting. Given the concerns
expressed about these attributes by regulators and users of financial statements, as well as the
extensive research suggesting that lengthy, complex, and redundant disclosure may be less
informative, we believe that understanding the trends and causes for these attributes is important
in its own right. That being said, our approach also permits us to provide direct (albeit
exploratory) evidence on the potential usefulness of the resulting disclosure to market
participants. An advantage of LDA in this regard is that it permits us to examine the association
between economic outcomes and the quantity of disclosure about a particular topic in the 10-K.
22
The fact that the increases in length, Fog, and redundancy are associated with specific FASB and SEC
requirements does not necessarily imply that the FASB and SEC are directly responsible. For example, it is possible
that firms implemented the requirements in a manner that was not anticipated when the rules were written because
of concerns over litigation. However, it is still the case that the rules, as implemented, substantially increased length,
Fog, and redundancy. Further, if the SEC wishes to reduce those attributes, the fact that LDA allows us to identify
the specific causes and locations of additional length, complexity, and redundancy should facilitate their efforts.
33
In particular, we examine the ability of the primary topics that have increased in length
over time (fair values, internal controls, and risk factors) to explain short-window market
outcomes and predict future fundamentals. In terms of market outcomes, we examine changes in
illiquidity (LnAmihud, BidAsk), turnover (Turnover), and cumulative abnormal returns (CAR)
over the 3-day window starting at the 10-K release date. If this disclosure is informative at the
time of the 10-K release, we would expect that it would be associated with lower illiquidity,
greater turnover, and stock price movement. There are, of course, challenges with this type of
analysis because the 10-K text, along with the associated numerical data, is released as a single
package making it difficult to isolate the effect of a particular type of disclosure. Additionally,
the magnitude of the market response to the 10-K release is generally relatively small because
much of the information may have previously been released to the market through earlier filings
(i.e., 10-Qs, 8-Ks) or the earnings announcement.
In Table 8 Panel A we assess the ability of risk factor, fair value, and internal control
disclosure to explain market outcomes around the 10-K filing date, controlling for a variety of
other factors as well as firm and year fixed effects. Of the three categories of disclosure, only
risk factors are consistently associated with market outcomes. In particular, more discussion of
risk factors is associated with lower levels of illiquidity in terms of both bid/ask spreads and
Amihud price impact, and with higher levels of turnover, consistent with these disclosures
decreasing information asymmetry among investors. In addition, discussion of risk factors is
associated with negative signed abnormal returns, consistent with the notion that they convey
information about potential future challenges facing the firm. 23
23
We use signed returns because we have directional predictions. Inferences are consistent with unsigned returns.
34
In contrast, discussion of fair values is not significantly correlated with illiquidity,
turnover, or returns around the 10-K release date. Similarly, internal control disclosure is not
significantly correlated with illiquidity or abnormal returns. The association between internal
control disclosure and turnover is negative and marginally significant, suggesting that, if
anything, investors are less willing to trade in firms with more discussion of internal control
issues. Overall, though, the analysis suggests that only risk factors appear to be consistently
associated with short-window market outcomes. 24
A second approach is to examine the association between discussion of the Top 3 topics
and future fundamentals. In particular, we focus on the ability of fair value/impairment
disclosure to predict future special items, internal control disclosure to predict future material
weaknesses, and risk factor disclosure to predict future betas. While the choice of future
fundamentals is admittedly subjective, recall from Table 6 that fair value/impairment, internal
control, and risk factor disclosures are correlated with concurrent special items, material
weaknesses, and beta, respectively. Our interest is in establishing whether they provide
information about future outcomes incremental to current outcomes.25
Table 8 Panel B reports results relating fair value, risk factor, and internal control
disclosure to future outcomes after controlling for a variety of factors including current levels of
the outcomes, as well as firm and year fixed effects. Results are very consistent with those based
on short-window market outcomes. In particular, risk factor disclosure appears to be effective in
24
Inferences are unchanged when Fog and Redundancy or the numeric equivalents of each topic examined in Table
6 are included as additional control variables.
25
The underlying presumption is that textual disclosures on risk factors, internal controls, and fair values are
designed, in part, to help investors predict future outcomes. For example, understanding the nature of risk factors
helps investors anticipate future adverse events, understanding the firm’s internal controls helps investors anticipate
future material weaknesses, and understanding how fair values are measured helps investors estimate future
securities gains/losses and impairments.
35
predicting future beta, controlling for current beta.26 Fair value and internal control disclosures,
on the other hand, are not significantly associated with future special items or material
weaknesses. 27
We emphasize that our analysis of outcomes is exploratory because it conditions on
either short-window market reactions or predictions of specific future items. However, it
illustrates a potential approach to using topical content generated by LDA to explore the
informativeness of specific types of disclosure. Further, it is striking that the conclusion that risk
factor disclosure is informative is consistent, both in terms of short window outcomes as well as
in predicting future risk. That conclusion is also consistent with the finding in the previous
section that risk factor disclosure tends to have very little redundancy. Fair value and internal
control disclosures have high levels of redundancy, suggesting that these disclosures may be
largely “boilerplate” in the sense that the firm is, for example, explaining the regulatory
requirements and describing the implementation of standards (e.g., definitions of Level 1, 2, and
3 assets, approaches for assessing internal control weaknesses, etc.). The finding that risk factor
disclosure is informative at the 10-K release date is also consistent with research such as
Campbell et al. (2014) and Bao and Datta (2014), and with the SEC’s rationale for requiring
formal disclosure of risk factors as a separate item beginning in 2005. 28
26
Results are robust to a variety of alternative outcome measures. For example, risk factor disclosure predicts not
only systematic risk (beta) but also return volatility and litigation risk.
27
Results remain insignificant if fair value disclosure is used to predict future levels and changes in accumulated
other comprehensive income (which reflects unrealized gains or losses on securities available for sale) or intangible
assets (which are a frequent source of impairments). Similarly, internal control disclosure does not predict future 10K filing lag (time between the fiscal year end and 10-K release) which has been linked to the quality of a firm’s
internal information system (Gallemore and Labro 2015).
28
Item 503(c) states that risk factors must appear in a separate section, Item 1A, and must be written in plain
English. In addition, it must include “a discussion of the most significant factors that make the offering speculative
or risky. This discussion must be concise and organized logically. Do not present risks that could apply to any issuer
or any offering. Explain how the risk affects the issuer or the securities being offered.”
36
VI. Conclusions and Ongoing Research
Despite frequent complaints by financial statement users, practitioners, and regulators
that financial disclosure is becoming more onerous, complex, and redundant, there has been little
empirical evidence to identify the extent of the problem or, more importantly, to determine the
cause. In this paper we attempt to inform this debate by providing empirical evidence that speaks
to these issues. First, we find that the length of 10-K disclosure (as well as Fog and redundancy)
is clearly increasing, and this trend cannot be explained by changes in underlying firm
economics or the demand for information. Second, LDA-based analysis indicates that, while the
increase in length is spread across multiple sections of the 10-K, it results primarily from
increases in Compliance-related disclosure. Third, most of the increase in Compliance-related
disclosure results from new SEC and accounting regulations related to fair value disclosure
required by SFAS 157, internal control disclosure under Sarbanes Oxley, and risk factor
disclosure mandated by the SEC in Item 1A. Fourth, increases in Compliance-related disclosure
in general, and fair values, risk factors, and internal controls in particular, explain not only the
increasing length of the 10-K but also the increasing levels of Fog and redundancy. Finally, of
the Top 3 increasing topics, only risk factor disclosure appears to be effective in explaining the
short-term market reaction around the 10-K release or in forecasting future economic outcomes.
Admittedly, our evidence is exploratory and subject to caveats. However, we believe that
our paper opens up a variety of other avenues that can be explored. For example, our technique
could be applied to identify specific sources of redundancy and Fog across sections of the annual
report, permitting more specific assessment of costs and benefits of complex and redundant
disclosures and, potentially, ways to reduce redundancy and complexity without sacrificing
information content. In some cases, for example, the Fog and redundancy may reflect
37
implementation decisions by the firm rather than intended disclosure by the standard setter, and
identifying specific complex and redundant text could suggest potential improvements to clarity
without sacrificing overall information content. Additionally, there are other textual attributes
which would be natural candidates for similar analysis, as well as other filings that could be
explored. For example, it might be useful to establish the extent to which specific topic
disclosure tends to be particularly “sticky” over time, is comparable across firms, and reflects
high levels of boilerplate using measures from the prior literature (e.g., Lang and Stice-Lawrence
2015). Likewise, registration and proxy statements likely raise similar issues.
While our analysis cannot answer normative questions such as which textual information
should be included in the 10-K, we believe our findings have the potential to contribute to the
ongoing regulatory discussion on topics such as disclosure effectiveness, redundancy, and
overload. More broadly, we highlight the potential contribution of LDA as a tool for
summarizing text for a large number of lengthy documents such as for the population of 10-Ks
and other regulatory filings. Our analysis suggests that LDA has promise in allowing researchers
to further open the “black box” of textual disclosure and understand the underlying information
in an objective manner that can be efficiently applied to large numbers of lengthy documents.
We expect further development of similar techniques to be useful to standard setters and
regulators in evaluating the effectiveness of both long-existing disclosure (e.g., for purposes of
assessing whether the disclosure has outlived its usefulness) and recently-enacted disclosure
(e.g., to assess how it is being applied in practice and whether it is serving its intended purpose)
as well as to investors in processing lengthy documents in a timely manner.
38
References
Ball, C., Hoberg, G., Maksimovic, V., 2014. Disclosure, business change and earnings quality.
Unpublished Working Paper.
Berardino, J., 2001. Enron: A wake-up call. Wall Street Journal, Dec 4, 2001.
Bao, Y., Datta, A., 2014. Simultaneously discovering and quantifying risk types from textual risk
disclosures. Management Science 60, 1371-1391.
Blei, D., Ng, A., Jordan, M., 2003. Latent dirichlet allocation. The Journal of Machine Learning
Research 3, 993-1022.
Bloomfield, R., 2008. Discussion of “Annual report readability, current earnings, and earnings
persistence.” Journal of Accounting and Economics 45, 248-252.
Campbell, J.L, Chen, H., Dhaliwal, D.S., Lu, H., Steele, L.B., 2014. The information content of
mandatory risk factor disclosures in corporate filings. Review of Accounting Studies 19, 396455.
Cazier, R.A., Pfeiffer, R.J., 2015a. Why are 10-K filings so long? Accounting Horizons,
Forthcoming.
Cazier, R.A., Pfeiffer, R.J., 2015b. Say again? Assessing redundancy in 10-K disclosures.
Unpublished Working Paper.
Chang, J., Boyd-Graber, J.L., Gerrish, S., Wang, C., Blei, D.M., 2009. Reading tea leaves: How
humans interpret topic models. Bengio Y, Schuurmans D, Lafferty JD, Williams CKI, Culotta A,
eds. Advanced Neural Information Processing Systems (Curran Associates, New York), 288–
296.
Chen, E., 2011. “Topic modeling the Sarah Palin emails,” Edwin Chen (blog).
http://blog.echen.me/2011/06/27/topic-modeling-the-sarah-palin-emails/
39
Gallemore, J., Labro, E., 2015. The importance of the internal information environment for tax
avoidance. Journal of Accounting and Economics 60, 149-167.
Groves, R. J., 1994. Here's the annual report. Got a few hours? Wall Street Journal, Aug 4, 1994.
Financial Accounting Standards Board, 2012. Disclosure Framework: Invitation to Comment.
Norwalk, CT.
Huang, A., Lehavy, R., Zang, A., Zheng, R., 2014. A thematic analysis of analyst information
discovery and information interpretation roles. Unpublished Working Paper.
Hoberg, G., Lewis, C., 2015. Do fraudulent firms produce abnormal disclosure? Unpublished
Working Paper.
Kim, I., Skinner, D.J., 2012. Measuring securities litigation risk. Journal of Accounting and
Economics 53, 290-310.
KPMG, 2011. Disclosure overload and complexity: hidden in plain sight. Available at:
http://www.kpmg.com/US/en/IssuesAndInsights/ArticlesPublications/Documents/disclosureoverload-complexity.pdf.
Lang, M., Stice-Lawrence, L., 2015. Textual analysis and international financial reporting: Large
sample evidence. Journal of Accounting and Economics 60, 110-135.
Li, F., 2008. Annual report readability, current earnings, and earnings persistence. Journal of
Accounting and Economics 43, 221-247.
Loughran, T., Mcdonald, B., 2014. Measuring readability in financial disclosures. The Journal of
Finance 69, 1643-71.
McCallum, A., 2002. MALLET: A Machine Learning for Language Toolkit.
http://mallet.cs.umass.edu.
Miller, B. P., 2010. The effects of reporting complexity on small and large investor trading. The
Accounting Review 85, 2107-43.
Monga, V., Chasan, E., 2015. The 109,894-word annual report; as regulators require more
disclosures, 10-Ks reach epic lengths; how much is too much? Wall Street Journal (Online), Jun
2, 2015.
Rennekamp, K., 2012. Processing fluency and investors’ reactions to disclosure readability.
Journal of Accounting Research 50, 1319–1354.
Securities and Exchange Commission (SEC), 2013. Report on Review of Disclosure
Requirements in Regulation S-K. Available at: http://www.sec.gov/news/studies/2013/reg-skdisclosure-requirements-review.pdf. SEC Offices, Washington D.C.
40
Srivastava, A., 2014. Why have measures of earnings quality changed over time? Journal of
Accounting and Economics 57, 196-217.
White, M.J., 2013. The Path Forward on Disclosure. National Harbor, MD.
41
Appendix
Variable Definitions
Textual Variables
Variable
Words
Description
The number of words used in the 10-K.
The natural logarithm of one plus the number of words used in either
LnWords
the 10-K or the specified 10-K item sections.
The Gunning (1952) Fog index, where Fog = 0.4*(average number
Fog
of words per sentence + percent of complex words), where complex
words are the words in excess of two syllables.
The number of words in sentences that are repeated verbatim in other
Redundant Words
portions of the 10-K.
Redundancy
The percent of redundant words in a given portion of text.
The loading (length) of the fair value/impairment topic identified by
Fair Value/Impairment
the LDA model. (The length is calculated by multiplying the loading
Loading (Length)
by the total length of the text.)
The loading (length) of the internal control topic identified by the
Internal Control
LDA model. (The length is calculated by multiplying the loading by
Loadings (Length)
the total length of the text.)
Risk Factor
The loading (length) of the risk factor disclosures topic identified by
Disclosures Loading the LDA model. (The length is calculated by multiplying the loading
(Length)
by the total length of the text.)
The percent of negative words (according to the Loughran &
Negative
McDonald business words dictionary) contained within paragraphs
pertaining to a given topic (for all topics).
The percent of uncertainty-related words (according to the Loughran
Uncertainty
& McDonald business words dictionary) contained within
paragraphs pertaining to a given topic (for all topics).
The percent of litigation-related words (according to the Loughran &
Litigation
McDonald business words dictionary) contained within paragraphs
pertaining to a given topic (for all topics).
Other Variables
Variable
Analyst
Beta
BidAsk
Description
The number of analysts providing EPS forecasts for the fiscal period.
If missing, this value is set to 0.
The market beta over the firm fiscal period, where beta is the
estimated coefficient from a regression of firm daily returns on value
weighted market returns, both adjusted for the risk free rate.
The bid-ask spread, calculated as the difference between the bid and
ask price, divided by the average of the two; this final ratio is then
multiplied by 1,000.
42
BigN
BusSeg
CAR
ForSeg
Inst
Leverage
Litigation Risk
LnAmihud
LnAssets
Loss
Material Weaknesses
Merger
MTB
NYSE
Peer AAER
Peer SEC Comments
ROA
S&P 500 Return
SEC Comments
Takes the value of 1 for the following five audit firms: Arthur
Andersen, Ernst and Young, Pricewaterhousecoopers, and KPMG.
Missing values are set to 0.
The number of business segments [BUSSEG]. If missing, this value
is set to 0.
The cumulative abnormal return, calculated using a market model
with value-weighted market returns.
The number of foreign segments [GEOSEG]. If missing, this value is
set to 0.
The percentage of shares outstanding owned by institutional
investors. If missing, this value is set to 0.
The long-term and current period debt scaled by total assets
([DLTT]+[DLC])/[AT].
The predicted litigation risk using Model (3) from Kim and Skinner
(2012) using the years 1996-2012. Litigationt = FPSt + log(Assets)t-1
+ Salesgrowtht-1 + Returnt-1 + Skewnesst-1 + STD_Returnt-1 +
Turnovert-1.
The natural logarithm of the Amihud price impact of trade measure
multiplied by 10 million.
The natural logarithm of total assets [AT].
An indicator variable that takes on the value of 1 when net income
[NI] is below 0, and 1 otherwise.
The number of material weaknesses in internal controls that a firm
reports for that period.
Takes on the value of 1 if in the Compustat footnote database, any of
the variable values are "AA", "AR", "AS", "FA", "FB", "FC", "FQ",
"VC". If no such variable values are found, then MERGER is 0.
The market value of equity [CSHO]*[PRCC_F] divided by the book
value of equity [CEQ].
Takes on a value of 1 if the firm is listed on either NYSE (EXCHCD
= 1) or AMEX (EXCHCD = 2), and 0 otherwise.
The total number of SEC AAER enforcements given to firms in the
industry, excluding the firm’s own AAER received if any. If no
industry AAER enforcements are received, then the variable takes on
the value of 0.
The total number of comments received in a firm's industry, during
the firm's fiscal year, minus the firm’s own comments received
during the fiscal year. If no industry comments were received, then
the variable takes on the value of 0.
The return on assets calculated as net income divided by total assets
[IB]/[AT].
The S&P 500 return, cumulated over the fiscal period.
The number of SEC comments received by a firm is calculated as the
total number of comments for 10-K or 10-Q reports received during
the firm fiscal year. If no comments are received, then the variable
takes on the value of 0.
43
Special Items
SUE
Trend
Turnover
Special items [SPI], scaled by total assets [AT]. If missing, this value
is set to 0.
The unexpected earnings surprise is calculated as the actual earnings
minus the median analyst earnings forecast, scaled by price.
A variable that increments by 1 each year, starting at 0 in the fiscal
year 1996.
The total number of shares traded divided by the total number of
common shares outstanding; this ratio is then multiplied by 1,000.
Sample Restrictions
Before analyzing any 10-K filings, we exclude all amended and small business (Form 10KSB) filings from our sample, as well as those filed before June 1, 1996 (when electronic filing
was still voluntary) and Form 10-K405s. We also exclude documents containing fewer than
3,000 words and those that are missing basic data from Compustat (assets, net income, shares
outstanding, price, and book value of equity).
10-K Cleaning Procedures
We use Perl to remove all HTML and non-relevant text from the 10-K filings in our
sample using procedures similar to those used in Li (2008). First, we remove all header and
appendix information, including the SEC header section at the start of all 10-K documents, as
well as any graphics, zip files, xml files, excel files, 101 exhibits, 100 exhibits, pdf files, and
XBRL. Second, we remove all HTML text from the file using the HTML::Parser Perl module.
Any remaining tags such as <TEXT>, <PAGE>, <DOCUMENT>, and <TYPE> are removed
following Miller (2010). We also delete lines with <S> and <C> following Miller (2010). Third,
we implement character restrictions to the document. We delete lines with fewer than 20
characters or 15 alphanumeric characters, which removes lines of just numbers as well as section
headings. Following Li (2008) we further delete paragraphs with more than 50 percent non-
44
alphabetic characters. Additionally, we remove paragraphs with fewer than 80 characters
(Blankespoore, 2014).
Identifying 10-K Item Sections
In order to uniquely identify each of the item sections within the 10-K, we implement the
first two steps in the 10-K cleaning procedure described above and then generate unique
identifiers for all instances where a reference to an item section was used in the 10-K
document.29 This identifier tracks the sequence in which the references to the section were used.
In order to identify which reference is the true starting location of the item section, we iteratively
remove section references that are inconsistent with logical ordering of the section numbers. In
this iterative process, we take the last full sequence, if multiple sequences exist, which removes
tables of content. If two references are referenced only once and are neighboring sections, we
remove references between. If an identified reference does not have the necessary sections that
follow or precede it (e.g. Section 7 is not followed by Section 8, or preceded by Section 6), then
it is removed. For those documents where this iterative process is reduced to a unique sequence
of all of the required sections, we break apart the 10-K at the locations of each section reference,
and then perform the final step of the 10-K cleaning procedure on each section separately.
Finally, we impose minimum word limits for some sections, to ensure that we are capturing the
actual section and not just a reference to the section. For sections 1, 7 and 8 this threshold level is
50 words. For sections 10, 11, and 12 the level is 20 words. The length of the remaining sections
was subject to too much natural variability for us to determine a reasonable cutoff. This process
allows us to identify all of the sections in 22,349 10-K filings.
29
We do not separately identify Section 2 or any section beyond 14 as Section 2 was often combined with Section 1
and those beyond 14 were not consistent throughout our sample period.
45
Perplexity
The formula for perplexity from Blei et al. (2003) is:
𝑝𝑒𝑟𝑝𝑙𝑒𝑥𝑖𝑡𝑦(𝐷𝑡𝑒𝑠𝑡 ) = 𝑒𝑥𝑝 {−
∑𝑀
𝑑=1 log 𝑝 (𝑤𝑑 )
}
∑𝑀
𝑑=1 𝑁𝑑
It is a function of the per-word likelihood, p(wd), and the number of words in each document, Nd.
Perplexity decreases as the likelihood of the model increases, or in other words when the
statistical fit is better.
In order to calculate the perplexity plotted below, we trained the model on 90% of our
data and then calculated the perplexity using a random hold-out sample of the remaining 10% of
the observations.
Word Intrusion Task
46
In order to identify the topic model with the best fit paired with high interpretability,
we perform a word intrusion task for LDA models estimated using 100, 150, and 200 topics. We
choose these three models because the incremental decrease in perplexity after 150 topics is
relatively small whereas there are obvious gains to model fit for less than 150 topics.
The word intrusion task is structured as follows. A human coder is presented with a set of
6 words in a random order. Five of the words are the words with the highest probability of
appearing in Topic X according to the model, whereas the sixth word has a low probability of
occurring in Topic X.30 Participants in the task are asked to choose the word which does not
match the other five words, or in other words the “intruder” word. For example, the set of 6
words could be: debt, loan, facility, term, inventory, revolving. In this case, the intruder word is
“inventory,” as the rest of the topic is about debt. These groups of six words were generated for
each potential topic in each of the three potential models and presented (unlabeled) to the coders
in a random order. Our two human coders reviewed each group of words and chose an intruder
word.31 The relevant statistic is the percent of the time the human coders agreed with the model,
where high agreement indicates high interpretability of the topics. For both coders, the model
with the highest interpretability was the 150-topic model so we use this in all of our subsequent
tests.
Word Constraints in the LDA Procedure
We place a few constraints on the documents that we use when estimating our LDA
model. We first remove all common stopwords such as “is,” “the,” and “and” as these are not
30
The intruder words that we select are among the 15% least probable words for the given topic. Following Chang
et al. (2009) we further constrain these words to be relatively common in at least one other topic to prevent coders
from identifying them as the intruder words by virtue of their rare usage in any financial topic. Thus our intruder
words must be in the top 20 most common words in at least one other topic.
31
Both coders have a background in accounting and business and are familiar with financial terminology.
47
useful in classifying topics and decrease performance, and all words that do not occur in at least
100 documents. Additionally, certain words that are extremely common in firm annual reports
(such as “company” and “value”) are so common that they prevent the model from estimating.
All words that occur in every document or are in the top 0.1% most common words are
excluded. These words are listed below:
company will value information years upon company's fiscal rate based report
sales management services form costs related tax ended certain market credit
products amount period net including operations securities cash time statements
income section common assets shares business plan year date interest december
agreement stock may financial million shall
Paragraph-Level Analysis
Measuring Textual Attributes at the Topic level
We measure textual attributes at the topic level by first breaking each document into
paragraphs. 32 We then use our trained topic model to “infer” topics at the paragraph level;
essentially this takes the probabilities per word that we calculated at the document level for the
entire corpus, and then uses the observed words in the given paragraph to estimate the topic
loadings of all of the topics for that paragraph. This can be done out of sample (for documents
not in the training sample), but we use it solely on paragraphs from the original sample (see Blei
et al. 2003 for more information on inferencing). We then identify the topic for which the
paragraph has the highest loading (i.e. the topic that is discussed the most) and we assign that
paragraph to that topic.
“Paragraphs” is a loose description. Specifically, what we refer to as “paragraphs” are portions of text separated
from all other text by two end of line markers (e.g., carriage returns).
32
48
After performing the above process for all paragraphs in the corpus, we end up with 150
groups of paragraphs, where each group consists of paragraphs that share the same dominant
topic. We can then calculate Fog, redundancy, and tone for each paragraph and estimate the
average statistics for these measures at the topic level (i.e. for all paragraphs sharing the same
topic).
Calculating Fog and tone at the paragraph level is a simple matter of using the
appropriate formulas, described in the Variable Definitions section of the Appendix, at the
paragraph instead of the document level. However, calculating redundancy at the paragraph level
is a little more complicated; we estimate it by identifying sentences within paragraphs attributed
to that topic which have previously been discussed in the annual report. We then count the
number of words within these redundant sentences and add them up within the topic. Our
approach relies on the relative ordering of sentences within the document to identify those which
are “redundant”; the first time that a sentence is written it is included in our non-redundant word
count, and is only categorized as redundant when it is repeated. This approach works well at the
document level, but can introduce noise when performed at the paragraph level. When two
identical sentences occur within paragraphs that relate to different (perhaps related) topics, the
sentence that occurs first will add no redundancy to the topic of the paragraph in which it
appears, while the second (identical) sentence will. This adds noise to our measure of
redundancy across topics within the same document, but only in those cases where identical
sentences occur in paragraphs relating to different topics, which is unlikely to be common. A
noise-free approach would be to assign both topics and redundancy at the sentence-level, in
which case redundant sentences would always be attributed to the same topic, regardless of their
49
ordering in the text. However, identifying topics at the sentence level is extremely
computationally intensive and not generally feasible for such a large corpus.
Identifying Representative Paragraphs
We follow an approach very similar to Hoberg and Lewis (2015) to identify the
representative paragraphs for our LDA topic model, except that we use inferencing at the
paragraph level instead of cosine similarity to identify the most prominent topic in each
paragraph. That is, for each topic we identify the 1,000 paragraphs with the highest topic loading
for that topic and then retain only the middle tercile of documents by length (number of words).
Among the remaining 333 paragraphs, we compare each paragraph with all of the other
paragraphs using cosine similarity and select the paragraph that has the highest average
similarity with the other paragraphs.
One thing to keep in mind with this procedure is that it favors picking paragraphs with
more “standardized” content because this will be shared across many firms. For example, in a
simple example where all documents contain two paragraphs about Topic A, where one of the
paragraphs quotes verbatim the accounting standard that applies to that topic and the other
paragraph describes the application of that standard to each particular firm, then this process
would select the standard paragraph as the representative paragraph. In other words, this process
may ignore important variability in discussions of a topic that can arise across firms in favor of
picking standardized and potentially “boilerplate” paragraphs. Inferences from these paragraphs
must therefore be drawn with that caveat in mind.
50
Figure 1. Trends in Textual Attributes Over Time
Panel B. 10-K Length Relative to 1996
Panel A. 10-K Length Over Time
70000
2.2
60000
2
1.8
50000
P75
40000
1.6
Mean
Median Length,
Survivor Firms
1.4
Median
30000
Median Length
Median Nonredundant Length
1.2
P25
20000
1
10000
0.8
Panel C. 10-K Redundant Words Over Time
Median Residual
Length
Panel D. 10-K Fog Over Time
6000
22.5
5000
22
4000
21.5
P75
3000
2000
P75
Mean
Mean
21
Median
Median
P25
P25
20.5
1000
20
0
Residual Length in Panel B is length after adjusting for the control variables in Table 2 Column 1.
51
Figure 2: Median 10-K Section Length by Year
40000
35000
30000
25000
20000
15000
10000
5000
0
1&2 (Business and Property Description)
3 (Legal)
5 (Security Market Information)
6 (Selected Financials)
7 (MD&A)
8 (Financial Statements)
9 (Auditor Disagreement/Internal Controls)
10 (Directors, Officers, Corp. Gov.)
11 (Executive Compensation)
12 (Security Ownership)
13 (Relationships/Independence)
Available only for the 22,349 documents for which we can identify all sections.
52
Figure 3. Disclosure Over Time by LDA Topic Category
Panel A. Median Disclosure Length by Topic Category
30000
1
2
25000
3
4
20000
5
6
15000
7
8
10000
9
10
5000
11
12
0
13
Panel B. Median Disclosure Loading By Topic Category
1
1
0.9
2
0.8
3
0.7
4
0.6
5
6
0.5
7
0.4
8
0.3
9
0.2
10
11
0.1
12
0
13
1. Performance, Revenues, and Customers
8.
Business Structure & M&A
2. Industry-Specific Disclosure
9.
Contracts & Legal
3. Empoyees & Executives
10. Geographic Location
4. Compliance with SEC & Accounting Standards 11. Investments, Securities, Derivatives
5. Loans, Debt, Banking
12. Intellectual Property & R&D
6. Business Operations & Strategy
13. Property and Leasing
7. Stock and Options
Although topic loadings within documents sum to 1, median loadings across firms do
not; for presentation purposes, median yearly loadings in Panel B are scaled to sum to 1.
53
Figure 4. Median Length of Top 3 Increasing Topics Over Time
Panel A. Fair Value/Impairment Disclosure
Panel B. Internal Control Disclosure
5000
5000
4000
4000
3000
3000
2000
2000
1000
1000
0
0
SFAS 157
Length
SOX 404
Length
Panel D. Top 3 Topics Combined
Panel C. Risk Factor Disclosure
2500
10000
2000
8000
1500
6000
1000
4000
500
2000
0
0
Item 1A
AS 5
Length
Combined Length
SFAS 157 was passed in September 2006 and became effective for fiscal reporting periods commencing after November 15, 2007 with early adoption
encouraged. SOX 404 became effective for accelerated filers for fiscal years ending on or after June 15, 2004, and for all other firms by 2007. AS 5 was
effective for fiscal years ending on or after November 15th, 2007. Item 1A became mandatory for firms filing with the SEC for fiscal periods ending on
or after December 1, 2005. Panel D plots the median of the sum of these three topics over time.
54
Figure 5. Contribution of Top 3 Topics to Fog Over Time
Panel A. Median Fog Over Time by LDA Topic
Category
24
Panel B. Number of "Foggy" Paragraphs Over Time
by LDA Topic Category, Relative to 1996
2.5
Compliance
23
2
22
1.5
21
19
0.5
18
0
Panel C. Median Proportion of Total "Foggy"
Paragraphs Over Time by LDA Topic Category
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Compliance
1
20
Panel D. Median Proportion of Total "Foggy"
Paragraphs Over Time for Top 3 Increasing Topics
0.25
0.2
0.15
0.1
Compliance
0.05
0
Risk Factors
Internal Control
Fair Value/Impairment
This table demonstrates how the Top 3 increasing topics by length contribute to overall Fog. Panel A shows yearly median Fog by topic
category. Panel B shows the yearly number of total "foggy" paragraphs that belong to each topic category relative to the total number of
"foggy" paragraphs in 1996, where "foggy" paragraphs are those with Fog higher than the median Fog in 1996 (21.2). Panel C shows the yearly
median proportion of total "foggy" paragraphs that belong to each topic category. Panel D shows the yearly median proportion of "foggy"
paragraphs that belong to just the Top 3 topics. For presentation purposes, total yearly median proportions are scaled to sum to 1.
55
Figure 6. Contribution of Top 3 Topics to Redundancy Over Time
Panel A. Median Redundant Words Over Time by LDA Topic
Category
2500
2000
1500
1000
Compliance
500
0
Panel B. Median Redundant Words Over Time for Top 3
Increasing Topics
500
400
300
200
100
0
Risk Factors
Internal Control
Fair Value/Impairment
This table demonstrates how the Top 3 increasing topics by length contribute to
overall redundancy. Panel A shows yearly median redundant words by topic
category. Panel B shows yearly median redundant words for only the Top 3 topics.
56
Table 1. Descriptive Statistics
Words
Fog
Redundant Words
BigN
Assets ($)
MTB
Leverage
BusSeg
ForSeg
NYSE
Loss
Mean
Std Dev.
Q1
Median
Q3
45,348.93 31,453.93 24,678.00 37,370.00 55,852.00
21.39
1.26
20.59
21.25
21.99
3,911.00 5,490.02 1,114.00 2,280.00 4,296.00
0.77
0.42
1.00
1.00
1.00
3,551.30 11,733.65
91.85
391.85
1,747.65
3.14
4.21
1.17
1.90
3.33
0.21
0.20
0.02
0.16
0.34
1.95
1.47
1.00
1.00
3.00
1.14
1.64
0.00
1.00
2.00
0.43
0.50
0.00
0.00
1.00
0.31
0.46
0.00
0.00
1.00
Assets are measured in millions and scaled to be in constant year 1996 dollars.
57
Table 2. Determinants of Overall Length
Trend
BigN
LnAssets
MTB
Leverage
Age
BusSeg
ForSeg
NYSE
Loss
Inst
Analyst
Litigation Risk
SEC Comments
Peer SEC Comments
Peer AAER
ROA
SUE
Merger
S&P 500 Return
LnWords
(1)
Coeff
t-stat
0.039 *** 65.907
0.047 *** 5.755
0.144 *** 49.894
0.005 *** 7.340
0.179 *** 8.600
-0.007 *** -18.578
0.020 *** 7.731
0.008 *** 3.974
-0.006
-0.685
0.230 *** 38.481
Observations
75,991
R-squared
0.367
Industry Fixed Effects
Y
LnWords
(2)
Coeff
t-stat
0.033 *** 40.173
0.033 *** 2.849
0.119 *** 22.690
0.002 ** 2.238
0.200 *** 6.916
-0.004 *** -9.135
0.017 *** 5.523
0.007 *** 2.733
0.006
0.495
0.154 *** 17.310
0.061 *** 3.943
-0.003 *** -3.636
0.903 *** 6.952
0.005 *** 5.789
0.000 *** 4.191
0.007 *** 6.701
-0.266 *** -10.129
-0.045
-0.526
0.057 *** 7.169
-0.050 *** -3.649
38,316
0.355
Y
Determinants of length, including cross-sectional determinants and a timeseries trend (Trend ). Assets are inflation-adjusted to be in constant year 1996
dollars. All continuous variables are winsorized at the 1st and 99th percentile
by year. Standard errors are adjusted for heteroskedasticity and clustered by
firm. *** p<0.01, ** p<0.05, * p<0.1
58
Table 3. Description of Topic Categories
Category Label
Description
Topics relating to day-to-day company operations or strategy (discussion of customers is in
Business Operations & Strategy
the topic relating to performance). EX: Products, advertising, accounts receivable,
contractors, software, and systems.
Discussions of the current business structure and organization, or changes to these. EX:
Business Structure & M&A
Subsidiaries, partnerships, acquisitions, bankruptcy and reorganization, trusts, joint ventures.
Discussions of SEC requirements and accounting standards, or disclosures to comply with
these requirements. EX: Issuance of new accounting standards, discussion of regulatory
Compliance with SEC & Accounting Standards
documents for the annual report or prospectus, (management's certification of) internal
controls, fair value disclosure, required risk factor disclosures.
Contracts & Legal
Disclosure about legal agreements or procedings. EX: Provisions of contracts, litigation.
Disclosure about employees and executives. EX: Salaries and benefits, retirement, unions,
Employees & Executives
executive backgrounds, indemnification agreements, code of conduct.
Discussions about various specific geographic regions (mostly in relation to regional
Geographic Location
operations). EX: Southwestern United States, China, Midwest, Latin America.
Intellectual property and research and development. EX: Patents, laboratory research,
Intellectual Property & R&D
licensing rights.
Discussion of the firm's investments. EX: General investment activity and risk, securities
Investments, Securities, Derivatives
investment and trading revenue, REITs, derivatives.
All discussions relating to loans and debt. EX: Loan obligations, payments, rates, and
Loans, Debt, Banking
collateral; mortgages; debentures; and default.
Discussion of performance, revenue, and customers. EX: Performance summary, clients
Performance, Revenues, and Customers
and revenue, customer accounts, distribution to customers, growth, special items.
Property and Leasing
Topics relating to properties. EX: Leases, tenant-landlord issues, and transactions.
Discussions relating to the company's own stock, including options, warrants, and
Stock and Options
dividends.
Industry-Specific Disclosure Categories
Healthcare & Medical
Insurance
Energy, Resources, and Utilities
Transportation
Media, Communications, and Leisure
Technology Industry
Consumer Products
Other Industry-Specific
For more detailed information on the specific topics that are included in each category, see the Internet Appendix.
59
Table 4. Top 15 Increasing LDA Topics
Change in
Median Length
4317.41
2216.18
2092.66
590.19
356.85
343.78
328.74
270.25
216.31
212.47
143.79
116.33
46.7
37.89
19.93
Topic Title
Fair Value/Impairment
Internal Control Disclosure
Risk Factor Disclosures
Customer Accounts
Financing (Facilities)
Accounting Standards Codification (ASC)
Derivatives
Acquisitions
Exhibits Incorporated by Reference
Growth
Foreign Currency Exchange
Special Items
Litigation
CEO/CFO Certification of Internal Controls
Pension and Retirement Plans
Topic Category
Compliance with SEC and Accounting Standards
Compliance with SEC and Accounting Standards
Compliance with SEC and Accounting Standards
Performance, Revenues, and Customers
Loans, Debt, Banking
Compliance with SEC and Accounting Standards
Investments, Securities, Derivatives
Business Structure & M&A
Compliance with SEC and Accounting Standards
Performance, Revenues, and Customers
Business Operations & Strategy
Performance, Revenues, and Customers
Contracts & Legal
Compliance with SEC and Accounting Standards
Employee & Executives
The change in median length for each topic is calculated as the average median length for 2013 and 2012 minus the average of median
length for 1996 and 1997.
60
Table 5. Representative Paragraphs for Top 3 Increasing Topics
Topic
Representative Paragraph
Fair Value/Impairment
(Additional Example
Paragraph)
In accordance with GAAP, the Company has the option to first assess qualitative factors to determine whether it is necessary to perform
a more detailed quantitative impairment test. If the Company is able to determine through the qualitative assessment that it is more likely
than not that the fair value of a reporting unit exceeds its carrying value, no further evaluation is necessary. However, if the Company
concludes otherwise, then the Company is required to perform the first step of the two-step impairment test by calculating the reporting
unit's fair value and comparing the fair value to the reporting unit's carrying amount, including goodwill. If a reporting unit's fair value
exceeds its carrying value, the second step of the impairment test is not required and no impairment loss is recognized. If a reporting unit's
carrying value exceeds its fair value, the second step of the impairment test is performed to measure the amount of the impairment loss
and an impairment charge is recorded equal to the difference between the carrying value of the reporting unit's goodwill and the implied
fair value of the reporting unit's goodwill. The implied fair value of goodwill is determined in the same manner as the amount of goodwill
recognized in a business combination where the excess of the fair value of the reporting unit over the fair value of the identifiable net
assets of the reporting unit is the implied fair value of goodwill. See Note 5 Goodwill and Intangible Assets, Net.
Each reporting period we review all of our investments in equity and debt securities, except for those classified as trading, to determine
whether a significant event or change in circumstances has occurred that may have an adverse effect on the fair value of each investment.
When such events or changes occur, we evaluate the fair value compared to our cost basis in the investment. We also perform this
evaluation every reporting period for each investment for which our cost basis exceeded the fair value in the prior period. The fair values
of most of our investments in publicly traded companies are often readily available based on quoted market prices. For investments in
nonpublicly traded companies, management's assessment of fair value is based on valuation methodologies including discounted cash
flows, estimates of sales proceeds and appraisals, as appropriate. We consider the assumptions that we believe hypothetical marketplace
participants would use in evaluating estimated future cash flows when employing the discounted cash flow or estimates of sales proceeds
valuation methodologies.
Internal Control Disclosure
Also, in our opinion, management's assessment, included in M anagement's Report on Internal Control over Financial Reporting appearing
under Item 8, that the Company maintained effective internal control over financial reporting as of December 31, 2004 based on criteria
established in Internal Control - Integrated Framework issued by the Committee of Sponsoring Organizations of the Treadway
Commission (COSO), is fairly stated, in all material respects, based on those criteria. Furthermore, in our opinion, the Company
maintained, in all material respects, effective internal control over financial reporting as of December 31, 2004, based on criteria
established in Internal Control Integrated Framework issued by the COSO. The Company's management is responsible for maintaining
effective internal control over financial reporting and for its assessment of the effectiveness of internal control over financial reporting.
Our responsibility is to express opinions on management's assessment and on the effectiveness of the Company's internal control over
financial reporting based on our audit. We conducted our audit of internal control over financial reporting in accordance with the standards
of the Public Company Accounting Oversight Board (United States). Those standards require that we plan and perform the audit to
obtain reasonable assurance about whether effective internal control over financial reporting was maintained in all material respects. An
audit of internal control over financial reporting includes obtaining an understanding of internal control over financial reporting, evaluating
management's assessment, testing and evaluating the design and operating effectiveness of internal control, and performing such other
procedures as we consider necessary in the circumstances. We believe that our audit provides a reasonable basis for our opinions.
Fair Value/Impairment
61
Risk Factor Disclosures
The Company's future performance depends to a significant degree upon the continued contributions of its officers and key management,
sales and technical personnel, many of whom would be difficult to replace. The loss of any of these individuals could have a material
adverse effect on the Company's business, financial condition, results of operations and business prospects. In addition, the Company's
future success and ability to manage growth will be dependent upon its ability to hire additional highly skilled employees for a variety of
management, engineering, technical and sales and marketing positions. The competition for such personnel is intense, however, and there
can be no assurance that the Company will be able to attract, assimilate or retain sufficient qualified personnel to achieve its future
business objectives. The failure to do so could have a material adverse effect on the Company's business, financial condition, results of
operations and business prospects. See "Risk Factors -- Dependence on Key Personnel."
Risk Factor Disclosures
(Additional Example
Paragraph)
In addition, such events could materially adversely affect our reputation with our customers, associates, and vendors, as well as our
operations, results of operations, financial condition and liquidity, and could result in litigation against us or the imposition of penalties or
liabilities, which may not be covered by our insurance policies. M oreover, a security breach could require us to devote significant
management resources to address the problems created by the security breach and to expend significant additional resources to upgrade
further the security measures that we employ to guard such important personal information against cyberattacks and other attempts to
access such information and could result in a disruption of our operations, particularly our online sales operations.
To see the Representative Paragraphs for the remaining topics, see the Internet Appendix.
62
Table 6. Numeric Counterparts of Top 3 Increasing Topics
Special Items
Material Weaknesses
Beta
LnWords
(1)
Coeff
-0.263
***
0.074
***
0.056
***
Trend
BigN
LnAssets
MTB
Leverage
Age
BusSeg
ForSeg
NYSE
Loss
0.037
0.044
0.136
0.004
0.202
-0.006
0.020
0.007
0.001
0.206
Observations
R-squared
Industry Fixed Effects
75,991
0.371
Y
***
***
***
***
***
***
***
***
t-stat
-8.536
14.661
12.033
63.176
5.372
45.369
6.025
9.670
-17.866
7.857
3.465
0.081
*** 33.698
LnFair Value/Impairment
(2)
Coeff
t-stat
-1.703
*** -11.952
LnInternal Control
(3)
Coeff
t-stat
LnRisk Factor Disclosures
(4)
Coeff
t-stat
0.634 *** 31.812
0.484
0.421
0.233
-0.013
0.075
-0.000
0.001
0.115
-0.034
0.013
***
***
***
***
***
75,991
0.592
Y
202.242
11.109
22.430
-4.761
0.945
-0.262
0.124
12.506
-0.971
0.500
0.429
-0.190
0.052
0.015
-0.703
-0.000
-0.050
-0.028
-0.046
-0.361
75,991
0.579
Y
***
***
***
***
***
***
***
*
***
224.527
-7.764
6.952
7.096
-12.079
-0.170
-6.372
-4.044
-1.957
-17.910
0.476
***
21.022
0.235
0.327
0.047
-0.004
-0.010
-0.046
-0.069
-0.016
-0.244
0.464
***
***
***
88.256
8.063
3.240
-1.079
-0.104
-28.124
-5.340
-1.352
-5.386
17.007
***
***
***
***
75,991
0.319
Y
Determinants of logged length of the entire document (LnWords ) and also logged length of our three main topics of interest, Fair Value/Impairment, Internal
Control, and Risk Factor Disclosures. For each topic of interest, we identify a numeric counterpart that we think captures similar information to the textual
disclosure. Material Weaknesses is coded 0 before the implementation of SOX in order to preserve the entire sample period; this is appropriate because there
was no requirement to identify or disclose internal control weaknesses prior to this period. Assets are inflation-adjusted to be in constant year 1996 dollars. All
continuous variables are winsorized at the 1st and 99th percentile by year. Standard errors are adjusted for heteroskedasticity and clustered by firm. *** p<0.01,
** p<0.05, * p<0.1
63
Table 7. Topic-Level Textual Characteristics
Redundancy
Fog
Negative Words Uncertain Words Litigious Words
Mean Median Mean Median Mean Median Mean Median Mean Median
Compliance vs. Non-Compliance
Compliance
Non-Compliance
10.1%
6.3%
9.3%
4.9%
22.92
21.08
22.86
20.86
1.5%
1.4%
1.5%
1.4%
1.4%
1.0%
1.4%
1.0%
1.3%
1.6%
1.1%
1.4%
Top 3 Increasing Topics
Risk Factors
Internal Control
Fair Value/Impairment
3.2% 0.6%
12.3% 12.9%
7.4% 2.7%
22.33
26.01
21.56
21.88
26.56
21.45
2.9%
1.3%
2.0%
3.0%
1.3%
1.9%
3.0%
1.1%
2.1%
3.0%
0.9%
2.0%
1.3%
0.5%
0.4%
1.1%
0.3%
0.3%
Non-Compliance Categories
Contracts & Legal
Business Op. & Strategy
Business Struct. & M&A
Empoyees & Executives
Geographic Location
Intellectual Prop. & R&D
Investments, Sec. & Deriv.
Loans, Debt, Banking
Performance, Rev. & Cust.
Property & Leasing
Stock & Options
Industry-Specific
13.8%
4.8%
6.0%
5.2%
5.1%
4.6%
4.4%
7.5%
4.7%
4.2%
4.9%
4.7%
21.55
21.60
21.49
21.51
20.15
21.52
20.24
21.03
20.35
20.05
19.01
20.73
21.04
21.38
21.09
21.25
19.92
21.02
20.23
20.67
20.33
19.57
18.67
20.70
3.1%
1.8%
1.1%
1.3%
0.5%
2.3%
1.3%
1.0%
1.4%
1.1%
0.7%
1.1%
2.7%
1.5%
0.9%
1.2%
0.3%
2.0%
1.2%
0.9%
1.3%
0.8%
0.7%
1.0%
1.0%
1.2%
1.4%
0.8%
0.8%
1.4%
1.9%
0.9%
0.9%
1.1%
0.8%
1.0%
0.9%
1.2%
1.2%
0.7%
0.7%
1.3%
1.8%
0.9%
0.9%
0.8%
0.8%
0.9%
6.4%
1.5%
1.1%
2.2%
1.1%
1.5%
0.7%
1.1%
0.4%
1.2%
1.0%
0.9%
6.4%
1.3%
0.6%
1.9%
0.7%
1.3%
0.6%
0.7%
0.4%
0.5%
0.6%
0.7%
7.5%
2.0%
2.9%
1.1%
0.0%
0.0%
0.9%
4.1%
3.6%
0.0%
2.0%
2.7%
Descriptive statistics for textual characteristics measured at the topic- and category-level. Non-Compliance includes all categories other than
Compliance . Variable definitions given in the Appendix.
64
Table 8. Informativeness of Top 3 Topics
Panel A. Short-Window Market Outcomes
LnAmihud
BidAsk
Turnover
CAR
(1)
(2)
(3)
(4)
Coeff
t-stat Coeff
t-stat Coeff
t-stat Coeff
t-stat
LnRisk Factor Disclosures
LnInternal Control
LnFair Value/Impairment
-0.017 *** -4.232 -0.220 *** -4.624 0.180 *** 7.048 -0.001 *** -3.512
-0.004
-0.969 -0.001
-0.017 -0.062 ** -2.355 -0.000
-0.029
-0.002
-0.467 -0.006
-0.141 0.008
0.393 -0.000
-0.214
LnOther Words
BigN
LnAssets
MTB
Leverage
BusSeg
ForSeg
NYSE
Loss
-0.015
-0.162
-1.302
-0.129
2.646
0.016
0.029
0.173
0.688
Observations
R-squared
Firm and Year FE
53,527
0.459
Y
***
***
***
***
*
***
***
***
-1.110
-4.907
-67.658
-35.392
31.632
1.946
3.473
2.848
33.420
0.352
2.152
-4.995
-0.744
18.406
0.146
0.264
4.756
4.811
52,358
0.232
Y
65
*
***
***
***
***
1.959
4.847
-19.384
-21.842
16.961
1.562
*** 2.934
*** 5.778
*** 17.071
0.375
1.991
2.053
0.395
-2.480
-0.144
-0.262
-2.243
-0.281
55,751
0.083
Y
***
***
***
***
***
***
***
***
**
4.574
11.627
15.087
18.073
-4.646
-2.661
-4.285
-6.380
-2.121
-0.000
-0.003
-0.007
-0.001
0.008
0.000
0.000
-0.000
-0.003
55,751
0.006
Y
***
***
***
**
***
-0.748
-2.653
-8.437
-4.959
2.357
0.311
1.110
-0.187
-2.800
Panel B. Forecasting Future Fundamentals
Betat+1
Material Weaknesses t+1 Special Items t+1
(1)
(2)
(3)
Coeff
t-stat Coeff
t-stat Coeff
t-stat
LnRisk Factor Disclosures
Beta
LnInternal Control
Material Weaknesses
LnFair Value/Impairment
Special Items
0.011 *** 9.824
0.306 *** 51.688
LnOtherWords
BigN
LnAssets
MTB
Leverage
BusSeg
ForSeg
NYSE
Loss
0.014
0.041
0.095
0.025
-0.187
0.002
-0.008
-0.025
0.042
Observations
R-squared
Firm and Year FE
64,436
0.274
Y
0.000
0.133
***
0.429
7.683
0.000
0.305
-0.064 *** -6.478
***
***
***
***
***
3.257
4.770
16.502
24.601
-7.965
0.665
*** -2.875
* -1.721
*** 7.609
0.005
0.001
0.004
-0.000
0.014
0.004
0.003
0.008
-0.005
64,436
0.043
Y
*
**
*
1.710 -0.001 ** -2.358
0.170 0.001
0.825
1.083 -0.020 *** -13.743
-0.809 0.001 *** 6.090
0.927 0.031 *** 8.236
2.485 0.001
1.618
1.746 0.001 ** 2.180
0.747 0.003 * 1.888
-1.128 -0.011 *** -11.184
64,436
0.056
Y
Regression analyses of the informativeness of the Top 3 increasing topics. Panel A shows the ability of the
Top 3 topics to explain short-window market outcomes in the 3-day window surrounding the 10-K release,
starting on the day of the release. Panel B demonstrates the ability of the Top 3 topics to predict the nextperiod value of fundamentals that were shown in Table 6 to be linked to current disclosure of each of the Top
3 topics, controlling for the current level of the fundamentals. All specifications control for the logged length
of all other topics not included in the current specification (LnOtherWords). Assets are inflation-adjusted to
be in constant year 1996 dollars. All continuous variables are winsorized at the 1st and 99th percentile by
year. Standard errors are adjusted for heteroskedasticity and clustered by firm. *** p<0.01, ** p<0.05, * p<0.1
66