Labor and Employment Alert
December 2008
www.klgates.com
With the economy now in a
recession, many companies are
examining workforce size and
composition to ensure they
remain competitive and properly
adjusted to the market. This is
one in a series of occasional
client alerts that will explore the
employment and labor needs
of employers in this turbulent
To Test or Not to Test: Statistical Analysis
for Small Layoffs?
economic environment.
George P. Barbatsuly
One recurring question: are some samples simply too small to produce meaningful
data? That issue is particularly important in reductions in force, because the number of
workers terminated is often relatively small. It is conventional wisdom that statistics
are irrelevant or unreliable when used to analyze small groups, which sometimes
lulls employers into a false sense of complacency about testing a layoff. However,
employers should not reflexively dismiss the usefulness of testing small-scale layoffs
as certain statistical tests, such as Fisher’s Exact Test, have been deemed reliable for
even small groups and can provide peace of mind and valuable legal protection.
973.848.4104
george.barbatsuly@klgates.com
Statistics and Layoffs
Authors:
Michael A. Pavlick
412.355.6275
michael.pavlick@klgates.com
K&L Gates comprises approximately
1,700 lawyers in 28 offices located
in North America, Europe and Asia,
and represents capital markets
participants, entrepreneurs, growth
and middle market companies,
leading FORTUNE 100 and FTSE
100 global corporations and public
sector entities. For more information,
visit www.klgates.com.
“There are three kinds of lies: lies, damned lies, and statistics.” So said Benjamin
Disraeli, and so perhaps might the employees, decision makers, human resources
specialists, attorneys and judges involved in employment discrimination cases.
Though the use of statistics in employment discrimination litigation is prevalent, the
debate continues over the methods and meaning of statistical analyses.
Statistics can be a valuable tool for evaluating layoffs and provide evidence either
supporting or rebutting claims that a reduction in force was unlawfully discriminatory.
Consider John and Mary, who decide to see who can roll “six” the most with a single
six-sided die. One might expect that in 100 rolls of the die, John or Mary would roll
“six” 16 or 17 times, or one out of every six rolls. But in their competition, John rolls
“six” 80 times out of 100.
Can Mary show that John cheated? If Mary is lucky, she has “direct” or “smoking
gun” evidence, such as a loaded die. Or maybe she saw John clandestinely turn the
die on multiple occasions so that the “six” was facing up. Typically, Mary would not
have such direct evidence, and rarely does a terminated employee have direct evidence
that a reduction in force was biased by age or race or gender.
What can Mary do? She can turn to “circumstantial” or “indirect” evidence, such as
showing it to be statistically improbable that John could roll 80 “sixes” in 100 rolls
without cheating. In other words, Mary can show that the difference between the
expected result (one of six rolls, or 16 or 17 “sixes” total) is so divergent from the
actual result (80 “sixes”) that the divergence itself is “statistically significant” and
unlikely to be the result of random chance.
Just as statistics could be helpful to Mary, so they could be helpful to the laid-off
employee. Statistics could help the employee prove that the fact that 80 females were
laid off in a 100-person reduction in force is statistically significant with respect to the
question of whether her termination was motivated by her gender.
Labor and Employment Alert
Courts have widely accepted the use of statistics
by employment discrimination plaintiffs and the
companies defending against such claims. Indeed,
proof in “disparate impact” claims (those claims
that a facially neutral policy, practice or decision
has the effect of disproportionately harming a
protected classification of employees) is predicated
on showing a statistical disparity.
Problems with Small Sample Sizes
While courts are generally receptive to statistical
analyses, they become increasingly skeptical of
their value as the sample size shrinks. The United
States Court of Appeals for the Fourth Circuit
discounted plaintiffs’ data in one such case because
it “simply involved too small a sample size to have
any probative value.” Birkbeck v. Marvel Lighting
Corp., 30 F.3d 507 (4th Cir. 1994). More recently,
the Eighth Circuit discounted statistical evidence
because the “sample size and pattern are too small
to be statistically significant,” Carraher v. Target
Corp., 503 F.3d 714 (8th Cir. 2007), and the Ninth
Circuit commented that “two data sets of sixteen
workers are too small to form a reliable basis for
analysis.” Diaz v. Eagle Produce Ltd. Partnership,
521 F.3d 1201 (9th Cir. 2008).
There is scientific reason behind the conventional
wisdom espoused by the courts. Some statistical
tests, like the “t-test” or the “chi-square,” involve
levels of approximation. The accuracy of the
approximation will decrease as the sample size
decreases and, similarly, so will the accuracy of
the test results. However, it would be a mistake
for employers to assume that small sample sizes
cannot be accurately tested notwithstanding the
conventional wisdom.
Fisher’s Exact Test
Fisher’s Exact Test is a statistical significance
test used to determine if there are nonrandom
associations between two categorical variables. It
is, as the name suggests, an “exact” test, which
means that all its assumptions are met regardless of
sample size (remember that in approximation tests,
the approximations are not met but become more
accurate with sample size).
The test is relatively simple. It tests for a relationship
between two variables, as for example, in reductions
in force, between gender and termination. Fisher’s
Exact Test is entirely independent of sample size,
which makes it ideal for the small groups that so vex
the approximation tests (the flip side is that Fisher’s
Exact Test becomes difficult to calculate with large
samples and/or well-balanced data).
Courts have acknowledged the efficacy of Fisher’s
Exact Test in employment discrimination cases with
small sample sizes. In Bridgeport Guardians, Inc.
v. City of Bridgeport, 735 F. Supp. 1126 (D. Conn.
1990), aff ’d, 933 F.2d 1140 (2d Cir.), the court
rejected the plaintiffs’ arguments that the employer’s
statistics were highly susceptible to error because
of small sample sizes. Plaintiffs, said the court,
“ignore[d] the utility and sensitivity of the Fisher’s
exact test in giving exact results, rather than the
chi-square’s approximations . . . . [S]uch analysis
is useful in this type of case because ‘it gets to the
heart of the matter [by computing] the likelihood
that the number of promotions is the result of
random chance.’” In Clark v. Commonwealth of
Pennsylvania, 885 F. Supp. 694 (E.D. Pa. 1994),
the court agreed with the parties’ assertion that
Fisher’s Exact Test was the appropriate test given
the smaller sample sizes, and in Pietras v. Bd.
of Fire Commissioners of the Farmingville Fire
District, 180 F.3d 468 (2d Cir. 1999), the Equal
Employment Opportunity Commission (EEOC)
countered arguments that a finding of discrimination
could not stand because of a small sample size by
urging the court to analyze the statistics by using
Fisher’s Exact Test (the court found discrimination
based upon non-statistical evidence and did not
entertain the EEOC’s request). Thus, parties in
employment discrimination litigation are on notice
that appropriately selected statistical analyses
will be probative of discrimination regardless of
sample size.
December 2008
Labor and Employment Alert
To Test or Not to Test
Regardless of the size of a reduction in force, prudent
employers should consider statistically testing the
layoff to determine whether it is susceptible to
attack. Many employers already test reductions in
force using the EEOC’s so-called “80 Percent Rule.”
The 80 Percent Rule provides that a selection rate
for any protected characteristic which is less than
80 percent of the rate for the group with the highest
rate will be regarded as evidence of disparate impact
discrimination. 29 C.F.R. s. 1607.4(D). However,
the 80 Percent Rule provides cold comfort to the
employer. It is not a statistical test and the courts
have described it as nothing more than a “rule-ofthumb.” Moreover, the 80 Percent Rule is of suspect
utility where sample sizes are small.
Savvy employers commonly test large-scale layoffs
using statistical analyses because of the inherent
risks of employment action of that magnitude. But
many employers, perhaps lulled into complacency
by the conventional wisdom that statistics do not
work for small sample sizes, do not perceive any
value in statistical testing of small reductions in
force. Given the accepted effectiveness of certain
tests like Fisher’s Exact Test, however, employers
are well advised to consider such testing for smaller
layoffs, particularly in those instances in which
the initial breakdown of selected employees and
additional risk factors prompt extra caution.
As a general guideline, layoffs of fewer than 100
employees are ripe for analysis using an exact test
like Fisher’s Exact Test. When conducting such
tests, the employer should retain legal counsel to
facilitate the testing. Experienced counsel can not
only identify the right questions to ask, but in the
event that the statistical analysis reveals a disparate
impact, subsequent reconsideration of the reduction
in force will enjoy some level of attorney-client
privilege protection.
In short, employers should not reflexively dismiss
the usefulness of statistically testing small-scale
layoffs. Rather, employers should consider whether
testing using Fisher’s Exact Test or similar tests
might provide the employer with peace of mind now
and valuable protection later if the reduction in force
is subsequently challenged in court.
K&L Gates comprises multiple affiliated partnerships: a limited liability partnership with the full name K&L Gates LLP qualified in Delaware and
maintaining offices throughout the U.S., in Berlin, in Beijing (K&L Gates LLP Beijing Representative Office), and in Shanghai (K&L Gates LLP
Shanghai Representative Office); a limited liability partnership (also named K&L Gates LLP) incorporated in England and maintaining our London
and Paris offices; a Taiwan general partnership (K&L Gates) which practices from our Taipei office; and a Hong Kong general partnership (K&L
Gates, Solicitors) which practices from our Hong Kong office. K&L Gates maintains appropriate registrations in the jurisdictions in which its offices
are located. A list of the partners in each entity is available for inspection at any K&L Gates office.
This publication/newsletter is for informational purposes and does not contain or convey legal advice. The information herein should not be used
or relied upon in regard to any particular facts or circumstances without first consulting a lawyer.
Data Protection Act 1998—We may contact you from time to time with information on K&L Gates LLP seminars and with our regular newsletters,
which may be of interest to you. We will not provide your details to any third parties. Please e-mail london@klgates.com if you would prefer not
to receive this information.
©1996-2008 K&L Gates LLP. All Rights Reserved.
December 2008