Statistics in Class Certification
Proceedings
What they’re good for, and how to
discredit them
Paul Karlsgodt, Baker Hostetler
pkarlsgodt@bakerlaw.com
303.764.4013
Brian Troyer, Thompson Hine
brian.troyer@thompsonhine.com
216.566.5654
Rick Preston, Hitachi Consulting
1
rpreston@hitachiconsulting.com
303.329.8993
Copyright 2012
Paul Karlsgodt
Brian Troyer
Rick Preston
All rights reserved
Agenda
 Part I – Introduction (~15 min.)
 Why is this topic important?
 What do we mean by “statistics”?
 How are statistics used in class certification?
 Part II – Case law on the use of statistics in class certification
(~40 min.)
 Part III – Practical tips on presenting and challenging statistics
(~20 min.)
 Question and Answer (~15 min.)
2
Part I – Introduction
3
Why is this topic important?
 Wal-Mart Stores, Inc. v. Dukes creates a more




4
demanding standard for class certification
The lower courts are starting to fill in the gaps left by
the Dukes Court’s analysis—see, for example, Duran
v. U.S. Bank National Association
Both sides are likely to attempt to create a more welldeveloped factual record
Statistics often provide an appealing way to illustrate
how aggregate or common proof is possible.
Data is more available and accessible than ever
before.
Rough Justice & Big Data
Over the past decade, as storage and
computing power have increased exponentially,
it has become increasingly tempting to use
statistical sampling as a proxy for the actual
adjudication of facts in class or mass actions.
“Big Data: What It Is and Why You Should Care”
IDC (June 2011)
Hard Disc
Storage Price/GB
Solid State Disc
Storage Price/GB
Sources of Data Growth
•
•
•
•
•
5
Email, collaboration tools, and mobile devices
Machine and sensor-generated messages
Digitization of business records and personal
content
Instrument devices
Governance, privacy, and regulatory compliance
requirements
General Overview
“Statistics is the science and art of describing data and drawing inferences
from them”*
Statistics
Descriptive
Statistics
Describes relationships,
correlations, events
Inferential Statistics
Makes inferences,
generalizations, estimates,
predictions
*(Finkelstein and Levin, p. 1)
6
Types of Class Actions in Which
Statistics Are Commonly Used
 Employment discrimination
 Wage and hour
 Securities fraud
 Pollution and toxic exposure
 Consumer/sales and prescriptions
 Product failures
 Antitrust
7
Common Uses of Statistics in Law
 Most commonly presented to prove commonality
(Rule 23(a)(2)), predominance and Superiority
(Rule 23(b)(3)), and cohesiveness (Rule 23(b)(2))
 As proof of a common policy or practice
 As proof of a common relationship between the
defendant’s conduct and some injury to class
members (e.g. reliance, causation, injury)
 As common proof of aggregate or class-wide
damages, restitution
 Less commonly presented to prove other factors
 E.g., In re Initial Public Offering Securities Litig., 471
F.3d 24 (2d Cir. 2006) (numerosity).
8
Part II – Case Law on the Use of
Statistics in Class Certification
9
Common Impacts:“Fraud on the
Market”
 The Fraud on the Market Theory in Securities Litigation - Basic Inc. v.
Levinson, 485 U.S. 224, 247, 108 S.Ct. 978, 991, 99 L.Ed. 194 (1988).
 Efficient Market - the market price of a security reflects all information
known to the market. In re Burlington Coat Factory Sec. Litig., 114 F.3d
1410, 1425 (3d Cir. 1997).
 When the market characteristics satisfy the FOTM prerequisites,
individual reliance is rebuttably presumed.
 Quantitative/statistical proof used to show efficient market, not market
response to adverse information.
 Reliance is separate from the element of loss causation.
 Loss causation need not be proved as a condition to class certification.
Erica P. John Fund v. Halliburton Corp.
 But it presumably must still be susceptible to common resolution (Dukes).
 Nevertheless, common proof of loss causation and damages also are
typically based on quantitative market analysis, and plaintiffs often plead
and argue loss causation facts in support of fraud on the market/reliance
(to show that the market responded to adverse information).
10
Borrowing “Fraud on the Market”
 Efforts to apply these concepts from securities fraud cases
in consumer class actions represent one of the most
prevalent uses of statistical and quantitative analysis in
class certification.
 Statistical and econometric analyses are typically offered
to show that prices and sales of a consumer product were
inflated because of fraud—fraud on the consumer market.
 The difference is that markets for consumer goods and
services are inherently different from securities trading
markets.
11
Common Impacts: McLaughlin
 Plaintiffs alleged implicit representation that light cigarettes are
healthier; sought $800 billion.
 Plaintiffs relied upon sixteen experts, including economists who
proposed statistical and econometric analyses.
 Judge Weinstein certified nationwide class of light cigarette
consumers under RICO, applying “price impact” theory of reliance
similar to the FOTM theory.
 Reversed by McLaughlin v. American Co., 522 F.3d 215 (2d
Cir. 2008).
 Individual proof was required: reliance, loss causation, injury, damages





12
(and limitations).
Market for light cigarettes is not efficient.
Individual facts presented to show non-reliance by customers.
Expert’s survey evidence “pure speculation.”
Statistical analysis did not prove the relevant facts.
Rejected “fluid recovery” approach of awarding aggregate “class”
damages followed by “simplified proof of claim procedure” and cy pres.
Common Impacts: In re Neurontin
Sales and Mktg. Practices Litig.
 Causation problem: Which off-label prescriptions were caused by
allegedly fraudulent promotion?
 Plaintiffs relied upon econometric analysis to try to show causation of
“all” off-label prescriptions.
 In first opinion, 244 F.R.D. 89 (D.Mass. 2007), Judge Saris gave
plaintiffs opportunity to show through “statistical proof” that essentially all
prescriptions in each category were caused by fraud.
 Second class certification motion also denied, 257 F.R.D. 315 (D. Mass.
2009):
 Not an efficient market.
 Defendant’s right to present evidence defeats predominance
 Closer scrutiny of expert opinions for class certification was mandated
that presumed in earlier opinion.
 Where expert’s opinion was that less than substantially all (e.g.
>99%) of prescriptions were caused by fraud, individual inquiry
required.
 Where expert’s opinion was that substantially all prescriptions were
caused by fraud, the expert analysis was flawed.
13
Common Impact: In re Zyprexa
 Judge Weinstein’s certification of off-label economic loss
class under RICO reversed by the Second Circuit.
 UFCW Local 1776 & Participating Health & Welfare Fund v. Eli
Lilly & Co., 620 F.3d 121 (2d Cir. 2010).
 “Excess price” analysis could not provide common proof of
 but-for (transactional) causation, because drug pricing is inelastic.
 proximate (direct) causation, because alleged chain of causation
was incomplete.
 “Excess sales” theory could not provide common proof of
causation because, e.g.,
 it assumed away all other factors affecting prescriptions.
 There was individualized evidence of non-reliance.
 it ignored alternative prescriptions and costs, some of which
could even have cost more.
14
Common Impacts: Rhodes
 Rhodes v. E.I. Du Pont de Nemours and Co., 253 F.R.D. 365
(S.D.W.V. 2008)
 Medical monitoring claim based on contamination of drinking
water with C-8.
 Problems with toxicologist’s and epidemiologist’s quantitative
opinions offered to establish common proof:
 Did not address the question of the relationship between exposure




15
and a significantly increased risk of health problems; and
Did not provide any common proof that any given individual
suffered a significantly increased risk of the exposure.
Preliminary and insufficient data was used.
Failed to rule out other variables.
Proposed remedy was a precautionary public health measure, not
something that can be awarded as a tort remedy.
Common Practice/Policy: Wal-Mart
Stores, Inc. v. Dukes

At issue: Title VII sex discrimination claims

Plaintiffs are required to prove a pattern or policy of
discrimination.

Ninth Circuit affirmed certification of a class of 1.5
million current and former female employees,
arguing that all female employees were subject to a
discriminatory policy.

Dukes reaffirmed:
 that Rule 23 is not a mere pleading standard, but that
the proponent must prove that the requirements are
satisfied.
 that a court must conduct a “rigorous analysis.”
 that “[f]requently that ‘rigorous analysis’ will entail some
overlap with the merits of the plaintiff’s underlying claim.
That cannot be helped.” 131 S.Ct. at 2551.
16
The Broad Question:
What does a
“rigorous analysis”
of statistical
evidence looks like?
Dukes: Proof of Common Injury

Two ways to bridge gap between the
individual’s claim and the existence of a class
who suffered the same injury
 Biased testing procedure (not at issue)
 Significant proof of a general policy of
discrimination

Plaintiffs offered a “social framework” analysis
by sociologist Dr. Bielby claiming to show that
Wal-Mart’s corporate culture made it vulnerable
to gender bias, but
 he could not determine with any specificity how
regularly stereotypes played a meaningful role,
and
 could not say whether 0.5% or 95% of decisions
were discriminatory.
17
“If Bielby admittedly
has no answer to
that question, we
can safely disregard
what he has to say.”
131 S.Ct. at 2554.
Dukes: Plaintiff’s Proof of the Existence
of a Common Policy

Plaintiffs attempted to show through statistical
and anecdotal evidence a “common mode” of
exercising discretion.
 Dr. Drogin (statistician) compared, by region, the
number of women promoted with the percentage
of women in pool of hourly workers.
 Dr. Bendich (labor economist) compared work-
force data of Wal-Mart and competitors,
concluding that Wal-Mart promoted lower
percentage of women.
18
The only allegedly
discriminatory
general policy
identified was that
Wal-Mart gave
supervisors
discretion.
Dukes: Court Finds No Proof of the
Existence of a Common Policy

These statistical analyses failed to show that
the existence of a general policy or practice
of discrimination was a question common to
all class members.
 First, there was a mismatch between the
statistical method and conclusion - regional and
national disparities failed to provide a basis to
infer a “uniform, store-by-store disparity” and
thus a company-wide policy.
 Second, even assuming a disparity in each store
from regional or national data, “[m]erely showing
that Wal-Mart’s policy of discretion has produced
an overall sex-based disparity does not suffice.”
131 S.Ct. at 2555-56.

19
Note the dissent’s charge that the majority
misunderstood the methods used.
There were inferential
gaps between
plaintiffs’ statistical
analyses and their
conclusions.
Dukes: Rejection of Trial by Formula

The Court also rejected class certification based
on “Trial by Formula.” 131 S.Ct. at 2561.
 A sample set of class members’ claims would be
tried.
 The percentage of valid claims and the average
backpay award to determine a “class recovery” to be
distributed without further individual proceedings.

“We disapprove that novel project.”
 This scheme would deprive Wal-Mart of the right to
litigate defenses to individual claims, and would
violate the Rules Enabling Act.
 This holding is similar to the one in McLaughlin.

20
Is it a class if some members would win and
some would lose? Would the losers recover a
share of the award?
Trial by formula . . .
“We disapprove of
that novel project”
Dukes in Summary

Does not change the landscape regarding statistics and class certification but
confirms necessity of rigorous scrutiny.

Gives a strong hint in favor of Daubert, but does not answer the question.

The Court examined the statistical analyses and found inferential gaps between
the policy that statistics were claimed to show and what they actually showed.
 Court evaluated the merits/substance of the statistics.

Illustrates and confirms inherent limitations of statistical and aggregate proof.

Confirms that, validity of statistics aside, conceptual gaps are critical.
 Even if statistics showed the claimed pattern, that pattern would not establish
commonality.
 Whether any individual decision was discriminatory would still require individual proof.

21
Keep in mind that the issue was whether statistical evidence could be used as
representative proof on behalf of all women at once, not whether it could be used
at all by individual plaintiffs.
Statistical Concepts in Dukes
 Descriptive statistics
 Average salary male>female 2001
 Why the average? What is the distribution?
 Why 2001?
 Inferential statistics
 Promotion analysis controls for feeder job,
store, and move year .
 Valid given Bielby’s assertion that relocation
across stores “creates a greater burden for
women”?1
 Break-out sub-set for further analysis
 Was it responsible for the overall
differences ?
 What is left in the set of observations for,
“Wal-Mart, not Sam’s Club?”
22
1 Class Cert p. 25
Statistical Concepts in Dukes (cont’d)
• Recall that regression analysis is used to
describe the relationship between phenomena
• Plaintiffs in Dukes . . .
• Tried to predict salary using job held, store where
person worked, promotions/transfers, full-/part-time,
salaried/hourly
• Outcome: Using gender in the equation made it a
better predictor of observed salary.
• So gender was in fact significant. Earnings between men
and women are disparate
• But it was not determined to be caused by an active policy
to discriminate against women. So the difference is not
“impact”
• Just because there is a difference, doesn’t
make it actionable
23
Visualizing The Issues
 Is there a common “answer” for all class members—i.e. did the same set of
circumstances apply to each class member; “Yes” in Halliburton; “No” in Dukes
 Is there perhaps some other explanation (other than gender)?
 Root Cause (Ishikawa) Diagram
What Else?
Region
Personal
Traits
Dept
Store
Discretion
Gender
Personal decisions
Age
Family Situation
Mobility
Workload Expectations
Management
Behaviors
24
Policies &
Procedures
Full-/part-time
Tenure
Role
Previous job
Performance
Employment
Status
Class Definition
“’[A]ll women [w]ho have
been or may be subjected
to Wal-Mart’s challenged
pay and management track
promotions policies and
practices.”
Paraphrasing: While disparity may
exist, the underlying root causes are
likely to be different among class
members
Human Nature &
Variable Complexity
Start of Analysis
Few Variables
Consideration of just a few
variables can lead to:
• Agreement on priorities, focus
• Expedited timeframes
Tension &
Vested Interest
“Return to Sanity”
“Point of Discomfort”
Objections:
“Yes, but we’re not considering . . .”
“We seem to be in denial of how many
moving pieces there are . . . “
“This is too simplistic”
25
Many Variables
Consideration of many
variables can lead to:
• Class re-definition
• Sub-classing
• Removal of damages
categories
• Class de-certification
Objections:
“Let’s keep it simple”
“It’s too complicated”
“It’s not manageable”
Post-Dukes Cases –
Effect on Class Certification
 In re Wells Fargo Residential Mortgage Lending Discrimination
Litig., slip op., No. 3:08-md-01931-MMC (Sept. 6, 2011).
 Denied certification of claims under Fair Housing Act and Equal
Credit Opportunity Act.
 Found regression analysis allegedly showing disparate impact of
discretionary policy insufficient.
 Daubert motion denied for purposes of decision.
 But see McReynolds v. Merrill Lynch, Pierce, Fenner & Smith, Inc.,
No. 11-3639 (7th Cir., Feb. 24, 2012) (Posner, J.).
 Disparate impact
 Distinguished Dukes on the ground that an affirmative policy was
being alleged to create a disparate impact on a protected class.
 Contemplates that a single, common body of evidence would be
used to prove or disprove that the policy had a discriminatory
impact.
26
Post-Dukes Cases – Trial By
Formula Violates Due Process
 Duran v. U.S. Bank National Association, No. A125557 &




A126827 (Cal. App., Feb. 6, 2012).
Same expert (Dr. Drogin) as in Dukes.
Court used Drogin’s analysis as a model but came up with its
own simplified analysis.
Court applied “statistical” analysis to estimate the number of
employees within the class that had been misclassified for
overtime pay purposes.
Court held:
 Methodology violated due process because it denied defendant
opportunity to provide relevant evidence and individualized defenses
relating to classification of each employee.
 Methodology was flawed because sample was arbitrary.
 Sampling would have been improper even if used to calculate
damages due to the high margin of error.
27
Post-Dukes Cases – Trial By Formula
Does Not Satisfy FRCP 23
 In re Facebook, Inc. PPC Advertising Litigation, No. C
09-3043 PJH, slip op. (N.D. Cal. Apr. 13, 2012).
 Allegation that Facebook breached “cost-per-click”
agreements with advertisers by charging for “invalid”
clicks.
 Plaintiffs proposed that their experts could create a
methodology that would distinguish between valid and
invalid clicks.
 Court rejected this argument, finding that “there is no
way to conduct this type of highly specialized and
individualized analysis for each of the thousands of
advertisers in the proposed class.”
28
Part III – Practical tips on
presenting and challenging
statistics
29
Summary of how statistics are used to support
class certification
 The existence of a common practice
 A relationship between the defendant’s conduct




and some injury to class members
The total damages or other impact caused by a
practice
The percentage of people impacted by a practice.
Given a set of characteristics, the probability that a
person was impacted by a practice.
Common reliance
 Truly common reliance, e.g. “fraud on the market”
 Reliance by “most” of the class
30
Challenging Statistics
 Daubert challenges
 The expert is not qualified
 The statistical model is not sound
 The methodology is flawed
 The underlying data is unreliable
 What is the applicable Daubert standard on class
certification?
 Other challenges
 The expert opinion is not relevant to any issue to be
decided at trial.
 The opinion does not show that an issue is
susceptible to common, class-wide proof.
31
General Considerations
 Does the statistical evidence satisfy Daubert?
 Even if it satisfies Daubert standards, does it hold up to rigorous analysis?
 Does it satisfy the proponent’s burden of proof (resolution of conflicting









32
opinions) that Rule 23’s requirements are satisfied?
Does it show what it purports to show?
Are there inferential gaps in the analysis itself?
Does it leave data or factors unaccounted for?
Is circular reasoning involved, or does it purport to prove what it actually
assumes?
Are there conceptual gaps between the data or conclusions and the true
requirements of Rule 23?
Does it show that the answer to a question necessarily is the same for all
class members, or does it merely generalize that the answer might be the
same for some of them?
Does it show that all class members were similarly affected, or only that
each one might have been?
Is it consistent with governing substantive law?
Is there a conceptual gap between the evidence and the proof
requirements of substantive law?
Common Fact Patterns to Watch Out For
 Single policy or practice (Dukes)
 Does a single policy exist? (McLaughlin)
 Is there a way to prove a causal link between the policy
and some alleged harm?
 Can the causal link be resolved by reference to
common, classwide evidence.
 Mass reliance/common impact—ask whether
 legal theory is such that individual reliance is not
required (if so, still have to consider the separate
question of causation)
 Reliance question can be both proved and resolved by
reference to common evidence.
33
Common Fact Patterns to Watch Out For
(cont’d)
 Winners and losers
 Some class members are actually better off as
a result of the alleged practice.
 Subclasses may cure this problem, but problem
might be in identifying who goes in which
category.
 Trial by formula
 Statistics used to estimate percentage of class
members to whom the defendant may be liable
 This violates due process according to Duran.
 Statistics used to aggregate and apportion
damages
 No, according to dictum in Dukes, but some courts
may be more welcoming of this argument.
34
Tools for challenging statistical evidence
 Assumption vs. conclusion – Does the analysis prove a






fact to be true, or does it assume the fact is true?
Underlying data – Where does it come from? Is it
complete? Is it being interpreted correctly?
Methodology – Is it peer reviewed? Has it been
discredited?
Relevance – Does the analysis address the right issue?
Sample size – Is it big enough to be predictive?
Error rate – How accurate are the predictions?
Other logical fallacies
 Is the analysis circular?
 Are variables ignored?
35
Tips for Dealing With Experts
Collect
Data
Draw
Inferences
(optional)
Analyze
Major Strategic Considerations
 How collected? Trusted source?
 Is the method / measurement




process reliable (consistent with
repetition)?
Valid?
Recorded properly?
Categories appropriate?
What is the non-response rate
(survey)? Why?
 Can the results be
generalized?
 How are charts/graphs
presented?
 What method is used to
select the units (or scale)?
 Do analyses reach different
opinions?
 What variables were left
out?
 Did the expert answer the
right question?
 How do I estimate whatever
is missing?
Ask, “What is missing? Who would know?”
36
Common Statistical Flaws
 Illusory commonality
 When (even reliable) statistics only purport to answer a question for X or
X% of a class, or show that X or X% of a proposed class is affected,
commonality does not exist (indeed, is disproved).
 Discrimination (Dukes)
 Consumer fraud (Zyprexa, Neurontin)
 Breach of contract (e.g., timeliness of payment)
 Overlooked factors and intervening causes.
 Alternative drugs might be more expensive for some.
 Some people smoke lights for flavor or because they are “cool.”
 Circularity/Assumed Reliance
 When an econometric analysis purportedly shows that causation can be
proved on a class-wide basis through a “price effect,” the analysis may
assume reliance or causation rather than prove them.
 Erroneous assumptions
 All off-label marketing is fraudulent (legal/factual error).
 Third-party payors have similar rates of reimbursement for off-label
prescriptions (factual error).
 All class members were unaware the drug was unapproved (factual error).
37
Confidence in Confidence
Question for the
Courts: At what
point do we get to
an acceptable level
of common proof?
38
Future Issues
 Minority Report technology
 Is there a point at which “trial by formula” becomes
acceptable?
 Will an increase in societal complexity require shortcuts in
class and other aggregate litigation?
 The “Most Class Members” Problem
 Example – Proof that 80% of class members were harmed
means that 20% were NOT harmed.
 What if the proof is that nearly all class members were
harmed? Is 90% enough? What about 99%?
 Confidence and Error Rate
 Don’t confuse this with the issue of the percentage of class
members injured.
 What confidence level and error rate will become
acceptable?
39
For Further Study









40
David H. Kaye & David A. Freedman, Reference Guide on Statistics, Reference
Manual on Scientific Evidence 2d Ed. (Federal Judicial Center 1981)
(http://www.fjc.gov/public/pdf.nsf/lookup/sciman02.pdf/$file/sciman02.pdf)
Robert Ambrogi, Statistics Surge as Evidence in Trials, IMS Newsletter, BullsEye:
August 2009, (http://www.ims-expertservices.com/newsletters/aug/statisticssurge-as-evidence-in-trials-081409.asp)
Edward K. Cheng, A Practical Solution to the Reference Class Problem, 109
Colum. L. Rev. 2081 (2009)
(http://www.columbialawreview.org/assets/pdfs/109/8/Cheng.pdf)
Denise Martin, Stephanie Plancich, and Mary Elizabeth Stern, Class Certification
in Wage and Hour Litigation: What Can We Learn from Statistics? (Nera
Economic Consulting 2009)
(http://www.nera.com/extImage/PUB_Wage_Hour_Litigation_1109_final.pdf)
Dukes, plaintiff’s Expert Dr. Richard Drogin’s Statistical Report
(http://www.walmartclass.com/all_reports.html)
Dukes, class certification
(http://www.walmartclass.com/staticdata/walmartclass/classcert.pdf)
Michael O. Finkelstein and Bruce Levin, Statistics for Lawyers: Second Edition
(Springer, 2001)
Finkelstein, Michael O., Basic Concepts of Probability and Statistics in the Law
(Springer, 2009)
Olive Jean Dunn and Virginia A. Clark, Applied Statistics: Analysis of Variance
and Regression, Second Edition (John Wiley & Sons, 1987)
Thank You
 Topics covered
 Increasing importance of statistics & growth of data
 Basic statistical concepts and use in litigation
 Case studies
 Practical tips
 Questions?
41