Scientific Evidence
Michael I. Shamos, Ph.D., J.D.
Institute for Software Research
School of Computer Science
Carnegie Mellon University
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Opinion Testimony
• Many elements in proving cases are matters of
opinion:
– Was she sane when she made her will?
– Did death result from an overdose of medication?
– Does this electronic circuit make use of Plaintiff’s
trade secrets?
• How is opinion evidence presented to a court?
• Fact witnesses are highly limited in their ability to
give opinion testimony
• It usually must be given by experts
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Opinion Testimony
“If the witness is not testifying as an expert, the witness'
testimony in the form of opinions or inferences is limited
to those opinions or inferences which are
(a) rationally based on the perception of the
witness, and
(b) helpful to a clear understanding of the
witness' testimony or the determination of a fact in
issue, and
(c) not based on scientific, technical, or other
specialized knowledge”
Federal Rule of Evidence 701
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Expert Computer Opinions
• ALL REQUIRE SCIENTIFIC OR TECHNICAL
KNOWLEDGE
• Examples:
– Does this software infringe Plaintiff’s patent?
– Does this software comply with its specifications?
– Why does this computer program infringe Plaintiff’s
copyright
– Have these emails had their creation dates altered?
– Is this data mining technique a trade secret?
– Did the survey reveal if the public was likely to
believe zippo.com is associated with Zippo
Manufacturing?
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Admissible Lay Opinions
• Boy who witnessed an auto accident could testify to
the speed of a vehicle prior to the collision
• Witnesses were allowed to testify that a participant in a
bar fight was “boisterous” and “acting in an arrogant
manner”
• Homeowner could give an opinion of her home's fair
market value it was damaged, in view of her familiarity
with neighborhood real estate prices, length of her
residency and employment in the town, and
conversations with neighbors before the damage
• Lay witnesses cannot base opinions on scientific,
technical or specialized knowledge
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Frye v. United States
293 F. 1013 (App. D.C. 1923)
• Frye was convicted of second degree murder
• At his trial, the judge refused to allow expert testimony
on the “systolic blood pressure deception test.”
Experiments showed that if someone is lying, his blood
pressure goes up. (Predecessor of the lie detector test.)
• Frye took the test before trial and passed, and offered
the expert who administered the test
• On appeal: “We think the systolic blood pressure
deception test has not yet gained such standing and
scientific recognition among physiological and
psychological authorities as would justify the courts in
admitting expert testimony deduced from the discovery,
development, and experiments thus far made.”
Conviction affirmed.
Frye v. United States
• “Just when a scientific principle or discovery crosses
the line between the experimental and demonstrable
stages is difficult to define. Somewhere in this twilight
zone the evidential force of the principle must be
recognized, and while courts will go a long way in
admitting expert testimony deduced from a wellrecognized scientific principle or discovery, the thing
from which the deduction is made must be sufficiently
established to have gained general acceptance in the
particular field in which it belongs.”
Conviction affirmed.
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Expert Testimony
If scientific, technical, or other specialized knowledge will
assist the trier of fact to understand the evidence or to
determine a fact in issue, a witness qualified as an expert
by knowledge, skill, experience, training, or education,
may testify thereto in the form of an opinion or otherwise,
if:
(1) the testimony is based upon sufficient facts or data,
(2) the testimony is the product of reliable principles and
methods, AND
(3) the witness has applied the principles and methods
reliably to the facts of the case
FRE 702
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
The Daubert Test
• The Frye “general acceptance” test set the standard for
expert testimony for 70 years. The Federal Rules of
Evidence were adopted in 1975
• Did they or did they not alter the Frye test?
• This was not decided until the Daubert case in 1993
• Under Daubert, the judge makes a “preliminary
assessment of whether the reasoning or methodology
underlying the testimony is scientifically valid and of
whether that reasoning or methodology properly can be
applied to the facts in issue”
• Pennsylvania and 24 other states still use a version of
Frye.
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Daubert v. Merrell Dow
Pharmaceuticals
509 U.S. 579 (1993)
• Jason Daubert was born with birth defects which he
claimed was caused by the drug Bendectin.
• He sued the manufacturer, Merrell Dow
• Merrell Dow submitted expert testimony that that no
published scientific study demonstrated a link between
Bendectin and birth defects.
• Daubert submitted expert evidence of their own that
suggested that Bendectin could cause birth defects,
based on animal studies, pharmacological studies, and
re-analysis of other published studies, and these
methodologies had not yet gained acceptance within
the general scientific community.
Daubert v. Merrell Dow
Pharmaceuticals
• Following Frye, the District court refused to admit the
testimony of Daubert’s witness and gave judgment for
Merrell Dow
• The Ninth Circuit affirmed
• Daubert appealed to the Supreme Court
• The Court held, “The Federal Rules of Evidence, not
Frye, provide the standard for admitting expert scientific
testimony in a federal trial … ‘General acceptance" is not
a necessary precondition to the admissibility of scientific
evidence under the [Rules]’”
• Created a new set of tests for expert testimony
• VACATED the judgment and REMANDED the case
• Daubert lost again under the new standard
Daubert Tests
• Standard is whether testimony is relevant and reliable.
• Trial judge is to act as a gatekeeper to exclude
unreliable methods
• Non-exclusive reliability factors:
– Can the technique be or has it been tested?
– Has the technique been through peer review?
– Is there a known or potential error rate of using the
technique?
– Are there published standards controlling its use?
– Has the theory or technique has gained “general
acceptance” within the relevant scientific community.
(Frye)
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Mike’s Train House v. Lionel
472 F.3d 398 (6th Cir. 2006)
• Both parties make model toy trains
• Mike’s Train House sued Lionel for misappropriation of
trade secrets (over 3000 detailed train design
drawings)
• The jury awarded Mike’s over $40 million
• The judge allowed Dr. Jeffrey Stein, a mechanical
engineering professor at U. of Michigan, to testify that
Lionel’s drawings had been copied from Mike’s.
• Stein created a method just for this purpose
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Mike’s Train House v. Lionel
• Stein examined two sets of design drawings, one from
[each party], concerning ten different train models, for a
total of 162 pairs of drawings
• His evaluation was based on 21 self-selected criteria,
including the drawing’s title and the part number
assigned to the drawing.
• Stein calculated a score from 0 to 1 based on the
criteria; 0 indicated no association between the
drawings, and 1 indicated an extremely high
association … “~55% … were copied … [D]esigners
working independently to develop similar design
drawings of the same train would not develop identical
or substantially similar drawings”
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Mike’s Train House v. Lionel
• “We conclude that the district court abandoned its gatekeeping function by failing to make any findings
regarding the reliability of Stein’s testimony. The district
court allowed Stein to testify as to his opinion of Lee’s
report, but failed to make specific findings regarding the
reliability of Stein’s own technique for comparing the
design drawings.”
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Mike’s Train House v. Lionel
• “Stein created the criteria with which he compared the
design drawings; however, there is no evidence that his
methodology had ever been tested, subjected to peer
review, possessed a known or potential rate of error, or
enjoyed general acceptance. Although it is true that “in
some instances well-grounded but innovative theories
will not have been published,” … the novelty of a theory
does not shield an expert’s testimony from judicial
scrutiny.”
• REVERSED
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
U.S. v. Frabizio
(D. Mass 2006)
• Frabizio was charged with possessing child
pornography
• The prosecution must prove that the images are of real
children (not computer-generated images)
• The U.S. offered the testimony of Dr. Farid, who
claimed to have written “a mathematical programming
code capable of determining whether electronic images
of children are computer-generated or depict actual
children”
• Frabizio challenged Farid on Daubert grounds, that his
method was not reliable
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
U.S. v. Frabizio
• A demonstration was held in Court
• 30 percent of the time, Farid's program classified a real
photograph as a computer-generated image. In one
case it classified a cartoon character as real
• The U.S. withdrew Dr. Farid as a witness during the
demo
• Next, the U.S. tried to introduce the testimony of
Musheno a forensic examiner of photographic evidence
in the FBI's Forensic Audio, Video, and Image Analysis
Unit.
• Musheno visually examined the photographs and
without using any specialized equipment, concluded
that 6 of 19 images were real and 10 “appeared to be
real”
U.S. v. Frabizio
• Musheno’s “methodology is not adequate to support the
conclusions he so confidently offers; that the images
downloaded from the internet are real children or appear
to be real children. With Musheno offered as the
government's sole expert, his testimony is not adequate to
answer the question before the jury: whether images are
real or virtual. It is therefore inadmissible in its entirety.”
• Next, the U.S. tried to offer testimony of a pediatrician as
to the ages of the children, based on “anatomical and
medical features”
• Excluded because it would suggest that the images were
real.
• The U.S. went to trial with NO experts. Frabizio was
convicted.
U.S. v. Frabizio Jury Verdict Form
Expert Disclosure
• An expert witness must provide “a written report
prepared and signed by the witness.”
• “The report shall contain
– a complete statement of all opinions to be expressed
and the basis and reasons therefor;
– the data or other information considered by the
witness in forming the opinions;
– any exhibits to be used as a summary of or support
for the opinions;
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Expert Disclosure
• The report must also contain:
• “the qualifications of the witness, including a list of all
publications authored by the witness within the
preceding ten years;
• the compensation to be paid for the study and
testimony; and
• a listing of any other cases in which the witness has
testified as an expert at trial or by deposition within the
preceding four years.”
F.R.C.P §26(a)(2)
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Major Ideas
• Only experts can give scientific opinions
• Scientific and technical testimony must be screened
carefully
• Lay jurors can easily be misled by “authoritative”
witnesses
• Court must make sure “junk science” does not creep in,
so serves as a gatekeeper for testimony
• The Frye test is “generally accepted” scientific method
• The Daubert test is “relevant and reliable,” a weaker
standard
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Q&A
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Post-Daubert Cases
• Daubert II, 43 F.3d 1311 (9th Cir. 1995), on remand
– Did the expert conduct independent research or research just
for this litigation?
– If the research was done for litigation, the proponent must offer
objective, verifiable evidence of scientific validity.
– One way to show this is by showing that the research relied
upon and the analysis conducted has been subject to peer
review.
– Without peer review, testimony may be admitted if experts
explain precisely how they went about reaching their
conclusions and point to some objective source to show that
they followed a scientific method.
• Kumho Tire Company v. Carmichael, 526 U.S. 137
(1999).
– Daubert applies to all expert testimony, not just scientific
testimony.
Post-Daubert Cases
• Clausen v. M/V New Carissa, 339 F.3d 1049 (9th Cir.
2003).
• (Clausen owned oyster beds off Oregon. He sued the
ship New Carissa, which spilled oil, which he alleged
killed his oyster beds. Experts testified to causation.)
• Has the expert adequately accounted for obvious
alternative explanations?
• The expert cannot “utterly fail” to explain why an
alternative cause was ruled out.
• The expert’s reasons for ruling out an alternative
explanation cannot be based upon subjective beliefs or
unsupported speculation.
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Cmwlth. of Pennsylvania v. Serge
896 A.2d 1170 (Pa. 2006), cert. den. (2006)
• Serge was sentenced to life for first degree murder
• At trial, the state’s expert was permitted to show a
computer animation (CGA) of the state’s theory of the
crime
• Court required the state to authenticate the animation
as both a fair and accurate depiction of expert
reconstructive testimony and exclude any inflammatory
features that may cause unfair prejudice.
• Court required the state to provide Serge with the CGA
before trial.
LAW OF COMPUTER TECHNOLOGY
FALL 2015
© 2015 MICHAEL I. SHAMOS
Cmwlth. of Pennsylvania v. Serge
896 A.2d 1170 (Pa. 2006), cert. den. (2006)
• The CGA showed the state’s theory based on the
forensic evidence: how Serge shot his wife first in the
lower back then through the heart as she knelt on the
living room floor of their home.
• CGA showed the location of Serge and his wife in the
living room, positions of their bodies, and the sequence,
path, trajectory, and impact sites of the bullets
• Court instructed the jury that the CGA was a purely
demonstrative exhibit, not substantive evidence, and it
was being offered solely as an illustration of the state’s
version of the events
• Court told the jury that not to confuse art with reality
and not to view the CGA as a definitive recreation