Developed through the APTR Initiative to Enhance Prevention and Population
Health Education in collaboration with the Brody School of Medicine at East
Carolina University with funding from the Centers for Disease Control and
Prevention
APTR wishes to acknowledge the following individuals that
developed this module:

Anna Zendell, PhD, MSW
Center for Public Health Continuing Education
University at Albany School of Public Health

Joseph Nicholas, MD, MPH
University of Rochester School of Medicine

Mary Applegate, MD, MPH
University at Albany School of Public Health

Cheryl Reeves, MS, MLS
Center for Public Health Continuing Education
University at Albany School of Public Health
This education module is made possible through the Centers for Disease Control and Prevention (CDC) and the
Association for Prevention Teaching and Research (APTR) Cooperative Agreement, No. 5U50CD300860. The module
represents the opinions of the author(s) and does not necessarily represent the views of the Centers for Disease
Control and Prevention or the Association for Prevention Teaching and Research.
Define screening and identify appropriate conditions
for screening
2. Evaluate screening tests in terms of their validity,
results and generalizability
3. Evaluate the effectiveness of a screening program and
discuss the common biases
4. Discuss ethical considerations in screening
1.

As you watch this clip and complete the module, think
about the implications for patient screening based on
this technology

Medical concerns?

Ethical considerations?

Access issues?

Informed decision-making after screening?
http://www.youtube.com/watch?v=6hlMlbmcSHg

Share common goals
 Enhance quality of life of patients
▪ Health promotion
▪ Disease and injury prevention

Preventive medicine promotes these goals at
the individual and population levels, while
public health focuses on populations.
Tertiary
Prevention
Secondary
Prevention
Primary
Prevention
McKenzie et al.: 2008

Presumptive identification of an unrecognized
disease through tests, examinations, or other
procedures which can be applied rapidly

Screening tests sort out apparently well
persons who probably have a disease from
those who probably do not.

Early detection
 Leads to early treatment
 Can lead to a decrease in morbidity and mortality
 Can break the chain of transmission and development of
new cases
 Is often cost-effective

The human body is continually changing
Jekel et al:, 1996; McKenzie et al:, 2008; Londrigan &
Lewenson: 2011

Screening starts before diagnosis
 History questions
 Physical exam findings
 Lab tests
 Pre-test probability

Results of screening trigger diagnostic work-up and
preventive interventions
Jekel et al:, 1996; McKenzie et al:, 2008
Screening Test
≠
Diagnostic
Test
Screening Test
Identifies
asymptomatic
people who may
have a disease
≠
Diagnostic
Test
Screening Test
Identifies
asymptomatic
people who may
have a disease
≠
Diagnostic
Test
Determines
presence or
absence of
disease when
patient shows
signs or
symptoms






Simple
Rapid
Inexpensive
Safe
Available
Acceptable

Pap smear screens for ___________________________

Fasting blood sugar screens for _________________

Fecal occult blood test screens for ______________

Blood pressure screens for ______________________

Bone densitometry screens for _________________

PSA test screens for _____________________________

PPD test screens for _____________________________

Mammography screens for ______________________
USPSTF: 2009

Pap smear screens for cervical cancer

Fasting blood sugar screens for diabetes

Fecal occult blood test screens for colorectal cancer

Blood pressure screens for hypertension

Bone densitometry screens for osteoporosis & osteopenia

PSA test screens for prostate cancer

PPD test screens for tuberculosis

Mammography screens for breast cancer.
USPSTF: 2009

Obesity

Weight, Body Mass Index

Dental caries, oral cancer

Oral examination

Drugs, Alcohol, and
Tobacco

Urine test, NMASSIST, or
Flagerstrom Tolerance
Test for Nicotine
Dependency
http://www.drugabuse.gov/NIDAMED/screening/

Standard practice
 Annual mammograms for women age 40+ years
 Start earlier if family history of breast cancer

2009 US Preventive Services Task Force (USPSTF)
recommendations
 Mammograms not universal for women age 40-50 years
 Bi-annual mammograms for women 50+ years

Cost-benefit analysis
 False positives
 Unnecessary invasive procedures


Multiple screening options

Colonoscopy – gold standard

Sigmoidoscopy

Virtual colonoscopy – CT colonoscopy

Barium enema

Fecal testing – occult blood, DNA test
Recommended age, frequency vary by test and
family history

Practice evaluation of diagnostic test characteristics
and screening programs

Discuss prevention concepts

Apply this at patient and
population level

Mandatory universal screen for disorders, including
metabolic, hormonal, hematologic, and infectious
conditions

States vary in what diseases they test for

Heel prick blood test 24-48 hours post birth - if done too
early, metabolic disease may not show up in blood

Family history may indicate need for additional screens

Reliability and validity are central concepts in
evaluating tests

Distinction between reliability and validity
 Reliability: consistency of test at different times or under
differing conditions
 Validity: how well test distinguishes between who has
disease and who does not
Fortune & Reid: 1998; Jekel et al:, 1996
VALIDITY and RELIABILITY
Fortune & Reid: 1998

Also known as consistency

Ability to yield the same results with repeated
measurements of same construct

Degree to which results are free from random error
Jekel: 1996; Al-Eisa: 2009
Intra-subject
Jekel: 1996; Al-Eisa: 2009
Intra-rater
Intra-subject
Jekel: 1996; Al-Eisa: 2009
Intra-rater
Intra-subject
Inter-rater
Jekel: 1996; Al-Eisa: 2009
Intra-rater
Intra-subject
Inter-rater
Instrument
Jekel: 1996; Al-Eisa: 2009

Measures validity of screening tests

Ability to identify those with disease correctly
 Minimizes false negatives – if test highly sensitive
 SNOUT – Sensitive test with Negative result rules
OUT disease

Ability to identify those without disease
correctly
 Minimizes false positives – if test highly specific
 SPIN – Specific test with Positive result rules IN
disease
PSA level
Sensitivity
Specificity
1.0
100
21
2.0
100
48
3.0
100
60
4.0
99
73
5.0
96
76
6.0
94
79
7.0
90
83
8.0
90
88
9.0
68
90
10.0
54
93
11.0
47
94
12.0
30
95
13.0
23
96
14.0
17
97
15.0
11
97
Morgan TO et al; NEJM, 1996
Disease Present
Disease Absent
Test +
True
Positive
False
Positive
Test -
False
Negative
True
Negative
Sensitivity
DISEASE
Present
Test +
Test -
Absent
True positive
False positive
False negative
True negative
Sensitivity=
True positives
True positives + false negatives
Specificity
DISEASE
Present
Test +
Test -
Absent
True positive
False positive
False negative
True negative
True negatives
= Specificity
True negatives + false positives

Positive predictive value

Negative predictive value

NOT inherent characteristic of a screening test

Percent of positive tests that are truly positive
 If test result is positive, what is probability that the
patient has the disease?

Is affected by several factors
 Specificity & specificity of the screening test
 Prevalence of disease

NOT inherent characteristic of a screening test

Percent of negative tests that are truly negative
 If test result is negative, what is the probability
that patient does not have the disease?

Sensitivity and specificity are constant for a
particular test

PPV and NPV vary dramatically, depending on
prevalence of target condition in population tested
Low prevalence  low PPV, high NPV
High prevalence  high PPV, low NPV
Predictive Value
100%
80%
60%
PVP
PVN
40%
20%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Prevalence
Disease
Diseased
Non-Diseased
PVs
↓
Positive
True Positive (TP)
False Positive (FP)
(Type 1 error)
TP
TP + FP
Negative
False Negative (FN)
(Type II error)
True Negative (TN)
TN
TN +FN
Sensitivity
TP
TP + FN
Specificity
TN
TN + FP
Test
Result
HIV Status
1,000 people at
Prenatal Clinic
Positive
HIV-Positive (15)
HIV-Negative (985)
1.5% Prevalence
False Positive (99)
14
14 + 99
= 12.4% PPV
False Negative (1)
True Negative (886)
886
1 + 886
= 99.9% NPV
Sensitivity (95%)
Specificity (90%)
True Positive (14)
ELISA
Result
Negative
HIV Status
1,000 people at
STD Clinic
Positive
HIV-Positive (60)
HIV-Negative (940)
6% Prevalence
False Positive (94)
57
57 + 94
= 37.75% PPV
False Negative (3)
True Negative (846)
846
3 + 846
= 99.6% NPV
Sensitivity (95%)
Specificity (90%)
True Positive (57)
ELISA
Result
Negative
HIV Status
1,000 people at
Clinic in Zambia
Positive
HIV-Positive (240)
HIV-Negative (760)
24% Prevalence
False Positive (76)
228
228 + 76
= 75% PPV
False Negative (12)
True Negative (684)
684
12 + 684
= 98.3% NPV
Sensitivity (95%)
Specificity (90%)
True Positive (228)
ELISA
Result
Negative

Use of different tests concurrently to screen for
same condition

Example: Prenatal multiple marker screening for
Down Syndrome
 Measures levels of 3 biomarkers in mother’s blood:
▪ AFP: alpha-fetoprotein, protein produced by fetus
▪ hCG: human chorionic gonadotropin, hormone produced by
placenta
▪ Estriol: a hormone produced by both fetus and placenta
 Results of ALL 3 tests increases sensitivity and specificity

Use of two-stage screening to target testing efforts

Example: Early pregnancy gestational diabetes
screening
 First trimester risk assessment—identifies women at higher
risk of gestational diabetes
 Oral Glucose Tolerance Test (OGTT) right away for those
whose first screen indicates high risk

Two-stage screening to maximize predictive value

Example: HIV screening in suburban primary care
office
 Risk assessment questionnaire about sexual and drug use
history
 HIV blood test for all patients whose questionnaire
indicates risk factors for HIV infection

Test characteristics (sensitivity & specificity) alone
are never sufficient for a sound decision about
whether to use a screening test

Other screening considerations
 Benefits vs. risks
 Prevalence of target condition
 Inconvenience
 Costs/resource expenditures
 Patient values and cultural norms
Guyatt: 2009

Assessing a screening/diagnostic test
 Properties & accuracy
 Comparison of test to “gold standard”


Most definitive diagnostic procedure or best available laboratory
test

Not always a gold standard for a procedure
USPSTF recommended

Systematically reviews the evidence of effectiveness
and develops recommendations for clinical
preventive services

Recommendations include:
 Screening tests
 Counseling
 Preventive medications
Courtesy of Diana Pettiti, USPSTF: 2010
1.
2.
3.
4.
5.
6.
Define analytic framework – outcomes & questions
Define and retrieve relevant evidence
Evaluate QUALITY of studies (good, fair, poor)
Synthesize and judge STRENGTH of overall evidence
(convincing, adequate, inadequate)
Determine BALANCE of benefits and harms
Benefits – Harms = Net Benefit
Link recommendation to judgment about net
benefits: Grades: A, B, C, D, I (inadequate evidence)
Courtesy of Diana Pettiti, USPSTF: 2010

Do the studies have the appropriate research
design to answer key questions?

Are the existing studies high quality?

Are the results of the studies applicable to the
general US primary care population and setting?
Courtesy of Diana Pettiti, USPSTF: 2010

How many relevant studies have been done?

How large are the studies?

How consistent are the results of the studies?

Are there other factors that help us assess the
certainty of the evidence? (e.g. dose-response
effects, biologic plausibility)
Courtesy of Diana Pettiti, USPSTF: 2010
Certainty of
Net Benefit
Magnitude of Net Benefit
Substantial
Moderate
High
A
B
C
D
Moderate
B
B
C
D
Low
Small Zero/negative
Insufficient (I Statement)
Courtesy of Diana Pettiti, USPSTF: 2010
Grade
A
Grade Definition
Suggestion for Practice
USPSTF recommends the service.
There is high certainty that the net benefit is substantial.
Offer or provide this service.
The USPSTF recommends the service.
B
C
There is high certainty that the net benefit is moderate or
there is moderate certainty that the net benefit is moderate to
substantial.
USPSTF recommends against routinely providing the
service. There may be considerations that support providing
the service in an individual patient.
There is moderate or high certainty that the net benefit is
small.
Offer or provide this service.
Offer or provide this service only if
there are other considerations in
support of the offering or providing the
service in an individual patient.
USPSTF recommends against the service.
D
I
There is moderate or high certainty that the service has no
net benefit or that the harms outweigh the benefits.
Discourage the use of this service.
USPSTF concludes that the current evidence is
insufficient to assess the balance of benefits and harms of
Read “Clinical Considerations” section
of USPSTF Recommendation
Statement. If offered the service,
patients should understand the
uncertainty about the balance of
benefits and harms.
the service.
Evidence is lacking, of poor quality, or conflicting, and the
balance of benefits and harms cannot be determined.
Courtesy of Diana Pettiti, USPSTF: 2010

Newer tests vs. gold standard

Best tests for your population
 Validity in your population
 Accessibility
 Cost
 Capacity of local health care system
 Need a system in place to be able to screen AND to
deal with positive results
Cervical Cancer
Liquid-based
Pap smear –
the NEW test
Patients without
evidence of
cervical cancer
Standard
Pap smear –
the GOLD
STANDARD
NO
Cervical Cancer
Cervical Cancer
NO
Cervical Cancer
1.
Are study methodology and results credible?
2.
Have sensitivity, specificity and predictive values
been calculated and reported?
3.
Is the population tested similar to my patient
population?
4.
How can I use these results in a screening program
or patient care?
5.
Does this screening improve the present state of
medical screening?

Lead time bias: over-estimation of survival rate
among screening-detected cases
 When survival is calculated from diagnosis point

Length bias: over-estimation of survival rate among
screening-detected cases
 Due to excess of slowly progressing cases among those
identified by screening
Koretz: 2009
Disease
Test
Condition
Test must be simple,
should be
safe, precise, valid.
important
health problem.
Epidemiology
of disease must
be understood.
Treatment
Cost
Programming
Must be
evidence of
effective
treatment and
that early
treatment will
lead to better
outcomes.
Evidence on
costeffectiveness
of screening
for
interventions
and outcomes.
Evidence from
randomized controlled
trials that program
effective in reducing
mortality and
morbidity.
Must be acceptable
to population and
health providers
giving test.
Benefits outweigh
physical and
psychological harms if
any.
Criteria/protocol on
next steps for
positive tests.
Protocols for
implementation and
evaluation of screening
program.

Definition: Identifying a disease that is unlikely to
impact patient over lifetime
 Prostate or breast cancer may be present in body
 May never become clinically apparent

Identifying pseudodisease is nearly impossible until
person dies from unrelated causes

Gold standard tests cannot predict future trajectory
of a condition
Durgin: 2005

Mandated screening

Genetic testing

Disparities

Creation of screening program

Newborn screening

Syphilis testing for marriage licenses

TB screening for health care workers

Drug testing for airline pilots

Need to assess costs vs. benefits at all levels
 Individual
 Societal
 Healthcare system

Pros
 Identify serious problems that could harm others OR where
immediate treatment is imperative

Cons
 Potential harm to patient autonomy
 Confidentiality concerns
 Testing low risk population reduces PPV
 Consequences of false positive tests

Learn if individual carries a gene for a disease and
might pass it on to children

Screen unborn fetus for disease

Test for genetic diseases in children or adults before
symptoms emerge

Results difficult to interpret; not always clear cut

Employment issues

Health/life insurance consequences

Added stress

Costs

Confidentiality

Ownership of DNA
Kalb, 2006

Geographic region
 Rural
 Inner city

Uninsured and underinsured
 Screening and follow-up testing/treatment
 Cost-prohibitive without health insurance

Minority and immigrant populations
 Lack of culturally competent healthcare providers
 Low health literacy

Cross-cultural differences in health literacy and
attitudes

Culturally relevant screening

Screening practices may need to be adapted
How ethical is it to:

Use a test that may tell people they have condition
when they do not?

Use a test that may tell people they do not have
condition when they actually do?

Use a test if there is no system in place to treat those
who test positive?
Truglio et al: 2011

Strategic targeting for screening
 Groups with higher prevalence – increase PPV
 Provider vs. patient – who is more likely to request?

Growing body of evidence-based medicine allows us to:
 Identify more precise screening protocols
 Weigh benefits/drawbacks of screening test

Strategic screening can be cost-effective






Screening is bedrock of secondary prevention
Screening and diagnosis are not the same
Sensitivity and specificity are characteristics of a
screening test that determine a test’s validity
Predictive values are affected by sensitivity &
specificity of test and by prevalence of the disease
Screening of high-risk populations increases positive
predictive value
Screening decisions must weigh acceptability and
applicability to practitioner, population, and
individual

Department of Public Health
Brody School of Medicine at East Carolina University

Department of Community & Family Medicine
Duke University School of Medicine
Mike Barry, CAE
Lorrie Basnight, MD
Nancy Bennett, MD, MS
Ruth Gaare Bernheim, JD, MPH
Amber Berrian, MPH
James Cawley, MPH, PA-C
Jack Dillenberg, DDS, MPH
Kristine Gebbie, RN, DrPH
Asim Jani, MD, MPH, FACP
Denise Koo, MD, MPH
Suzanne Lazorick, MD, MPH
Rika Maeshiro, MD, MPH
Dan Mareck, MD
Steve McCurdy, MD, MPH
Susan M. Meyer, PhD
Sallie Rixey, MD, MEd
Nawraz Shawir, MBBS

Sharon Hull, MD, MPH
President

Allison L. Lewis
Executive Director

O. Kent Nordvig, MEd
Project Representative