Module 2
Elaine A. Borawski, PhD
Module Overview
 Defining research and evaluation
 What makes a study “research”
 Roots of participatory research
 Applied/basic
 Qualitative/quantitative
 Program Evaluation Example
How do respond when you hear
the word research or evaluation?
3
 Academics use the word research to describe what
they do
 More often than not professional people working in
community organizations/health departments say
“We don’t do research,” or “Yes we collect data, but
it’s not research.”
 Is it the different between research and evaluation?
 Or is it the difference between research and non-
research.
 Different views.
Research vs. Non-Research
 Sometimes federal grants will say “application cannot
include research”
 What does this mean?
 Different distinction than research vs. evaluation, as
evaluation can be considered research
 CDC document (handout)
CDC Definition of Research
The regulations state that "research means a
systematic investigation, including research
development, testing and evaluation, designed to
develop or contribute to generalizable knowledge"
(45 CFR 46.102(d).
Is it the Methods
That Makes it Research?
 Obtaining and analyzing data are essential to the usual




practice of public health.
For many public health practice activities, data are
systematically collected and analyzed.
Scientific methods are used in both public health research
as well as public health practice activities.
Knowledge is generated in both cases. Furthermore, the
extent to which knowledge is generalizable might not differ
greatly in research and non-research.
Thus, non-research and research activities cannot be
easily defined by the methods they employ.
Federal definitions of research
 The major difference between research and non-
research lies in the purpose of the activity.
 The purpose of research is to generate or contribute to
generalizable knowledge.
 The purpose of non-research (in public health) is to
prevent or control disease or injury and improve
health, or to improve a public health program or
service.
Federal definitions of research
 However, knowledge might be gained in any public
health endeavor designed to prevent disease or injury
or to improve a program or service.
 In some cases, that knowledge might be generalizable,
but the purpose of the endeavor is to benefit clients
participating in a public health program or a
population by controlling a health problem in the
population from which the information is gathered.
Furthermore
 Other attributes, such as publication of findings,
statutory authority, methodological design, selection
of participants, and hypothesis testing or generating,
do not differentiate research from non-research,
because these types of attributes can be shared by both
research and non-research activities.
CDC Definition of Research
The regulations state that "research means a
systematic investigation, including research
development, testing and evaluation, designed to
develop or contribute to generalizable knowledge"
(45 CFR 46.102(d).
Program Evaluation:
Non-Research Examples
Evaluation of School-based HIV Prevention Program - As part of the
evaluation of the school-based HIV prevention program in Denver public
schools, principals, teachers, student contact staff, students, and parents
were interviewed. HIV program efforts in policy awareness, staff
development, curriculum implementation, and status of students
receiving HIV prevention education were assessed.
The purpose (primary intent) of the program evaluation was to provide
information to Denver public schools that will be used to improve their
school-based HIV prevention programs. The results from the evaluation
were used to assess the success of the interventions in a specific population
(Denver public school children) and to refine the interventions in that
population.
Program Evaluation:
Non-Research Examples
IMPACT Progress Reports - The Office on Smoking and Health awarded 32 states
and the District of Columbia health departments cooperative agreements to build
capacity to conduct tobacco use prevention and control programs. These cooperative
agreements are part of CDC’s nationwide effort to establish comprehensive,
coordinated tobacco use prevention programs. Evaluation of IMPACT is comprised
of awardees submitting semi-annual progress reports. Information in the
evaluation includes staffing, coalition composition and efforts, status of a
state tobacco control plan, development of a resource center, training efforts,
community outreach and mobilization, and participation in CDC national
campaigns.
The primary intent of these state tobacco control program evaluations is to assess
the success of the intervention activities within each state. The information
gained from the evaluation is used to refine the interventions in that state. In
addition, the information is used nationally to evaluate the success of the IMPACT
program.
Program Evaluation:
Research Examples
 Evaluation of Community Based Organization
Intervention to Reduce Sexually Transmitted Disease
(STD) Rates Among STD Patients in Miami - Male STD
Patients were randomized to either the standard HIV prevention
counseling or intensive counseling comprised of four sessions of
HIV counseling from a community based organization. STD
clinic records were reviewed to determine whether there was
a difference in return rates with new STDs between the
groups. The objective of intervention and evaluation is to
determine whether intensive counseling reduces the
acquisition of new STDs among high risk people attending
a STD clinic. The purpose of the project was to evaluate a new
intervention for reducing the transmission of STDs. Knowledge
gained from this evaluation would be used to generalize to other
sites.
Program Evaluation:
Research Examples
 A Comprehensive Evaluation for Project DIRECT (Diabetes
Intervention: Reaching and Educating Communities Together) Project DIRECT is a community diabetes demonstration project
targeting African American adults residing in Raleigh, North Carolina.
The project is three-tiered and addresses diabetes care, community
screening for persons at high risk for developing diabetes, and
population based approaches to increase physical activity and reduce
dietary fat intake (two risk factors for diabetes). The goals of the
community project are to reduce preventable complications of diabetes
via a health systems approach, increase the proportion of persons at
risk for diabetes who are screened, and increase the proportion who
participate in regular vigorous physical activity and eat a reduced fat
diet. Baseline and follow-up population-based surveys are
planned to evaluate the community intervention. The purpose of
this project is to evaluate new and innovative interventions to
prevent diabetes and its complications. Knowledge gained from this
project will be used to develop similar intervention projects in
other communities.
Can (or when) Non-Research
Become Research?
 A non-research activity can develop or contribute to
generalizable knowledge after the project is undertaken
even though generating this knowledge was not part of the
original purpose.
 In this case, because the purpose was not to develop or
contribute to generalizable knowledge, the project is not
classified as research at the outset.
 However, if subsequent analysis of identifiable private
information is undertaken to develop or contribute to
generalizable knowledge, the analysis constitutes human
research that now requires further consideration under 45
CFR part 46.
Your Project Ideas….
Research or Non-Research?
Development of
Community Engaged Research
 Kurt Lewin 1940s
 Advocated using research cycle (planning, action,
investigating the results of the action) to solve
practical problems
 Lewin rejected positivist belief that data derived from
sensory experience, and the logical and mathematical
analyses of the data are the exclusive source of
authoritative knowledge.
 Participatory research links science of research and
social activism (community identified issue, research
process leading to change)
 Participatory research serves an identified need or
problem
 Applied research
driven by organization/community
identified need or problem
 Basic Research
driven by researcher interests
no immediate practical application
 Belief among traditional researchers that basic
research is: more objective
more generalizable
more rigorous
 Supports notion that applied research not “real
research”
 Drive for objectivity lead to the double blind drug trial
–neither physician or participant knows if they are
receiving the drug or placebo – belief that the data
would be more accurate
 Now argued that distancing the researcher from
people that are being researched increased distrust
and therefore decreased the accuracy of the data
gathered
 Generalizability – assumed in basic research and
statistical studies involving large data bases
 Applied research may not be so generalizable –
findings from one community may not be applicable to
another – cultural differences and barriers
Internal vs. External Validity
 Internal Validity: exists if the observed results of a
study are real and not caused by extraneous factors
(how well/controlled/rigorous) the study is.
 External Validity: the ability to generalize the study
results to other groups and settings beyond those in
the current study.
 Community-engaged research greatly increases the
external validity of a study – but are the threats to
internal validity too high (study limitations too many,
rigor not there?)
Qualitative Research
 Qualitative research is a method of inquiry employed
in many different academic disciplines, traditionally in
the social sciences.
 Qualitative researchers aim to gather an in-depth
understanding of human behavior and the reasons
that govern such behavior.
 The qualitative method investigates the why and how
of decision making, not just what, where, when.
 Hence, smaller but focused samples are more often
needed than large samples.
 In the conventional view, qualitative methods produce
information only on the particular cases studied, and
any more general conclusions are only propositions
(informed assertions).
 Quantitative methods can then be used to seek
empirical support for such research hypotheses.
 This has led to the more contemporary approach
called Mixed Methods Research.
Quantitative Research
 Quantitative Research refers to the systematic
empirical investigation of social phenomena via
statistical, mathematical or computational techniques.
 Quantitative data is any data that is in numerical form
such as statistics, percentages.
 Quantitative researchers ask a specific, narrow
question and collect numerical data from participants
to answer the question.
 The researcher conducts a statistical analysis of the
data the results of which may be generalizable to a
larger population.
 Large scale studies may suggest cause and effect
relationships that can then be tested in real world
settings
 Community organizations wanting to identify the
cause of a community problem may start with the
results of an extensive research project (local data sets)
before designing a more focused research project
Why Evaluate?
 Improve existing programs
 Measure effectiveness
 Demonstrate accountability
 Share effective strategies and lessons learned
 Ensure funding and sustainability
 Evaluation is a tool that can both measure and
contribute to the success of your program
30
 In community settings research is often project based
– lead into We Run This City (marathon) presentation