
By Using the Scientific Process to Evaluate Your
Programs, You Avoid the Following Errors:
Overgeneralizations
 Selective Observation
 Illogical Reasoning
 Resistance to Change
 Adherence to Authority


Overgeneralization


Selective Observation


Jumping to conclusions based on illogical reasoning
Adherence to Authority


Choosing to look only at things that are in line with our preferences/beliefs
Illogical Reasoning


Assuming that what is true for one or two cases is true for all
Believing the person making the claim has the knowledge
Resistance to Change

Reluctance to change our ideas in light of new information
 Ego based commitment
 Devotion to tradition

Overgeneralization – She observed only two cats – both of whom
were previously confined indoors. On the basis of these two
observations she generalized to all cats.
Dear Ann,
I just moved my two cats to a house in the country. When I lived in
the city, I felt guilty because I never let my cats outside. When
we arrived in the country I noticed several cats outside, so I
threw open my back door to let my cats roam – free at last! My
cats cautiously went to the door and looked outside for a while
and then returned to the living room and lay down. I’ve
realized that people should not feel guilty if they keep their cats
inside – even when cats have the opportunity, they don’t want to
play outside.

Selective Observation – She observed the cats only once – and
focused only on her cats.
Dear Ann,
I just moved my two cats to a house in the country. When I lived in
the city, I felt guilty because I never let my cats outside. When
we arrived in the country I noticed several cats outside, so I
threw open my back door to let my cats roam – free at last! My
cats cautiously went to the door and looked outside for a while
and then returned to the living room and lay down. I’ve
realized that people should not feel guilty if they keep their cats
inside – even when cats have the opportunity, they don’t want to
play outside.

Illogical Reasoning – She assumed that others feel guilty about
keeping cats indoors, and that cats are motivated by emotions.
Dear Ann,
I just moved my two cats to a house in the country. When I lived in
the city, I felt guilty because I never let my cats outside. When
we arrived in the country I noticed several cats outside, so I
threw open my back door to let my cats roam – free at last! My
cats cautiously went to the door and looked outside for a while
and then returned to the living room and lay down. I’ve
realized that people should not feel guilty if they keep their cats
inside – even when cats have the opportunity, they don’t want to
play outside.

Resistance to Change – She was quick to conclude that she had no
need to change her approach to cats.
Dear Ann,
I just moved my two cats to a house in the country. When I lived in
the city, I felt guilty because I never let my cats outside. When
we arrived in the country I noticed several cats outside, so I
threw open my back door to let my cats roam – free at last! My
cats cautiously went to the door and looked outside for a while
and then returned to the living room and lay down. I’ve
realized that people should not feel guilty if they keep their cats
inside – even when cats have the opportunity, they don’t want to
play outside.

Adherence to Authority – She was writing to Ann as an “expert”
to validate/support her conclusion.
Dear Ann,
I just moved my two cats to a house in the country. When I lived in
the city, I felt guilty because I never let my cats outside. When
we arrived in the country I noticed several cats outside, so I
threw open my back door to let my cats roam – free at last! My
cats cautiously went to the door and looked outside for a while
and then returned to the living room and lay down. I’ve
realized that people should not feel guilty if they keep their cats
inside – even when cats have the opportunity, they don’t want to
play outside.

These same kinds of errors lead to the following
conclusions:

Affirmative action results in reverse
discrimination

People are poor because they are lazy

Politically conservative people are prejudiced





Overgeneralization – Because they read about one or two cases in the newspaper, they
believe it is common
Selective Observation – They focus only on these examples and not on evidence that
could discount their claims
Illogical Reasoning – They assume that affirmative action legislation is the problem and
not erroneous implementation of the legislation by individuals
Resistance to Change – They discount empirical evidence that this statement is not true
because they had a negative personal experience related to affirmative action (i.e., a
relative who experienced reverse discrimination)
Adherence to Authority – They refer to a popular political figure who has made this
statement





Overgeneralization – Because they observed one or two people who
“took advantage of the system” they assume this is the same with all
people who are poor
Selective Perception – They focus only on those who are lazy and don’t
consider all of the working poor
Illogical Reasoning – They assume that there are jobs available for people
who aren’t working
Resistance to Change – They haven’t been generous to people in poverty,
so they want to believe they deserved it. OR they have been raised to
believe you can “be anything you want to be”
Adherence to Authority – They refer to popular political leaders who
have made this claim



Overgeneralization – May know one or several people who are
prejudiced and conservative
Selective Perception – Discount evidence that conservative people may be
very involved in volunteer work, etc. Only focus the on fact that they
don’t vote to support social programs
Illogical Reasoning – Don’t realize that people can have some prejudices,
but not others



Also assume that lack of support of social programs means they are prejudiced.
Resistance to Change – Have made disparaging remarks in past, and so
don’t want to admit they, themselves, may be prejudiced – prejudiced
against conservative people
Adherence to Authority – Refer to political leaders or people in Social
Work profession who have made this claim

Know the research!

Make claims based on empirical evidence

Know how to evaluate information/research


Distinguish good research from poor research
Avoid “never” and “always”
Understand that research is based on the idea that we claim
to know what MOST people will do MOST of the time!
 Recognize there will be exceptions and realize behavior is
generally explained by multiple factors


Overgeneralization


Selective Observation


Jumping to conclusions based on illogical reasoning
Adherence to Authority


Choosing to look only at things that are in line with our preferences/beliefs
Illogical Reasoning


Assuming that what is true for one or two cases is true for all
Believing the person making the claim has the knowledge
Resistance to Change

Reluctance to change our ideas in light of new information
 Ego based commitment
 Devotion to tradition

Use random sample to study individuals/groups

Use explicit criteria for determining cause

Measure and sample phenomena systematically


Use scientific method
Use evidence that can be examined and critiqued by
others (repetition)



Every case in the population must have an
equal probability of being selected for study
group
Sample groups must be of adequate size
If you do not have a random sample then you
can determine what CAN happen, but not
what TENDS to happen

X and Y must be related



Change in independent or causal variable will mean
change in the dependent or effect variable
X (independent variable) must come before Y
(dependent variable)
There is no Z variable (other factor) explaining
the relationship between X and Y

Example 1
X is gender (male versus female)
 Y is grade point average
 Mean grade point average is higher for females than
males
 Thus gender and GPA are related


Example 1
X is amount of time student studies
 Y is score on exams
 Students who study more have higher scores on
exams
 Thus amount of time students study (x) and scores
on exams (y) are related


Which comes first….

The chicken or the egg

Watching violence on TV or being violent

Education or income

Testosterone or aggression

Is damage caused by firemen?

Is shoplifting caused by large hands?

Does race explain criminal behavior?

Does height influence IQ?




As the number of firemen increases so does the
amount of damage
People with larger hands are more likely to
shoplift
African American males are more likely to commit
crimes than Anglo American males
Taller second graders have higher IQ scores than
shorter second graders

The relationship between the amount of damage and
the number of firemen is explained by size of fire-These two are related because they are both related to
size of fire



As size of fire increases so does amount of damage
As size of fire increases so does number of firemen called
The relationship between hand size and the monetary
value of things that are shoplifted is explained by
gender

Men have larger hands than women and men are more likely
than women to shoplift items of greater monetary value.


The relationship between race and criminal
behavior is explained by socioeconomic factors
(income, family structure, etc.)
The relationship between height and IQ is
explained by their relationship to age

In second grade, older kids are taller and older kids
have higher IQs

Identify a research question and do a literature search

Identify and operationalize concepts


Identify population and draw sample


How you want to study them/it
Analyze data


Who you want to study
Collect data


What you want to study
How you will study them/it
Write a report and present information

Carefully document this process so someone else could repeat it—The more time an
experiment is repeated, the more valid and reliable it becomes



Using the best current research evidence allows us to assess:
 Accuracy of assessment tools
 Effectiveness of different interventions
Research helps program developers identify:
 Client values (i.e., prefer individual vs. group
interventions)
 Community values (i.e., understanding of what
interventions are acceptable)
 Client Circumstances (i.e., voluntary or court ordered)
Research expertise results in program expertise
 Skilled extension agents can integrate past experience and
skills with “sound” empirical evidence
Extension workers can use research:

To influence policy makers



Guide policy development
Influence distribution of resources
To assess intervention strategies

Make judgments about appropriate treatments

To implement and evaluate programs

To identify characteristics of current clients

To communicate with clients – give results

Descriptive Research


Exploratory Research


Explore unique program situation(s)
Explanatory Research


Describe the sample, program, etc.
Explain WHY people behave in certain ways under
certain conditions
Evaluation Research

Evaluate current programs


Engel, Rafael J. and Russell K. Schutt—The
Practice of Research in Social Work
Dr. Carol Albrecht



Assessment Specialist USU Ext
carol.albrecht@usu.edu
(979) 777-2421