Ch 9 Methodology
Research Design
What is the research question?
 (What is the hypothesis?)
 Who are your population?
 Time and Budget
 Define types of the research
1.Quantitative Research
2.Qualitative Research
 Looking for the exist instrument or establishing a new
instrument
 Data analysis
 Plan the research procure

Research Design (Continued)
Get the permission
 Do the survey
 Collect data
 Analyze the data
 Final report

Quantitative Research Design

Types of Quantitative Research
1. Survey Research Designs
-- Cross Sectional Research Designs
-- Longitudinal Studies
2. Experimental Research Designs
3. Quasi Experimental Research Designs
4. Causal-Comparative Research Designs
5. Correlational Research Designs
Survey Research

Major Characteristics
-- Information is collected from a group of people in
order to describe some aspects or characteristics of the
population of which the group is a part
-- Information is collected through asking questions; the
answers to these questions by the members of the group
constitute the data of the study
-- Information is collected from a sample rather than
from every member of the population
Survey Research (Continued 1)


Purpose of Survey Research
-- To describe the characteristics of a population
-- To determine how a population distributes itself on
one or more variables
Types of Surveys
1. Cross sectional
-- Information that is drawn from a predetermined
population
-- Information is collected at one point in time
-- A census would be typical
Survey Research (Continued 2)
2. Longitudinal
-- Information is collected at different points in time
to determine changes over a period of time
-- Trend studies would be typical-changing
population surveyed at different times
-- Panel study-survey same population at different
times
during the course of the study
Survey Research (Continued 3)

Steps in Survey Research
1. Define the problem
-- What is it the researcher wants to find out
-- Establish research questions
-- Establish research hypothesis (es)
2. Identify the target population
-- Sampling will be discussed later
-- Sampling will be based on your research problem
and what population can best attest to your problem
Survey Research (Continued 4)
3. Data gathering
-- Direct administration: Need access to the entire
sample
or population
Need time and /or facilities
a. Mail survey
-- Mailed to participants
-- Request to complete surveys
-- Request to return surveys
-- Consent forms
-- Confidentiality
-- Return rate
Survey Research (Continued 5)
b. E-Mail / Web surveys
c. Telephone survey
7. Personal interviews
Advantages and Disadvantages of Survey Data-Collection
Methods
Direct Administration
Telephone
Mail
Interview
Comparative cost
Lowest
Same
Same
High
Facilities needed?
Yes
No
No
Yes
Require training of questioner?
Yes
Yes
No
Yes
Data collection time
Shortest
Short
Longer
Longest
Response rate
Very high
Good
Poorest
Very high
Group administration possible?
Yes
No
No
Yes
Allow for random sampling?
Possibly
Yes
Yes
Yes
Require literate sample?
Yes
No
Yes
No
Permit follow-up questions?
No
Yes
No
Yes
Encourage response to sensitive
topics?
Somewhat
Somewhat
Easy
Hardest
Standardization of responses
Easy
Somewhat
Easy
Hardest
Advantages and Disadvantages of Closed-Ended vs. OpenEnded Questions
Advantages






Enhances consistency of
response across respondents
Easier and faster to tabulate
More popular with
respondents
Allows more freedom of
response
Easier to construct
Permits follow-up by
interviewer
Disadvantages






May limit breadth of
responses
Takes more time to construct
Requires more questions to
cover the research topic
Responses tend to be
inconsistent in length and
content across respondents
Both questions and responses
subject to misinterpretation
Hard to tabulate and
synthesize
Preparing the Instrument

Questionnaire
1. Appearance
2. Questions clear
--- Is this a question that can be asked exactly the way
it
written?
--- Is this a question that will mean the same thing to
everybody?
--- Is this a question that people can answer?
--- Is this a question that people will be willing to
answer,
given the data collection procedures?
--- Pilot test
Preparing the Instrument (Continued 1)

Types of questions
1. Open-ended
Used if you want an individuals true feelings
Gives people an opportunity to individualize
responses
2. Close- ended
Respondents can select responses from a number of
options
Questions need to be unambiguous
Keep the focus as simple as possible
Keep questions short
Preparing the Instrument (Continued 2)
2. Close- ended
Use language the respondents will be familiar with
Avoid using terms that might bias responses
Avoid leading questions
Avoid double negatives
3. Pilot Test
4. Format and cover letter
There are several different sources that will assist
the researcher to format their questionnaire and cover
letter
Preparing the Instrument (Continued 3)
5. Nonresponse
Send reminders
Can create an additional sample until desired
number is
reached
May have to discard surveys if there are missed
questions
6. Obtain demographics that will be pertinent to the
study
Causal-Comparative Research

Definition
1. An attempt to determine the cause or consequence of
differences that already exist between two or more
groups of individuals
2. Sometimes called correlational research
--- Seeks to explore relationships among variables
--- Identifies variables for further research
--- Differs in that it compare groups rather than
scores
--- Differs as it usually has group members rather
than
quantitative variables
Causal-Comparative Research (Continued 1)
3. Used to study differences between groups (malefemale)
4. Looks for noted differences between groups and
causes or consequences of the difference
5. Can be used as a substitute for experimental research
the only difference would be that in experimental
research the independent variable is manipulated by a
treatment
6. This is non-experimental research
7. Describe current conditions (cancer)
8. Identify or explore the past for causes of the a current
condition (smoking)
Examples of the basic Causal-Comparative
Design
(a)
Group
Independent Variable
Dependent Variable
I
C
O
Giving a little gift
Level of customer satisfaction
(-C)
O
Not giving a little gift
Level of customer satisfaction
Group
Independent Variable
Dependent Variable
I
C1
O
Counselors
Amount of job satisfaction
C2
O
Teachers
Amount of job satisfaction
II
(b)
II
Steps Involved in Causal-Comparative
Research
1.
Problem
* Establish the phenomena and cause or consequences
* Once established become more specific
* Develop hypothesis (es) by comparing the variables
with
the causes that have been developed through
your
research problem
* How does one variable effect the other (independent
variable’s effect on the dependent variables)
Steps Involved in Causal-Comparative
Research ( Continued 1)
2. Sample
Define the characteristic to be studied with the idea of
selecting a group that would be differing in regards to
the characteristic but similar with one characteristic
(male-female) (9th grade-11th grade)
3. Instrument
Open-end
4. Design
Select two or more groups that differ on at least one
variable but may be similar in another
Steps Involved in Causal-Comparative
Research ( Continued 2)
5. One group possesses a characteristic (criteria) the other
group don’
Investigate a categorical variable vs. a quantitative
variable (group vs. score)
6. Data analysis
* Compare the differences in means using t-test or
analysis of
covariance
* Use frequency polygons
* Just report the cause and the effect do not try to prove
causes
or effects
Correlational Research

Definition
1. The relationships among two or more variables are
studied without any influence or treatment
2. It describes an existing relationship between variables
3. Scores within a certain range on one variable are
associated with scores within a certain range on the
other variable
Correlational Research (Continued 1)

Purposes
1. Explain important human behaviors
To clarify our understanding of important phenomena by
identifying relations between variables
2. Prediction studies
* Based on a high relationship between two variables it
becomes possible to predict a score on either variable if the score on
the other variable is known
* Scattergram
-- Ordinate-vertical axis
-- Abscissa-horizontal axis
-- Regression line-line that comes closest to all scores depicted may
reflect a perfect correlation
3. There are many complex correlational techniques
Correlational Research (Continued 2)

Basic Steps in Correlational Research
1. Problem Selection
* The relationship two or more variables
* The differences between two or more variables
2. Sample
* Represent the research variables
* Should be selected randomly
* Need 30 or more participants for meaningful results
3. Instruments
* Need to measure two or more variables
* Need to yield qualitative data
Correlational Research (Continued 3)
4. Design
* Two scores are obtained from each participant
* One score for each variable
5. Data analysis
* Correlation coefficient is produced
* A decimal between 0.00 and 1.0
* The closer to 1.0 the coefficient establishes a stronger relationship
* A correlation 3.5 show only a slight relationship between variables
* Correlations between .40 and .60 may have theoretical or practical
value
* Correlations of .65 or higher one can make reasonable predictions
* Correlations over .85 have a strong relationship between variables
Experimental Research

What is it?
* Research that directly attempts to influence a particular
variable
* Looks at the effect of an independent variable on one or
dependent variables
* Independent variable is the experimental or treatment
variable
* Dependent variable is the criterion or outcome variable
* Researchers manipulate the independent variable
--- Researcher decide on the treatment
What is going to happen to the subject?
To whom is the treatment going to be applied
To what extent is the treatment going to be applied
more
Experimental Research (Continued 1)
* Comparison of groups
Experimental group receives the treatment
Control group receives no treatment
Comparison group receives a different treatment

Randomization
* Random selection: every member of the population
has an equal chance of being selected to be a member
of the sample
* Random assignment: any of the participants has an
equal chance of being assigned to any of the groups
Takes place before the experiment takes place
It’s a process of assigning not just a distribution
Experimental Research (Continued 2)
The groups should be equivalent (as much as human being can
be)
Has the possibility of eliminating extraneous variables that
might
effect the outcome of the study by controlling for
the characteristics
that might have an impact
* Overcoming extraneous variables
--- The researcher has more control over the research
than in
other research designs
a. Researcher determines the treatment
b. Researcher selects the sample
Experimental Research (Continued 3)
* Overcoming extraneous variables
--- The researcher determines the treatment
--- The researcher selects the sample
--- The researcher assigns individuals to groups
--- The researcher decides which group will get the
treatment, controls and different treatment
--- The researcher controls for outside factors that
might
influence the study
a. The researcher observes or measures the effect of the
treatment on the groups when the treatment is completed
Experimental Research (Continued 4)
--- Ways the researcher can minimize threats due to
subject characteristics
a. Randomization-assume individuals within a group will have
different characteristics
b. Control variable by removing it from the study
c. Build needed variable into the study (gender, age, culture,
etc.)
d. Match characteristics into the different groups
e. Control subjects by giving them each specific treatments
f. Control groups by grouping into certain variables based on
pre-test,
pilot test
Experimental Research (Continued 5)
* Weak experimental designs
--- One shot study: single group is exposed to the treatment
and dependent variable is measured without any controls
--- One group pretest-posttest: measured before and after
treatment to determine if there is any change in the
dependent variable
--- Static group comparison: pre-formed groups compared
based on different treatments
--- Example of a one-group pretest-posttest design
Pretest: Twenty-item attitude scale completed by employees
(Dependent variable)
Treatment: Ten weeks of counseling
Post-test: Twenty-item attitude scale completed by employees
(Dependent variable)
Experimental Research (Continued 6)
--- Example of a one-shot case
study design
X
○
New textbook
Attitude scale to measure interest
(Dependent variable)
--- Example of a static-group
comparison design
X1
○
New rules
Attitude scale to measure interest
X2
○
Old rules
Attitude scale to measure interest
Experimental Research (Continued 7)

True experimental designs
* Subjects are randomly assigned to treatment groups
* Randomized posttest only control group design
--- One group received experimental treatment the
other is
the control group, both groups are post
tested on a dependent variable
* Randomized pretest-posttest control group design
--- Measured at the pretest and measured again at
posttest to determine whether there has been any
change in the
dependent variable
Experimental Research (Continued 8)

Randomized Solomon four group design
--- Two groups are pretested and two are not
--- One of the pretested and one of the non-pretested
groups are exposed to treatment
--- Two of the groups are the control groups
--- All four are posttested
Example of a Randomized Posttest-Only Control Group
Design
R
X1
O
Random
assignment of 50
employees to
experimental
group
Treatment:
sensitivity
training
workshops
Posttest: Faculty
morale
questionnaire
(Dependent
variable)
100 employees
randomly
selected
R
X2
O
Random
assignment of 50
employees to
control group
No treatment: Do
not receive
sensitivity
training
Posttest: Faculty
morale
questionnaire
(Dependent
variable)
Example of a Randomized Pretest-Posttest Control
Group Design
R
O
X1
O
Random
assignment of 50
employees to
experimental group
Pretest: Faculty
morale
questionnaire
Treatment:
sensitivity training
workshops
Posttest:
Faculty
morale
questionnair
e
(Dependent
variable)
(Dependent
variable)
100 employees
randomly
selected
R
O
X2
O
Random
assignment of 50
employees to
control group
Pretest: Faculty
morale
questionnaire
Treatment:
Workshops that do
not included
sensitivity training
Posttest:
Faculty
morale
questionnair
e
(Dependent
variable)
(Dependent
variable)
Experimental Research (Continued 9)
* Random assignment with matching
--- Attempt to increase the likelihood that the groups of
subjects in an experiment will be equivalent,
participants
would be matched on certain
variables to insure equivalence on these variables
--- The variables matched will be determined by the
researcher
a. Mechanical matching
Pairing two persons based on similar scores on a
particular variable
If there is no match some subjects could be
eliminated
from the study
Experimental Research (Continued 10)
b. Statistical matching
Based on a correlation a subject is given a
predictable
score on a dependent variable with
the variable the
subject is being matched
Matches could occur based on pretest scores
Quasi Experimental designs


Does not include the use of random assignment but other
control techniques
Matching only design
* Subjects are matched in the experimental and control groups
based on
certain variables
* There are no assurances they are equivalent on other variables
* Correlations used need to be significant (above 0.04)

Counterbalanced designs
* Another way of equating experimental and control groups
* Each group is exposed to a treatment, but different groups will
receive
treatments at different times
Quasi Experimental designs (Continued 1)


Time series designs
* Measurements or observations are at different times
before and after treatment, whether pretest or posttest
Factor designs
* Extend the number of relationships
* Study the interaction of the independent variable with
more than one moderator variable (treatment or
characteristic variable)
* Uses a factor analysis statistic