Comprehensive
Exam Review
Click the LEFT mouse key ONCE to continue
Research and
Program
Evaluation
Part 3
Click the LEFT mouse key ONCE to continue
Ethical Aspects of Research
General ethical principles include:
Nonmaleficence - do no harm
Beneficence - do as much good as possible
Justice - equality for all
Fidelity - fulfill all obligations
Ethics Related to Scholarly Work
The information derived from a research study
should promote the welfare of members of
society (Beneficence).
The principal researcher has responsibility for
all aspects of execution of the study, including
the behaviors of all research participants
(Beneficence and Fidelity).
The results of research must be reported
accurately, honestly, and fairly (Beneficence
and Nonmaleficence).
Credit for all contributions to the research
must be given accurately and adequately
(Justice).
Acknowledgment must be given to original
contributions and/or scholarly insights of
others and distinguished clearly from those of
the author(s)/researcher(s); plagiarism is
always unethical (Justice).
Ethics Related to Subjects
Researchers must strive to minimize harm
and/or risk to subjects (Nonmaleficence).
Researchers must strive to maintain or
improve subjects’ welfare and dignity
(Beneficence).
Researchers must use informed consent
procedures that are to the subjects’ benefit
(Beneficence and Fidelity).
Researchers must respect subjects’ privacy by
maintaining confidentiality and/or anonymity
(Fidelity).
Researchers must fulfill all treatment
obligations and responsibilities offered to
subjects (Fidelity).
Researchers must be extremely careful to
protect subjects’ welfare and dignity in the
conduct of experiments involving deception
(Nonmaleficence and Fidelity).
Researchers must provide subject debriefing,
including dehoaxing or desensitization, for
any research involving deception (Beneficence
and Fidelity).
Researchers must avoid use of any form of
“pressure” to coerce subject participation
(Nonmaleficence).
Researchers are obligated to attempt to
countermand any negative consequence(s) of
participation in the research (Beneficence and
Fidelity).
Research Questions
and Hypotheses
The decision of whether to use a research
question, null hypothesis, or directional
hypothesis is made on the basis of what is
known about the topic being studied.
Research questions are used when relatively
little is known about the topic (i.e., when there
is very little basis for making a conjecture
about what the results might be).
Research questions typically are in the form:
“What [is] / [are] the [differences]/
[relationships] [between] / [among] ...?”
(The “is” or “are” and “between” or “among”
decisions are made simply to achieve correct
syntax, i.e., to achieve correct grammatical
structure.)
The decision of whether to use “relationships” or
“differences” is made based on the types of
variables involved.
In general, variables studied in research can be
classified as either “discrete” or “continuous.”
In general, a “discrete” variable may be
thought of as having categories, whereas a
“continuous” variable may be thought of as
having “scores” (i.e., a full range of values).
Typical examples of discrete variables
include gender, race/ethnicity, marital
status, grade or class level, state of
residence, or diagnostic classification.
Typical examples of continuous variables
include age, years of work experience, annual
income, scores on a test, or frequency of a
particular behavior.
Use the word “relationship” in a research
question or an hypothesis if all the variables
involved are continuous.
Use the word “difference” in a research
question or an hypothesis if at least one of the
variables involved is discrete.
Examples of research questions include:
What is the relationship between levels of
assertiveness and stress among adult, working
women?
What are the differences among married and
divorced males’ and females’ attitudes toward
divorce?
What is the difference in college seniors’ GRE
scores following participation in a test
preparation workshop?
The general form of a directional hypothesis is
a declarative sentence.
An effective directional hypothesis should specify
both the nature and direction of the relationship
or the difference.
Examples of a directional hypothesis include:
Males engage in abusive verbal behavior more
frequently than females.
The “test breaker” activity is effective in
reducing students’ test anxiety.
There is a positive relationship between
frequencies of use of alcoholic beverages and
cigarettes.
In order for a general statement to be “true,”
it must hold as specified for all occasions.
If any occasion is an exception to the general
statement, then the statement is “not true” (i.e.,
“one exception disproves the rule”).
Therefore, hypotheses are often stated in “null
form” because researchers study only one
occasion at a time and thus cannot prove but
can only disprove a “null” statement.
There are two commonly used forms of null
hypotheses: the “traditional” form and the
“modern” form.
An example of a traditional null hypothesis is:
There will be no significant relationship
between counselor trainees’ personality
needs, as measured by the Personality
Research Form, and their counseling
effectiveness, as measured by the Counselor
Evaluation Rating Scale.
The “modern form” is written in the present
tense, does not use the word significant, and
does not indicate the measurement tools.
The “modern” form of the previous hypothesis
is:
There is no relationship between counselor
trainees’ personality needs and their
counseling effectiveness.
A study of differences in academic performance
on the basis of gender and residence classification
would have two major hypotheses:
H1: There is no difference in academic
performance based on gender.
H2: There is no difference in academic
performance based on residence
classification.
But there are the combinations that
need to be considered.
Residence
Female
Rural
Academic
Performance
Academic
Performance
Male
Gender
Urban
Academic
Performance
Academic
Performance
Therefore, subhypotheses need to
be added to the primary
hypothesis.
H1: There is no difference in academic
performance based on gender.
H1a: There is no difference in academic
performance among males based on
residence classification.
H1b: There is no difference in academic
performance among females based
on residence classification.
There is also the possibility that the two
variables of gender and residence classification
may somehow “interact” with one another to
yield a unique result.
Therefore, in such situations, an “interaction
hypothesis” also should be presented:
H1c: There is no gender by residence
classification interaction for academic
performance.
The evaluation response to a research question
is a declarative statement that provides a direct
answer to the question posed.
There are only two possible evaluation
responses for a null hypothesis: “reject” or
“fail to reject.”
A null hypothesis is never “accepted,”
because one study cannot “prove” the truth
of the (null) hypothesis.
There are two types of errors that can be made
in evaluating hypotheses:
A Type I Error occurs when the researcher
rejects the null hypothesis when it is in fact
true.
A Type II Error occurs when the null
hypothesis is not rejected when it should have
been rejected.
Sampling
Subjects are the people who participate in the
research.
Sampling is the procedure used to identify and
enlist the subjects.
A sample is obtained from a population, which
is the group of people to whom the results of
the study are to be applied.
Good sampling starts with effective description
of the population.
Description of the population includes consideration of:
(a) demographic characteristics and
(b) psychosocial characteristics.
Demography is the statistical study of populations (i.e., identifiable groups of people),
and demographic information comes typically
from statistical summaries (aka statistical
abstracts).
Psychosocial characteristics are those
attributes, behaviors, and characteristics
typically associated with an identifiable group
of people.
Psychosocial characteristic information comes
from the professional literature.
Generally, effective sampling works as follows.
First, the population for the study is identified,
usually using colloquial descriptors.
Next, the primary (or major) demographic
characteristics of interest are selected.
Third, demographic information about the
population is found from some statistical
resource.
Next, the psychosocial characteristics of the
population are identified from the professional
literature.
Next, sampling is done based on
demographic characteristics.
Finally, it is assumed that the psychosocial
characteristics of the sample equal those of the
population.
It should be remembered that representativeness
is the key criterion for the effectiveness of
sampling, not the sampling procedure used!
There are two categories of
approaches to sampling: probability
and nonprobability.
Probability sampling means that subjects are
drawn from the population in such a way that
the probability of selecting each member of the
population is known.
Nonprobability sampling means that subjects
are drawn from the population in a logical way
such that representativeness can be assumed
reasonably.
Probability sampling methods include:
Random Sampling, in which each person has
an equal likelihood of being selected and the
selection of one does not affect the selection of
another.
Systematic Sampling, in which every nth person
from a list of all persons in the population is
selected.
Stratified Sampling, in which each person who
is a member of the “stratum” (i.e., category) is
eligible to be selected and selection is made
through a first encounter process.
Stratified Random Sampling, in which each
member of a stratum has an equal likelihood
of being selected, selection of one within the
stratum does not affect the selection of
another, and the number of subjects drawn
within strata may be either proportional or
disproportional to the population.
Cluster Sampling, in which naturally occurring
groups or units are the population (e.g., classes
or neighborhoods), clusters of groups or units
are randomly selected from the population of
groups or units, and individuals are randomly
selected from the clusters selected.
In all probability methods, representativeness
is assumed to be achieved if the laws of
probability are effectively operationalized.
Nonprobability sampling methods
include:
Convenience Sampling, in which an intact
group of people is used as the subjects based
simply on ease of access to them.
Purposeful Sampling (sometimes called judgmental sampling), in which specific persons are
selected from the population because they are
“judged” to be representative and/or
informative about the topic being studied.
Quota Sampling, which is used when a
probability sampling method cannot be used,
but subjects are able to be selected to represent
identified characteristics of the population.
Quota sampling procedures typically involve
use of a “sampling framework,” which is a
diagram of the proportionate relationships
among the variables upon which the sampling is
based.
There are two basic ways to determine the
minimum sample size needed.
If a quota (or sampling frame) approach is
used, effective fulfillment of the various cell
entries is the minimum total needed to
represent the population effectively.
If a probability sampling approach is used,
“statistical power” (which is the probability
that a given statistical technique will result in
rejection of a false null hypothesis) should be
computed.
The level of significance of a statistical test is
closely related to sample size.
For a given level of statistical significance, a
statistical power formula provides an
estimate of the minimum sample size needed
for that significance level.
For example, fewer subjects are needed to reject
the null hypothesis at the .05 level than at the .01
level.
Statistical power is always enhanced by large
sample size.
Other factors to consider in sampling:
Type of research: correlational research requires
larger samples than does experimental research.
Research Hypotheses: expectation of smaller
differences requires larger samples.
Cost: what is the largest possible sample size
that can be achieved with available resources?
Importance of Results: the greater the
implications, the larger the sample should be.
Number of Variables: the greater the number
of variables, the larger the sample needed.
Data Collection Methods: larger samples are
needed if there is considerable measurement
error.
Accuracy Needed: the greater the accuracy
needed, the larger the sample needed.
Size of Population: the larger the population,
the smaller the percentage needed to represent
it.
This concludes Part 3 of the
presentation on
RESEARCH AND
PROGRAM
DEVELOPMENT