Research Reviewer
I. VALIDITY AND RELIABILITY
-
In this section, the suggestions, corrections,
and refinement of the draft may be
explained fully thoroughly. The different
persons involved in the correction and
refinement of the research instrument may
be mentioned. After the refinement of the
instrument, testing the validity and
reliability of the instrument may be done.
1. For validity - experts in line with the field of the
study may be requested to go over the research
instrument. Criteria for validation may be
recommended and given to experts as their bases.
It is the extent to which the scores from a measure
represent the variable they are intended to
Internal validity
- refers to how well an experiment is done,
especially whether it avoids confounding (more
than one possible independent variable [cause]
acting at the same time). The less chance for
confounding in a study, the higher its internal
validity is.
Therefore, internal validity refers to how well a
piece of research allows you to choose among
alternate explanations of something. A research
study with high internal validity lets you choose
one explanation over another with a lot of
confidence, because it avoids (many possible)
confounds.
External validity
- refers to how well data and theories from one
setting apply to another. This question is usually
asked about laboratory research: Does it apply in
the everyday "real" world outside the lab?
2. For reliability, the researcher may use any of the
four methods. These are (1) test-retest, (2) splithalf, (3) parallel-form, and (4) internal consistency.
If test-retest is used, the questionnaire is
administered twice to the pilot sample with the
same field of specialization in other institutions but
not included as subjects of the study. Spearman
rank difference correlation coefficient is used to
determine the reliability of the responses. If the
reliability value is high, it means that the research
instrument is reliable, it is now ready to be
administered to the subjects.
Test-retest reliability
- is the extent to which this is actually the case. For
example, intelligence is generally thought to be
consistent across time. A person who is highly
intelligent today will be highly intelligent next
week. This means that any good measure of
intelligence should produce roughly the same
scores for this individual next week as it does
today. Clearly, a measure that produces highly
inconsistent scores over time cannot be a very
good measure of a construct that is supposed to be
consistent.
Split-half correlation
This involves splitting the items into two sets, such
as the first and second halves of the items or the
even- and odd-numbered items. Then a score is
computed for each set of items, and the
relationship between the two sets of scores is
examined.
Internal consistency
- is the consistency of people’s responses across
the items on a multiple-item measure.
Parallel-Forms Reliability

One problem with questions or assessments
is knowing what questions are the best
ones to ask. A way of discovering this is do
two tests in parallel, using different
questions.



Parallel-forms reliability evaluates different
questions and question sets that seek to
assess the same construct.
Parallel-Forms evaluation may be done in
combination with other methods, such as
Split-half, which divides items that measure
the same construct into two tests and
applies them to the same group of people.
The figure at the right summarize external
and internal validity and the relation
between the two. The green ellipse
represents internal validity, and the blue
rounded rectangle around it represents
external validity,
(focused in the present condition – what is);
experimental design (future – what will be);
and case study design (past, present and
future).
Four Parts of the Experimental Procedure
1. Selection of the appropriate materials for
the tests;
2. Specification of the variable to be
measured;
3. Selection of the procedure to be used in the
measurement of the variable; and
4. Specification of the procedure to be used to
determine whether the measurements
support the hypothesis.
Principles of a Research Design
 Replication
 Randomization
 Local control
 Control of extraneous variables
Replication
II. Research Design
Research Design
The success or failure of an investigation usually
depends on the design of the experiment.
 It is a master plan specifying the methods
and procedures for collecting and analyzing
the needed information in a research study.
 Are of four kinds: historical design (focused
in the past – what was); descriptive design
 A very essential element in the design of
any research. Doing just the basic
experiment is not enough. The experiment
should be repeated several times to find an
estimate of variations among observations
on the group of subjects treated alike.
 Replication allows the researcher to assess
the significance of the observed differences.
Only through several repetitions of the
experiment will make statistical test of
significance possible.
Randomization
 Refers to the assignment of the
experimental subjects to the treatments by
chance.
 It is done to create equivalent groups prior
to the experiment. It tends to average out
differences among groups.
EXPERIMENTAL DESIGN
Experimental Design: The Basic Building Blocks
•
•
 It reduces possible bias in comparative
experiments.
 It also assures a valid or unbiased estimate
of population parameters and the validity of
the statistical test of significance.
It is a problem-solving approach that the
study is described in the future on what will
be when certain variables are carefully
controlled or manipulated.
•
This design is most useful in the natural
sciences such as Botany, Zoology, Biology,
Phycology, Ichthyology, Chemistry, Physics,
Biochemistry, Molecular Biology,
Microbiology, Biotechnology and many
others.
 Experimental units are allocated to block in
such a manner that the units within the
block are relatively homogeneous.
 Experimental units may be blocked
according to certain characteristics such as
height, age, weight, gender, grade level, pH
level, and so on.
Control of Extraneous Variables
 Make sure that no outside or extraneous
variables may affect the experimental
subjects.
 Identical conditions must be provided for
the control and experimental groups. These
two groups should differ only in terms of
the absence (control) or presence of the
treatment (experimental)
 The principles of experimental design can
be applied to SINGLE-FACTOR, TWOFACTOR and THREE-or more-FACTOR
experiments.
The general plan for selecting
participants, assigning participants
to experimental conditions,
controlling extraneous variables,
and gathering data.
•
Local Control
 Done when balancing, grouping and
blocking of experimental units are
employed in the adopted design.
Experimental design
…the researcher selects participants and divides
them into two or more groups having similar
characteristics and, then, applies the treatment(s)
to the groups and measures the effects upon the
groups.
•
An EXPERIMENTAL DESIGN is a description
of what a researcher would like to find out
and how to find it out.
Types of Experimental Design
1. Single Group Design- The single group
design is a design in which a group of
subjects are administered a treatment and
then measured (or observed).
2. Two-group design. Two comparable groups
are employed as experimental and control
groups.
3. Two-pair group design. This design is an
elaboration of the two group design
wherein there are two experimental groups
and two control groups.
4. A pretest posttest design is an experiment
where measurements are taken both
before and after a treatment. The design
means that you are able to see the effects
of some type of treatment on a group.
5. 5. Randomized Complete Block Design
(RCBD or RCB). This experimental design
uses a group of test plants and animals as
subjects of the study which are studied
once but subsequent treatments applied
are replicated to determine the cause of
change. There is control in this design and
the subjects will undergo randomization
process.
6. 6. Complete Randomized Design. This is a
design in which a group of test plants or
animals is studied only once but subsequent
(kasunod) or successive (sunod-sunod)
treatment is applied to determine the cause
of change. There is no control in this design
but the subjects will undergo randomization
process.
7. 7. Parallel-group design. This type of
experimental design consists of three or
more groups wherein one group is control
group with two or more experimental
groups. The control group or parallel group
serves as control or basis for comparison of
the experimental groups which is
manipulated or changed, but the
experimental groups vary.
8. 8. Three-group design. In this design, there
are three groups of independent variables.
9. 9. Counter-balanced or Latin square
design. This design is also called “rotation
design”. It involves an exchange of two or
more treatments taken by the subjects
during the experiment. The arrangement
employed in the design is Latin square in
which each variable is a form of square
occurring once in each row or column. This
is called quasi-experimental design.
10. 10. Correlational design. This design is used
to determine the relationship of two
dependent variables (X and Y) on how they
are manipulated by the independent
variable.
Single Group Design
 The single group design is a design in which
a group of subjects are administered a
treatment and then measured (or
observed).
 Usually, with this design, an intact group of
subjects is given the treatment and then
measured or observed.
 No attempt is made to randomly assign
subjects to the groups, nor does the design
provide for any additional groups as
comparisons.
 Single Group design does NOT have
experimental group and control group
The Two-Group Design
 Two-group design. Two comparable groups
are employed as experimental and control
groups.
 Experimental group
–
In a two-group design, the group of
participants that receives the
Independent Variable.
 Control group
In a two-group design, the group of participants
that does not receive the Independent Variable.
Two-pair group design
This design is an elaboration of the two group
design wherein there are two experimental groups
and two control groups.
Parallel-group design
This type of experimental design consists of three
or more groups wherein one group is control group
with two or more experimental groups. The control
group or parallel group serves as control or basis
for comparison of the experimental groups which is
manipulated or changed, but the experimental
groups vary.
When will we use ANOVA?
ANOVA is used when your research calls for
comparison of the means of two or more groups.
The F test is a global test and as such one test in an
analysis is done.
Therefore Parallel group design is a design in
which two or more groups are used at the same
time.
This design has a control group, parallel group and
experimental group.
CONTROL GROUP- manipulated for change.
EXPERIMENTAL GROUP- this is the group that
varies.
Parallel Group-Serves as control for comparative
purposes.
Pre-test and Post-test group design
This design involves the experimental group and
the control group which are carefully selected
through randomization procedures. Both groups
are given pretest at the beginning of the semester
and posttest at the end of the semester. But the
control group is isolated from all experimental
influences.
A pretest posttest design is an experiment
where measurements are taken both before and
after a treatment. The design means that you are
able to see the effects of some type of treatment
on a group.
 A pretest-posttest design is usually a quasiexperiment where participants are studied
before and after the experimental
manipulation. Remember, quasiexperimental simply means participants are
not randomly assigned. It is possible to have
a control group, or a group who doesn't
receive the manipulation. In a pretestposttest design, there is only one group and
all of them are in the experimental
condition.
 The reason you run a pretest-posttest
experiment is to see if your manipulation,
the thing you're looking at, has caused a
change in the participants. Since everyone
is being manipulated in the same way, any
changes you see across the group of
participants is likely from the manipulation.
This means you test them before doing the
experiment, then you run your
experimental manipulation, and then you
test them again to see if there are any
changes.
Three-group design
In this design, there are three groups of
independent variables.
Counter-balanced or Latin square design
This design is also called “rotation design”. It
involves an exchange of two or more treatments
taken by the subjects during the experiment. The
arrangement employed in the design is Latin
square in which each variable is a form of square
occurring once in each row or column. This is called
quasi-experimental design.
Complete Randomized Design
This is a design in which a group of test plants or
animals is studied only once but subsequent
(kasunod) or successive (sunod-sunod) treatment is
applied to determine the cause of change. There is
no control in this design but the subjects will
undergo randomization process.
•
Treatments are assigned randomly to the
experimental subjects without restriction.
•
The experimental subjects in this case
should be homogeneous with respect to all
other factors which could affect the
treatments being compared if they were
not controlled.
•
It is flexible and is limited only by the
number of experimental subjects. It is
therefore possible that the number of
replicates per treatment is not the same for
the different groups.
•
Statistical analysis remains simple and easy
even if some experimental subjects or an
entire treatment group is missing. The loss
of information due to missing observations
is small compared to those with other
design.
It is appropriate to use CRD if the experimental
subjects have the same characteristics and if
several drop-outs are expected
Using the CRD
In single-factor experiments, only a single factor
varies while all others are kept constant. The CRD
can be used specially for experiments with
homogeneous. Experimental units or in
experiments where environmental effects are
easily controlled like in laboratory experiments.
This is rarely used in field experiments.
Randomization can be accomplished by assigning
treatments to experimental units entirely at
random. This can be done by using a table of
random numbers or by drawing lots.
Randomized Complete Block Design (RCBD or
RCB)
The RCB is the standard design for agricultural
experiments. This experimental design uses a
group of test plants and animals as subjects of the
study which are studied once but subsequent
treatments applied are replicated to determine the
cause of change. There is control in this design and
the subjects will undergo randomization process.
•
Divides the experimental subjects into more
or less homogenous groups called BLOCKS.
•
Blocking is done to make sure that the
experimental subjects in a group have
similar characteristics so that the observed
differences among the groups will be largely
due to the treatments.
Disadvantages
•
Not easy to obtain experimental subjects
with homogeneous characteristics.
•
The accuracy of the design is reduced when
the subjects differ significantly.
•
Any variations between the experimental
subjects enter into the experimental error.
•
Blocking also helps on giving more accurate
results than the CRD.
•
The statistical test used may indicate
significant differences among the
treatments when actually there is none.
•
There are no restrictions as to the number
of treatments or number of blocks in the
experiments.
Use of RCBD
•
The subjects can be categorized according
to certain characteristics like gender, age,
weight and height which can affect
comparison of treatments.
•
A few drop-outs are expected.
Using the RCBD
•
•
•
•
•
RCBD is used when randomization will tend
to produce groups which are not nearly
equivalent for comparison. This happen
when the experimental subjects are not
homogeneous but can be divided into more
or less homogeneous groups or blocks in
which each has subjects that have similar
characteristics or properties. The
experimental units in each block are
randomly assigned to the different
treatments so that the treatments are
more or less alike in composition un terms
of the subjects’ characteristics.
RCBD is one of the most widely used
experimental designs in field researchers.
This is characterized by blocks of equal size,
each of which contains a complete set of all
treatments.
RCBD reduces experimental error
through proper blocking while retaining
much of the flexibility and simplicity of the
CRD.
Blocking is done to reduce as much
as possible heterogeneity among
experimental units within each block. It
increases the differences among blocks
while leaving the subjects within a block
more homogeneous.
Except for different treatments, the
assigned experimental units within each
block should be managed as uniformly as
possible.
•
The experimental area is divided
into as many blocks as the number of
replicates. Each block is then divided
further into groups as many as the number
of treatments. Randomization is then done
separately for each block. It is specified that
all treatments must appear once in each
replication. Randomization may be done
with the use of a table of random numbers.
Correlational design
This design is used to determine the
relationship of two dependent variables (X and
Y) on how they are manipulated by the
independent variable.
APA
What is APA?
 American Psychological Association
 is the most commonly used format for
manuscripts APA regulates
 Style
 In text Citations
 References
 A list of all sources used
When is APA Style used?
 Term Papers
 Research Reports
 Empirical Studies
 Literature Reviews
 Theoretical Articles
 Methodological Articles
 Case Studies
Where to place in-text citation
In-text Citations: Paraphrase
1. Idea-focused- Placed the author(s) and date(s) in
parentheses at a common appropriate place in or
at the end of the sentence.
 A paraphrase is material from a source
which has been summarized in your own
words.
Example: Researchers have pointed out that
the lack of trained staff is a common barrier in
providing adequate research education (Miranda,
2013).
 All paraphrases must be cited, using the
author’s last name and year of publication.
2. Researcher-focused- Placed only the date in
parentheses.
Example: Miranda (2013) recommended
that research education be required for college
graduate in Philippines
3. Chronologically-focused- Integrate both the
author and the date into the same sentence.
Example- In 2001, Weist proposed using the
Accounting and Teaching Scheme to analyze and
develop accounting mental skill programs for
college students.
Text Citations
Three types of text citations:
1. Paraphrase (summary)
2. Short Quote = less than 40 words
3. Long Quote= more than 40 words
Three key points in APA citations
1. Use author, date format whenever possible.
2. If the source is quoted, you MUST include
the page or paragraph number.
3. If author’s name is not available, use the
title of the document in place of the
author’s name
 No page number is required.
Paraphrase : Example
 If the author’s name is a part of the
sentence structure, the year follows the
author’s names in parenthesis. Quotation
marks are not included.
 Example: Author’s last name, year of
publication
Berk (2007) found that children begin to
play organized games with rules, once they reach
school-age.
 When the author’s name is not part of the
sentence structure (parenthetical citation),
the author’s name and year of publication
are included at the end of the citation, in
parenthesis. Quotation marks are not
included.
 Example: Author’s last name, year of
publication
Studies conducted found that children
begin to use organized play and games with rules
at school-age (Berk, 2007
 If the author’s name is part of the
sentence, place only the year of
publication in parentheses.
 Example:
Berk (2007) found that children begin to
play organized games with rules, once they reach
school-age.
 When the author’s name is not part of the
sentence, place both the name and the
year, separated by a comma, in parentheses
Berk (2007) found the following to be true:
We have seen that middle
childhood brings major advances in perspective
taking, the capacity to imagine what other people
may be thinking and feeling. These changes
support self-esteem, understanding of others, and
a wide variety of social skills. (p. 336)
 Example:
Studies conducted found that children
begin to use organized play and games with rules
at school-age (Berk, 2007).
Citations
In-text Citations: Short Quote
 A direct quote is information included in a
paper, which is taken directly from the
source.
 Short quotes = less than 40 words
 Enclosed within quotation marks
 Cited using the author’s last name,
year of publication, and page
number (or paragraph number, if
page number is not available.)
Table 6.1
Basic Citation Styles
Type of Citation
First Citation in Text Subsequent
Citations in Text
One Author
Two Authors
Walker (2000)
Walker (2000)
Walker and Allen (2004) Walker and Allen (2004) (Walker & Allen, 2004) (Walker & Allen, 2004)
Three Authors
Bradley, Ramirez, and
Soo (1999)
Bradley, Ramirez, Soo,
and Walsh (2006)
Walker, Allen, Bradley,
Ramiriz, and Soo
(2008)
Wasserstein et al.
(2005)
National Institute of
Mental Health (NIMH,
2003)
University of Pittsburgh
(2005)
Four Authors
Five Authors
Six + Authors
Groups (abbrev.)
Groups (no abbrev.)
Short Quote (< 40 words) Examples
 “Reading makes use of many skills at once,
taxing all aspects of our informationprocessing systems” (Berk, 2007, p. 306).
 Research has shown that “mathematics
teaching in elementary school builds on and
greatly enriches children’s informal
knowledge of number concepts and
counting” (Berk, 2007, p. 307).
Long Quote (>40 words) Example
 Quotes over 40 words must be block
formatted.
 Quotation marks are not used. The entire
block quote is indented five spaces, and
double-spaced.
 Example:
Bradley et al. (1999)
Bradley et al. (2006)
Walker et al. (2008)
Parenthetical
Format, First
Citations2007)
(Walker,
Parenthetical
Format, Second
Citations2007)
(Walker,
(Bradley, Ramirez, & (Bradley et al., 1999)
Soo, 1999)
(Bradley, Ramirez, Soo, (Bradley et al., 2006)
& Walsh, 2006)
(Walker, Allen, Bradley, (Walker et al., 2008)
Ramirez, & Soo, 2008)
Wasserstein et al.
(2005)
NIHM (2003)
(Wasserstein et al.,
(Wasserstein et al.,
2005)
2005)
(National Institute of
(NIMH, 2003)
Mental Health [NIMH],
2003)
University of Pittsburgh (University of Pittsburgh, (University of
(2005)
2005)
Pittsburgh, 2005)
In-text Citations: Electronic sources
 When citing an electronic document,
whenever possible, cite it in the authordate style.
 If electronic source lacks page numbers,
locate and identify paragraph number or
paragraph heading
 Example:
Recent research has yielded similar results (Smith,
1997, para. 6).
In-text Citations: Unknown author
 When the author’s name is designated as
“Anonymous” cite in text the word
Anonymous followed by a comma and the
date.
 Example:
(Anonymous, 1998)
 In the Reference list, an anonymous work is
alphabetized by the word Anonymous as
the author with the remaining publication
data.
EXAMPLES
A. In-text citations help readers locate the cited
source in the References section of the paper.
Whenever you use a source, provide in
parenthesis:
 When a work has no identified author, cite
the title of the reference and the year. If it
is a long title, use the first few words
 Use quotation marks around the
title of an article, a chapter or a
webpage
•
the author’s name and the date of
publication
•
for quotations and close
paraphrases, provide the author’s
name, date of publication, and a
page number
 Italicize the title of a journal, a book,
a brochure or a report.
 Examples
 Recent research reveals (“Six Sites
Meet,” 2006) significantly…
 Full publication title is “Six
sites meet for
comprehensive anti-gang
initiative conference.” Notice
the title was shortened for
the in text citation because
the title was long
 The book College Bound Seniors
(2008) asserts that…
 Because there is no author listed for these
sources, use the title in place of the
author’s name in the citation and on the
References page, in alphabetical order.
 Examples
 Recent research reveals (“Six Sites
Meet,” 2006) significantly…
 The book College Bound Seniors
(2008) asserts that…
B. When quoting:
•
Introduce the quotation with a signal
phrase
•
Include the author’s name, year of
publication, and page number
•
Keep the citation brief—do not repeat the
information
Provide the author’s last name and the year of
Introduce quotations with signal phrases, e.g.:
publication in parenthesis after a summary or a
paraphrase.
According to Xavier (2008), “….” (p. 3).
Xavier (2008) argued that “……” (p. 3).
Use such signal verbs such as:
acknowledged, contended, maintained,
responded, reported, argued, concluded,
etc.
Use the past tense or the present perfect tense of
verbs in signal phrases when they discuss past
events.
Include the author’s name in the signal phrase,
followed by the year of publication in parenthesis.
When the parenthetical citation includes two or
more works, order them in the same way they
appear in the reference list—the author’s name,
the year of publication—separated by a semicolon.
C. When including the quotation in a
summary/paraphrase, also provide a page
number in parenthesis after the quotation:
When citing a work with two authors, use
In the signal phrase, use “and” in between the
authors’
names.
(“Indiana,” 2008)
Titles:
Articles and Chapters = “ ”
Books and Reports = italicize
When citing an organization:
In parenthesis, use “&” between names
•
mention the organization the first time you
cite the source in the signal phrase or the
parenthetical citation.
•
If the organization has a well-known
abbreviation, include the abbreviation in
brackets the first time the source is cited
and then use only the abbreviation in later
citations.
When citing a work with three to five authors,
identify all authors in the signal phrase or in
parenthesis.
Example:
(Harklau, Siegal, & Losey, 1999)
In subsequent citations, only use the first author's
last name followed by "et al." in the signal phrase
or in parentheses.
Example:
(Harklau et al., 1999)
When citing a work with six and more authors,
identify the first author’s name followed by “et
al.”
Example: Smith et al. (2006) maintained that….
(Smith et al., 2006)
When citing authors with the same last names,
use first initials with the last names.
When citing a work of unknown author:
•
use the source’s full title in the signal
phrase
•
cite the first word of the title followed by
the year of publication in parenthesis.
(B. Kachru, 2005; Y. Kachru, 2008)
When citing two or more works by the same
author and published in the same year, use lowercase letters (a, b, c) after the year of publication
to order the references.
Smith’s (1998a) study of adolescent
According to “Indiana Joins Federal
Accountability System” (2008)
immigrants…
OR
When citing interviews, letters, e-mails, etc.,
include the communicator’s name, the fact that it
was personal communication, and the date of the
communication.
Do not include personal communication in the
reference list.
When citing an electronic document, whenever
possible, cite it in the author-date style. If
electronic source lacks page numbers, locate and
identify paragraph number/paragraph heading.
EXAMPLES