EDPSY 505
Conducting Quantitative
Educational Research
Syllabus
• Key points
– Introductory course
• Research design for producers
• 500 is offered for consumers
– Check with your program
» Some will accept either.
– Marley’s Office hours
• Mon 2:00 to 3:30 and Wed 2:00 to 3:30
• Or by appointment
– John’s Office hours
• Insert Here
Syllabus
• Key points (cont.).
– Assessment plan (we’ll talk more about this when Marley
returns).
• Two exams (100 points each).
– Before exams we will generate a study guide.
– Tend to select items from the textbook website.
» Multiple choice
» Short Answer
» Essay
– First exam items can be corrected for half credit.
» We’ll talk about this more and after the first exam.
Assessment plan (cont.)
• Research Proposal (100 points)
– Intermediate research proposal assignments
(5 points each, of which one will be dropped).
• Study questions for each exam (5 points
each).
• Presentation/outline of empirical study (40
points)
Syllabus
• Assessment (cont.).
– No extra credit.
– No incompletes.
• Except for extreme circumstances (e.g., Illness,
death of family member).
• Study Habits
– Rule of thumb: 3 hours of study for every
credit hour.
Syllabus
• Professionalism
– Pathfinder
• Honesty
• Integrity
– Cheating, plagiarism, etc. will not be tolerated and will result in
a referral to whoever is in charge of this place.
• Behavior
–
–
–
–
Class starts at promptly at 4:00
Turn off cell phones
If you must leave early let me know
Be respectful of others
Syllabus
• Work habits.
– Read!!!!!
– Due dates are non-negotiable.
– All work should conform to APA guidelines.
• If in a field that does not use APA let me know in advance
what guidelines your field uses.
• Other course policies.
– Religious accommodations.
– Disabilities.
• Inform and provide documentation.
Who Is Here?
• In groups of three or four.
– Identify yourself and get know one another.
• Ask the following questions.
– What are your professional experiences?
– What is your program of study?
– What topics within your program interest you?
– What is your favorite recreational activity?
– What is your claim to fame?
– Be prepared to introduce one member of your group.
– Exchange email addresses and/or phone numbers.
– Contact if you miss class.
– Bounce ideas.
– New friends.
Discussion
• As a group discuss the following
questions:
– What is research?
– Is reality knowable?
– What does it mean to know something?
– How does one go about knowing reality?
What Is Research?
It depends on how you answered the three
other questions:
What is reality?
Ontology
What is knowledge and
what does it mean to
know something?
Epistemology
How does one go about
knowing reality?
Methodology
A Paradigm Is
•
"A set of basic beliefs (or metaphysics) that
deals with ultimates or first principles. It
represents a worldview that defines, for its
holder, the nature of the 'world', the
individual's place in it, and the range of
possible relationships to that world and its
parts, as, for example, cosmologies and
theologies do." Guba & Lincoln.
A Paradigm Is
• Kuhn defines paradigms as having two
characteristics:
– "Their achievement was sufficiently
unprecedented to attract an enduring group of
adherents away from competing modes of
scientific activity."
– "It was sufficiently open-ended to leave all sorts of
problems for the redefined group of practitioners
to solve."
So what?
"Differences in paradigm assumptions
cannot be dismissed as mere
'philosophical' differences; implicitly or
explicitly, these positions have important
consequences for the practical conduct
of inquiry, as well as for the interpretation
of findings and policy choices."
Guba & Lincoln
Paradigms As Human
Constructions
• Any given paradigm represents simply the
most informed and sophisticated view of
its proponents.
– Must rely on utility and persuasiveness rather
than proof.
Paradigms
• Positivism
– Deterministic
– Reductionism
– Empirical observation and measurement
• Methods
– Experimental, manipulative, verification
Paradigms (Cont.)
• Postpositivism
– Theory testing
– Probabilistic
– Know reality imperfectly
– Replication
• Methods
– Experimental, surveys, causal-comparative,
observational, interviews
Paradigms (Cont.)
• Critical theory
– Political
– Empowerment
– Collaborative
– Change-oriented
– Social justice
• Methods
– Participatory action research
Paradigms (cont.)
• Constructivism
– Understanding
– Multiple participant meanings
– Social construction
– Theory generation
• Methods
– Grounded theory, case studies, narrative
research
Critiques of Positivism
•
•
•
•
•
•
•
•
Context stripping
Exclusion of meaning and purpose
Etic (outsider) vs. emic (insider)
Inapplicability of general data to individual cases
Exclusion of discovery dimension in inquiry
Theory-ladenness of facts
Value-ladenness of facts
Interactive nature of inquirer-inquired into dyad
“T”ruth
“creatings”
Verification
Falsification
Accretion
“Is this the right
conclusion?”
Misunderstood
as a central
element
“Passive”
Researcher
“Active”
Researcher
What “is”
can only be
“what’s
known”
C
P
1
9
4
???
What Makes Research
Scientific?
According to the Dictionary
Science:
•
Is the systematic observation,
identification, description, experimental
investigation, and theoretical explanation
of phenomena.
How does the No Child Left
Behind Act of 2001 (NCLB)
impact educational research?
Who Cares? You should.
• The No Child Left Behind Act of 2001 uses
the phrase “scientifically-based research”
(SBR) 111 times.
• This has spawned an industry of
consultants.
• It has created a very volatile atmosphere.
What Is Scientific Research?
(According to NCLB)
• The application of rigorous, systematic, and objective
procedures to obtain reliable and valid knowledge.
• Systematic, empirical methods that draw on observation
or experiment.
• Involves rigorous data analyses that are adequate to test
the hypotheses.
• Is evaluated using experimental or quasi-experimental
designs.
• Is reported in sufficient detail to allow replication.
• Has been accepted by a peer-reviewed journal or
approved by an independent panel of experts through
rigorous, objective, and scientific review.
What Paradigm Appears to Be
Influencing NCLB?
Postivism?
Post-positivism?
Critical Theory?
Constructivism?
Between a Rock and a Hard Place
The rock.
The hard place.
• Calling for scientifically
based research is good and
needed.
• “The recent enactment of no
child left behind, and its central
principle that federal funds
should support educational
activities backed by
“scientifically-based research,”
offers an opportunity to bring
rapid, evidence-driven progress
– for the first time – to U.S.
Elementary and secondary
education.” – Coalition for
evidence-based policy.
• Defining SBR as
randomized experimental
designs is over-restrictive.
• “The requirement that
research methods be
restricted to group design
with a preference for
randomized clinical trials will
significantly inhibit the
development and validation
of new scientific knowledge
in education.” – American
association on mental
retardation (AAMR) board of
directors.
“Council recognizes randomized trials among
the sound methodologies to be used in the
conduct of educational research and
commends increased attention to their use as
is particularly appropriate to intervention and
evaluation studies. However, the council of
the association expresses dismay that the
department of education through its public
statements and programs of funding is
devoting singular attention to this one tool of
science, jeopardizing a broader range of
problems best addressed through other
scientific methods. The council urges the
department of education to expand its current
conception of scientifically-based research.” –
AERA council
What Is Scientific Research?
(According to the NRC)
• Science poses significant questions that
can be investigated empirically.
• Science links research to relevant theory.
• Science uses methods that permit direct
investigation of the question.
• Science provides a coherent and explicit
chain of reasoning.
What Is Scientific Research?
(According to the NRC)
• Scientific findings replicate and generalize
across studies.
• Scientists disclose research and
encourage professional scrutiny and
critique.
Mayer (2000)
• Let’s take a few minutes and read Mayer.
• What makes research “scientific”?
• How important is it that educational
research be respected in “academia and in
society in general”?
• Should “science” and “research” mean the
same things in different disciplines?
• What questions/ comments do you have?
Mayer
(2000)
The Big Picture
• There are many different research
processes
• Each has its own:
– Philosophy of inquiry
– Methods of inquiry
– Purposes for doing research
– Processes and “rules”
• Quantitative research has its own
• Here is one process:
A Scientific Process
Research Question
Defining the Problem
Review
Literature
Define
Hypothesis
Articulate
Theory
Testing the Hypothesis




Subject sampling
Instrumentation
Research design
Piloting
 Collecting Data
 Choosing analyses
 Conducting analyses
Results or Findings
Conclusions
Scientific Thinking Vs. Everyday
Thinking
• Everyday thinking
– Biased questions
• Do you really support the war?
– Limited sampling
• Your friends and family are different from my
friends and family
– Selective attention
• Confirmation bias
– Inaccurate generalization
• Stereotypes
Scientific Thinking Vs. Everyday
Thinking (Cont.)
• Scientific thinking.
– Empirical observations.
• Empirical: capable of being confirmed, verified, or
disproved by observation or experiment.
– Systematic.
– Objective.
• Less dependent on emotion or personal
prejudices.
– Replicable.
Purposes of Scientific Research
• Exploratory
– What is out there?
• Descriptive
– What does this group look like?
• Explanatory
– Why and how are these constructs related?
• Evaluation
– Does this program work?
• Prediction
– Who will become depressed?