University of Oklahoma
Ed. Leadership and Policy
EACS 6023: Applied Quantitative Research Methods
Instructor
Curt Adams, Assistant Professor
Email: Curt.Adams-1@ou.edu
Office: 918-660-3891
Cell: 918-671-9637
College of Education Mission
The mission of the college of education is to promote inquiry that fosters democratic life
and is fundamental to productive activity in the interrelated areas of teaching, research and
service in a multicultural society.
EACS Mission
The Educational Administration, Curriculum and Supervision (EACS) area believes that
PK-12 schools are complex organizations that need leaders who understand: 1) the theoretical
and conceptual aspects of schools, their people, and programs for both children and adults who
are constantly learning from the context of the organization; 2) the technical knowledge of the
content areas and areas of specialization found within the EACS knowledge bases; 3) the
integration of theory, research, and practice as a means of grounding new best practices.
Course Materials
Hoy, W. (working manuscript). Quantitative research in education: A primer. The Ohio State
University.
Muijs, D. (2004). Doing quantitative research in education with SPSS. Los Angeles, CA: Sage.
Ancillary Readings: Research studies that employ quantitative designs will be used to enhance
students understanding of quantitative research methods. Studies will be provided by the
instructor.
Course Description
The guiding aim of this course is to prepare advanced doctoral students to become
scholarly practitioners and scholars of practice through inquiry techniques that are grounded in a
positivist/post-positivist research paradigm. Being a quantitative methods course, students will
learn how to frame research questions, design studies, collect data, analyze data, and interpret
findings through a quantitative lens. Unlike a traditional methods course that centers instruction
on the scholarship of analytical techniques and problems, the instructional core of this course is
based on the application of quantitative designs to problems of practice. The difference in the
approaches is more than just one of semantics; it entails a difference in the object of the
investigation. Our objective centers on conceptual understanding of quantitative designs and
their application, and not as much on the statistical formulas that underlie different descriptive,
1
experimental, and correlational data analyses. With that stated, quantitative research and
statistics are inextricably related so some understanding of statistics is necessary.
Course Objectives
1) To understand how hypotheses on the relationship between two or more constructs are
deduced from theory;
2) To understand the use of descriptive statistics in quantitative research and performance
management;
3) To understand the use of ANOVA for answering research questions and/or testing
research hypotheses;
4) To understand the use of correlational designs for answering research questions and/or
testing research hypotheses
5) To be able to perform a One-Way and Two-Way ANOVA with post hocs in SPSS
6) To be able to run bivariate correlations and simple multiple regression in SPSS
7) To be able to interpret SPSS output, understanding the T-Ratio and F-Ratio, significance,
effect size, power, squared multiple correlation, and standardized/unstandardized
regression coefficients;
8) To be able to critique quantitative methods used in existing research;
9) To value the applicability of quantitative research to practice;
10) To value the role of objectivity in the generation of new knowledge.
At the conclusion of the course, the student will be able to:
1. Supervise the production, analysis, and dissemination of quantitative data (e.g., attendance and
student achievement) in such a way that policies and practices can be aligned with theories that
explain educational phenomena;
2. Utilize technology to analyze and interpret educational data, issues, and trends for boards of
education, committees/taskforces, and other groups;
3. Use data to inform decisions, to plan and assess school programs, to design accountability
systems, to plan for school improvement, and to develop and conduct research;
4. Stimulate staff engagement in the ongoing study of effective practices and relevant research
for the purpose of continuous district and school improvement;
6. Independently analyze and interpret data, and utilize the data in guiding the decision-making
process.
ELCC Standard: Information Management and Evaluation
The institution’s program prepares school leaders who demonstrate an understanding of,
and the capability to:
2.1 Conduct needs assessment by collecting information on the students; on staff and the school
environment; on family and community values, expectations and priorities; and on national and
global conditions affecting schools.
2
2.2 Use qualitative and quantitative data to inform decisions, to plan and assess school programs,
to design accountability systems, to plan for school improvement, and to develop and conduct
research.
2.3 Engage staff in an ongoing study of current best practices and relevant research and
demographic data, and analyze their implications for school improvement.
2.4 Analyze and interpret educational data, issues, and trends for boards, committees, and other
groups, outlining possible actions and their implications.
Preparation Work
1) Read Chapters 1 and 2 in the Hoy text.
2) Familiarize yourself with the key terms.
3) Respond to the discussion prompts that will be posted on Desire-to-Learn.
4) Generate a research question on a relationship between two or more variables. Identify
the independent variable and the dependent variable as well as any confounding
variables. Select a relationship pertaining to a line of inquiry you are considering as a
dissertation topic. Be prepared to share your question with the class. This exercise is not
for a grade. It is designed to assess your prior research knowledge.
Course Outline
Wed./Mon.
- Syllabus Review
- The Nature of Research and Science
- Concepts, Variables, and Research Problems
Saturday, 10/25
- Foundations of Quantitative Research (measures of central tendency and dispersion)
- Experimental and Quasi-experimental Design
- Hypotheses testing
- T-Test, Chi-Square
Thursday, 11/13
- Quantitative Methods: A Framework for Continuous Improvement
- Case Study: The FOCUS program (Kirt Hartzler, Associate Superintendent, Union Public
Schools)
Saturday, 11/15
- ANOVA, Repeated Measures
- Effect size, power analyses, output interpretation
- Internal / External validity Issues
- Bivariate analysis
3
Thursday, 12/4
- Ex Post Facto Designs: Causal Comparative and Correlational Research
- Bivariate analysis
Saturday, 12/6
- Multivariate Analysis
- Foundations of Multiple Regression
Grading
The grading process is transparent and criteria based. Evaluation rubrics are reviewed
before each project, ensuring students know and understand the criteria used to measure their
application of the knowledge learned in class. Grades are based on class participation (40
percent), the problem of practice (40 percent) and a final exam (20 percent).
Class Participation .........................................................................................40 percent
Problem of Practice ......................................................................................40 percent
Final Exam .....................................................................................................20 percent
Class Participation
Your level of learning is proportional to your class participation and out of class study time.
Class participation is measured by three interdependent factors: 1) Attendance, 2) Engagement,
and 3) Research Critiques. Each element is explicated below.
1) Attendance - For every three hours of class time missed, an extra policy brief will be
required to earn attendance points. Students who miss more than 7 hours of instructional
time must withdraw from class. If exceptional circumstances arise, it is your
responsibility to contact and inform the instructor.
2) Engagement – Knowledge is not passively acquired. You learn from doing. Thus, it is
expected that you will participate in discussions, contribute to group activities, read
independently, and work on practical examples.
3) Research Critiques – Reading research exposes the reader to new ideas and possibilities
that broaden the lenses one uses to understand reality. Additionally, reading research
helps students become consumers of scholarly material. Two critiques of quantitative
research studies are required. The instructor will provide the articles in class.
Problem of Practice: Quantitative Research Design
Design a quantitative study on the relationship between two or more concepts that
interest you. Your concepts should manifest within one of four conceptual fames of school
organizations: student experiences, the professional life of teachers, the governance and
management of schools, and school and community cooperation.
Part One (20 pts): Due by 11/13
4
A) Problem Statement
- research question(s)
- purpose
B) Conceptual Framework
- Conceptual definition
Part Two (20 pts): Due by 12/15
A)
-
Methods
Research design
Sample
Data collection (Describe how data would be collected if you were to execute the study)
Operational definitions
Analytical technique (Describe the analytical technique you would use)
Problem of Practice Evaluation Rubric
Part One (20 pts): Due by 11/13
1) Problem Statement
- (2pts)The problem underlying the research is described in one or two paragraphs.
- (2pts)Arguments on the existence of the problem are stated.
- (2pts)Evidence from the relevant literature support your arguments.
2) Purpose Statement
- (2pts)A statement of research purpose is made in one or two paragraphs.
- (2pts)The statement clearly articulates the relationship between the concepts of interest.
3) Research Question(s)
- (2pts)One or more research questions related to the problem are identified.
- (2pts)Research questions address both the problem and purpose statements.
4) Conceptual Framework
- (2pts)A Conceptual definition from the literature is provided for each concept.
- (2pts)A model that shows the theoretical relationships between and among the concepts
of interest and any extraneous variables that should be controlled is developed.
Part Two (20 pts): Due by 12/15
5) Research Design
- (3pts) The proposed research design for the study is described.
- (2pts) The proposed research design is appropriate for the questions asked.
6) Sample and Data Collection
5
-
(2pts) The proposed unit of analysis for the study is identified correctly.
(3pts) A proposed data collection process that includes all relevant subjects is described.
7) Operational Definitions
- (2pts) An operational definition is provided for each of your measures.
- (3pts) Literature is cited to support your choice of measure.
8) Analytical Technique
- (3pts) The proposed analytical technique is described.
- (2pts) The proposed analytical technique is appropriate for the research questions asked.
Final Exam
The final exam will take place on 12/6. It will simulate part I of your competence exam
by asking you to design a quantitative research study that provides objective data on a specified
problem of practice. Core concepts covered on the exam will be provided on 11/15.
Final Exam Review
The final exam is based on a research problem in which you need to design a study on the relationship
between two or more variables, state research questions related to the variables, make a prediction about
the relationship, draw conclusions from SPSS output, and report findings . You need to be able to do the
following:
1)
2)
3)
4)
State a research question commensurate to the problem;
State a hypothesis that predicts the relationship between two variables;
Chose either a quasi experimental research design or a correlational design;
Identify the appropriate analytical technique (T-Test; ANOVA, Correlation, Regression) to test
your hypothesis;
5) Describe your operational definitions (how you can measure the variables);
6) Interpret SPSS output (p-value, effect size, post hoc, main effect, interaction effect, correlation
coefficient, regression coefficient);
7) Report Findings.
Below is an example of a problem I could give for the final (This is just an example!!!!)…
Measuring the Effectiveness of a Collegial Teacher Evaluation Model
Your administrative team plans to implement the model in three high poverty schools, an elementary
school, a middle school, and a high school. These pilot schools were part of a group of 12 schools
identified as having low levels of collective teacher efficacy by the original research. The purpose of the
new model is to improve teaching and learning, but mediating this outcome are social, affective, and
cognitive norms that underlie behavior. Thus, one measure of effectiveness will be improved collective
teacher efficacy beliefs. You and your team must design a study to evaluate this outcome. The focus
should center on the difference in collective teacher efficacy between teachers in schools with the new
6
program and those in comparable schools without it, as well as between school configurations
(elementary, middle, and high).
Your study should cover the following research components:
1) State a research question for the main effect of teacher evaluation type on collective efficacy.
State a research question for the interaction effect of teacher evaluation type and grade
configuration on collective teacher efficacy.
2) State a research hypothesis for the effect of teacher evaluation type on collective teacher efficacy.
3) Run a 2-way ANOVA in SPSS. Perform a scheffe post hoc for grade configuration.
4) Create an ANOVA summary table.
5) Report findings in a way that answers both research questions and tests your research hypothesis.
Predicting School Performance
Data were collected by a team of researchers on conditions of collective efficacy and school
identification within your district’s 79 schools. Based on previous evidence you conjecture that
both of these conditions will be related to school performance. Test your belief by following the
following steps.
1) State a research question(s) that examines the relationship between collective teacher
efficacy and school performance, as well as the relationship between school identification
and school performance.
2) Turn your question(s) into a hypothesis or hypotheses.
3) Draw a model that illustrates the interrelationships among SES, school size, collective
efficacy, school identification and school performance.
4) Test your model by performing three multiple regressions.
5) Create a Regression Summary Table.
6) Report findings from each regression run. Make sure you report the variance explained
for each model and beta weights for each model.
7
Research Critique Framework








Citation
Problem Statement
Purpose/Research Question(s)
Sample
Data Collection
Operational Definitions
Analytical Technique
Findings
8
EACS 6023: Statistical Application
The FOCUS program is an intervention targeted at improving learning conditions for students who have
exhibited signs of disengagement from the educational process. The intervention is predicted on the
assumption that a reengineered school structure will influence critical social conditions that are known to
underlie learning and development. Union School District has contacted you to examine the effect of the
program on students’ social/emotional states and parents’ support for learning. In particular, Union is
interested in knowing if there is a difference between ethnic and gender groups in perceived levels of
student trust and perceived parent influence in learning. Present your findings in a report that briefly
explains the purpose of the investigation, research methods, and findings. Use the attached framework as
a guide.
Descriptive
1) How many students from each ethnic group completed surveys?
2) What is the percentage of female students to male students in the sample?
3) What is the average time in the district for students in the program?
Comparison Between Two Groups
1) Create a table that delineates the relationship between gender and student trust and gender and
parent influence.
2) What is the appropriate analytical technique to use for each analysis? Why?
3) Is there a difference in levels of student trust by gender? Is there a difference in parent influence
by gender? Place the mean for each group in your respective tables.
4) If there is a difference, is the difference significant (report the p –value)?
5) Using cohon’s effect size formula, calculate the effect size of gender on student trust and for
gender on parent influence.
Comparison Between More Than Two Groups
1) Create a table that describes the relationship between ethnicity and student trust and ethnicity and
parent influence.
2) What is the appropriate analytical technique to use for each analysis? Why?
3) Is there a difference in levels of student trust by ethnicity? Is there a difference in parent influence
by ethnicity? Place the mean for each ethnic group in your respective tables.
4) If there is a difference, is the difference significant (report the p –value)?
5) Where does the difference occur?
6) Identify the effect size of ethnicity on student trust and for ethnicity on parent influence.
Comparison Across Two Independent Variables
1) Create a table that describes the relationship between ethnicity, gender, and student trust, as well
as for the relationship between ethnicity, gender, and parent influence.
2) What is the appropriate analytical technique to use for each analysis? Why?
3) Is there a difference in levels of student trust by ethnicity? By Gender? Place the mean for each
ethnic group in your table.
4) If there is a difference, is the difference significant (report the p –value)?
5) Is there an interaction effect on student trust across gender and ethnicity?
6) Is there a difference in levels of parent influence by ethnicity? By Gender? Place the mean for
each ethnic group in your table.
7) If there is a difference, is the difference significant (report the p –value)?
8) Is there an interaction effect on parent influence across gender and ethnicity?
9) What are the effect sizes of ethnicity, gender, and ethnicity*gender on parent influence.
10) Report your findings in one paragraph.
9
FOCUS Report Framework
I Purpose of the Investigation
a. Introduce the purpose of the program
b. X – Y relationship
II Research Methods
1. Sample
a. Unit of Analysis and number of students
2. Measures (See attachment for a description)
a. Student Trust
b. Parent Influence
3. Analytical Techniques
a. Descriptive Statistics
b. T-Test
c. One Way ANOVA
d. Two Way ANOVA
III Findings
1. Descriptive Statistics
a. Percentage of male, female, and ethnic group
b. Average years in the district
2. T-test Findings
a. Group difference
b. Significance
c. Effect size
3. One-Way ANOVA Findings
a. Group difference
b. Significance
c. Post-hoc – locate the difference
d. Effect size
4. Two-Way ANOVA Findings
a. Group difference
b. Significance
c. Interaction effect
d. Effect sizes
10
Measures
a.) Student Trust will be measured with the Student Trust of Teachers Scale. The scale
measures the quality of student-teacher relationships from the perception of students.
Construct validity is supported by its strong correlation with the Omnibus Trust Scale
(Hoy & Tschannen-Moran, 1999). An alpha of .91 (Adams, 2008) indicates strong item
consistency.
b.) Parent Support for Student Learning measures students' perceptions of their parents'
support for their school performance. Questions ask students how often their parents or
other adults encourage them to work hard, do their homework, and take responsibility for
their actions. High levels indicate strong parental support. Items come from the
Consortium on Chicago School Research (2007). Item reliability is strong at .87.
11
A Study of District Professional Development
Independent Variables
1. Type of professional Development
2. Nationally Board Certified
3. Teaching Experience
Dependent Variables
1. Perceived Effectiveness
2. Perceived usefulness
3. Change in teaching
4. Student achievement
1. Compose a research question to study the relationship between two or more of the
above variables.
2. Create an ANOVA table for your study.
3. Run the analysis in SPSS. Print your output
4. Write-up your findings.
An Examination of the Relationship between Collective Efficacy and Academic Achievement
Independent Variables
1. Collective efficacy
2. Socioeconomic status
Dependent Variable
1. API
1. Compose a research question to study the relationship between collective efficacy and
school performance.
2. Run a bivariate correlational analysis in SPSS. Print your output
3. Correlation Matrix
4. Write-up your findings.
The Determinants of Parent Trust
Independent Variables
1. SES
2. Prior Performance
3. School Identification
4. School Size
1.
2.
3.
4.
5.
Dependent Variable
Parent Trust
Compose a research question for the determinants of parent trust
Run a bivariate correlational analysis in SPSS. Print your output
Run a multiple regression analysis in SPSS. Print your output
Create a Correlation Matrix and Regression Table
Write-up your findings
12
EACS 6023: Class Agenda
10/15 and 10/20
I Introduction
Syllabus review
Student questions
II Characteristics of Scientific Inquiry
Discussion
Illustrations (research abstracts, Trust Theory)
III Understanding the Research Process
Description of each phase
Research Questions
Conceptual and Operational Definitions
IV Measuring the Community School Concept
Community School Framework
Design an Operational Definition
V Introduction to SPSS
VI Action Item
Read Muijs (2004) chapters 1 and 2
13
EACS 6023: Class Agenda
10/25
I Introduction
Student questions
Review the role of theory in quantitative research
II Understanding Variables
Definition of Variables
Types of Variables
Measured and Manipulated Variables
Mediating and Moderating Variables
SPSS Application
III Lunch and Reading
Chapter 3 in Hoy “Conceptual Foundations of Statistics” (p.36-42)
IV Experimental/Quasi Experimental Designs
Measures of Central Tendency and Dispersion
Normal Curve
Design Characteristics of Experimental/Quasi Experimental Research
Application
VI Action Items
Read: Bulterman-Bos, J. (2008). Will a clinical approach make education research more
relevant for practice? Educational Researcher, 37(7), 412-420.
Read: Hoy Chapter 3
Read: Muijs Chapter 7
Work on Part I of problem of practice
14
EACS 6023: Class Agenda
11/13
I Introduction
1) Problem of Practice
2) Review characteristics of Experimental / Quasi-Experimental Designs
II Conceptual Characteristics of Statistical Tests for Experimental Designs
3) T-Test
4) F-Test
5) Chi-Square
III Research Critique
6) Borman, G. & Dowling, M. (2006). Longitudinal achievement effects of multiyear
summer school: Evidence from the teach Baltimore randomized field trial. Educational
Evaluation and Policy Analysis, 28(1), 25-48.
IV Quantitative Research Methods Applied to Strategic Management/Continuous Improvement:
The Case of FOCUS
7) Dr. Kirt Hartzler, Associate Superintendent, Union Public Schools
VI Action Items
a. Research Critique: “When Elementary Schools Stay Open Late: Results from the
National Evaluation of 21st Century Community Learning Centers Program” Or, “Raising
the Bar: Curricular Intensity and Academic Performance”
b. Review Chapter 3 in Hoy and Chapter 7 in Muijs
15
EACS 6023: Class Agenda
11/15
I Introduction
1) Research Critique
2) Questions from Thursday’s class
II Conceptual Characteristics of Statistical Tests for Experimental Designs
3) T-Test
4) F-Test
5) Chi-Square
III SPSS Application
6) What is the relationship between reading instruction and reading achievement?
7) Output interpretation
8) Main Effects Vs. Interaction Effects
IV Lunch and Reading
9) Read Chapter 10 in Muijs
V Application
10) FOCUS evaluationls
VI Action Items
11) Research Critique: Leithwood, K., & Jantzi, D. (2008). Linking leadership to student
learning: The contributions of leader efficacy. Educational Administration Quarterly,
44(4), 496-528. I will post discussion prompts on desire-to-learn. You are expected to
contribute three responses to the discussion.
12) Reading: Muijs chapter 8; Hoy chapter 3 page 47-51.
13) Work on Part II of the Problem of Practice
14) Review for Final Exam
16
EACS 6023: Class Agenda
12/4
I Introduction
1) Review ANOVA Designs
2) Review ANOVA Output
II Bivariate Analysis – The relationship between two variables
3) Pearson’s r
4) Correlation Matrix
III SPSS Application and Problem of Practice Consultation
5) A study of District Professional Development
- Research question(s)
- ANOVA Table
- Findings
6)
-
An examination of collective efficacy and achievement
Research question(s)
Correlation table
Findings
VI Action Items
7) Reading: Muijs chapter 9; Hoy chapter 3 page 47-51.
8) Work on Part II of the Problem of Practice
9) Review for Final Exam
17
EACS 6023: Class Agenda
12/4
I Introduction
1. Review ANOVA Designs
2. Review ANOVA Output
3. Review Bivariate Correlation
II Multiple Regression
4. Purpose
5. Explaining Variance
6. Terminology / Output
III SPSS Application
7. A study of Parent Trust
- Research question(s)
- Correlation Matrix
- Regression Output
- Findings
IV Lunch
V Quantitative Research Applications – Final Exam
8. ANOVA
9. Correlation
10. Regression
18