Full Implementation Results for
General Psychology
@ Frostburg State University
Primary FSU NCAT Team:
•Megan E. Bradley, Ph.D.
•Bill Southerly, Ph.D.
MCRI Workshop
May 29, 2009
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
•Cindy D. Hay, MDE & MS
•Joseph Hoffman, Ph.D.
•John Bowman, Ph.D.
• Psyc150: General Psychology
•
•
•
•
Frostburg State University in Frostburg, Maryland
Annual enrollment: About 900
Mostly traditional students and 1st year students
Required course for Psychology Majors and 5
other majors
• Replacement Model
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Impact on Student Learning
General Psychology Students
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Comparisons
N compared ADMIT GPA*
to prior #
FSU GPA*
Traditional
16% smaller
3.11
2.76
Redesign
100% larger
2.92
2.17
*Significantly different; p = .005 (eta2 = .027), p = .000 (eta2 = .075)
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Results: 50 Question Common
Comprehensive Final Exam*
Mean Test Scores
78%
75%
76%
74%
72%
70%
68%
68%
Traditional
Redesign
66%
64%
62%
Course Type
*Instructors
blind to exam
content
*A one-way ANOVA of section on final exam percentage grades was significant, F = 23.251, p = .000,
eta2 = .090. Also significant with GPA as covariate: F = 29.192, p = .000, eta 2 = .11.
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Results: 50 Question Common
Comprehensive Final Exam
Section
Admit GPA
FSU GPA
Mean % on
Final Exam
Section 1
Traditional
3.02
2.72
67.5%
Section 2
Traditional
3.21
2.81
69%
Section 3
Redesign
2.95
2.46
75.3%
Section 4
Redesign
2.89
2.08
75%
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Results: 50 Question Common
Comprehensive Final Exam
• Final exam scores positively correlated
with average scores on MQs
• r =.523, p = .000
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Results: Extra Credit Final
Exam Essay Question on Prejudice
• Assessment #2: Comparison of extra credit
essay question on final exam
• All instructors blind to exam content
• Question: Discuss the psychological
phenomenon of prejudice. In your answer, feel
free to reflect on causes/explanations,
consequences of being a victim of prejudice,
and/or ideas to counter prejudice.
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Results: Extra Credit Final
Exam Essay Question on Prejudice
• Assessment #2: Comparison of extra credit
essay question on final exam
• Scoring rubric emphasized use of psychological
concepts
•
•
•
•
No upper limit for scores
1 point per correct use of psychological concepts
1 point per accurate definition of terms
1 point per additional psychological concept
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Results: Extra Credit Final
Exam Essay Question on Prejudice
Mean Test Scores
3.50
2.845
3.00
2.50
2.00
1.50
1.092
Traditional
Redesign
•
1.00
0.50
0.00
Mode
• Traditional = 0 or 1
• Redesign = 1, 2, 2, 3
Section Type
*A one-way ANOVA of section on essay grades was significant, F = 6.787, p = .000, eta2 = .153.
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Results: Extra Credit Final
Exam Essay Question on Prejudice
• Essay score positively correlated with
• Grade on semester long prejudice project
• r =.328, p = .000
• Grade across all online discussions
• r = .244, p = .005
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Redesign Issues &
Implementation Solutions
• Increase use of more learning principles
• Spacing effect: 3 unit exams
• Deeper learning: Reduced overall coverage
• “Deadline Disorder”
• Reduced to: MQs, Discussions, Prejudice Activity
• 2 weeks to complete
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Redesign Issues &
Implementation Solutions
• Students not ready for blended design
• “Freshmen don’t do optional” +
• “Students need structure”
• Required computer lab 1x/wk
• Students needed more in-class assistance
• Updated our online instructor’s manual to include
brief direct instruction
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Pilot Redesign Issues &
Implementation Solutions
• Need for more campus-wide support
• Held 3 workshops on redesign
• Implemented student support services programs
• Tutoring, Supplemental Instruction
• ORIE + Learning Communities
• Wellness initiative
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Full Implementation Results: 43
Common Questions (3 exams)*
Mean Test Scores
78%
76%
74%
72%
70%
68%
66%
64%
62%
60%
58%
77%
70%
65%
Traditional
Pilot Redesign
Full Redesign
*Instructors
blind to exam
content
Course Type
*A one-way ANOVA of section (3 total) on common question percentage was significant, F = 25.852, p
= .000, eta2 = .825.
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Factually-Based versus
Conceptually-Based Questions
Mean Test Scores
82%
80%
78%
76%
74%
72%
70%
68%
66%
64%
62%
Mean Test Scores
81%
80%
76%
69%
75%
Traditional
70%
Pilot Redesign
65%
Full Redesign
60%
72%
63%
58%
Traditional
Pilot Redesign
Full Redesign
55%
50%
Course Type
Course Type
*Factual: F = 18.480, p = .000, eta2 = .771
Conceptual: F = 23.941, p = .000, eta2 = .813
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Full Implementation Results: Pre vs.
Post Prejudice Essay
Mean Test Scores
3.50
3.00
2.28
2.50
2.00
1.50
Pre-Essay
Post-Essay
1.00
•
0.50
0.310
0.00
Mode
• Pre-Essay = 0 or 1
• Post-Essay = 0,1, 2, 3, 4
Section Type
*A one-way repeated measures ANOVA on essay grades was significant, F = 230.71, p = .000, eta2 = .420.
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Full Implementation Results: Pre vs.
Post Prejudice Essay
• Essay score positively correlated with
• Grade on semester long prejudice project
• r =.119, p = .030
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
DWF Rate
• Previous average: 12.5%
• 18% prior to pilot
• Pilot Semester
• Traditional sections: 4%
• Redesign sections: 22%
• Full Implementation - Fall
• 12.8%
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Impact on Student Learning
ULAs
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
ULA Benefits
• “Field Experience” course for top students
• Leadership in Psychology Certificate Program
• Interning as a ULA
• Research experience included
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
ULA Benefits
• Future opportunities
• Graduate School
• Teaching or Research Assistantship
• Prestigious Internships
• Most recent: Leadership Institute
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Cost Savings
• Plan: $89 to $32
• Reality: $89 to $26
• Reason: Only 50% of ULAs getting paid
• Biggest cost effectiveness:
• Tripling capacity in class
• Staffing
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Cost Savings
• Use of savings
• Realized:
• Communication Response System Implementation (“clickers”)
• Laptop lab
• Future sustainability
• Support to Chair?
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley
Special Thanks
• Our administrative team…for the support
• Joseph Hoffman & John Bowman
• USM…for seizing an opportunity
• Don Spicer & Nancy Shapiro
• Board of Regents
• NCAT…for the vision, knowledge & skills
• Carol Twigg & Carolyn Jarmon
Presented at the MCRI Workshop May 2009 by Dr. Megan E. Bradley