Assessment of a Preparatory MOOC
in Biology
Adrienne Williams
Project Specialist, HHMI-UCI Professor Program
Department of Developmental and Cell Biology, UC Irvine
June 2014
Rationale
Average scores in 12th Grade NAEP mathematics, 2013
Asian
White
Hispanic
Black
0
50
100
Exam score
150
National Assessment of Educational Progress Exam,
http://nationsreportcard.gov/reading_math_g12_2013/#/group-results
200
Rationale
Average Math SAT scores for students at each final letter grade, Bio 93
700
Ave Math SAT
600
500
11.1%
URM
400
22.6%
URM
25.9%
URM
300
51.6%
URM
200
100
0
A
N = 803, Fall 2012 Bio 93.
B
C
Final Grade in Introductory Biology
D/F
Experimental Question
Will a preparation MOOC help students do better in
introductory biology?
Experiment 1 Design
UCI students
Preparation for Introductory Biology
MOOC
Introductory
Biology at
UCI
~ 1800 students
MOOC Course Design
Basics Lecture Videos
Unit Peer
Assessments
Basics Quizzes
Write your answers
Grade answers of peers
Research Packet
Research Peer
Assessments
Normal Certificate
Write your answers
Grade answers of peers
Distinction Certificate
MOOC Participation
40000
37908
Number of Total Students
35000
30000
25000
20000
15000
9401
10000
4460
5000
3155
2441
0
Enrolled
Introductory Last Unit 1
Video
Quiz
Last Unit 2
Quiz
Last Unit 3
Quiz
MOOC Completion
Distinction
Normal
Distinction
Did not
complete
Did not
complete
Normal
All Courserians
UCI Students
N = 9401
N = 382
Results
Math SAT scores for all participation levels
Average Math SAT
Asian (910)
600
White (287)
500
URM (498)
400
300
200
100
0
Did not take
N = 1313
Low Participation
129
Normal
Distinction
130
130
MOOC Participation
Results
Percent of MOOC Participant Group
All ethnicities participate similarly
100%
80%
60%
White
URM
40%
Asian
20%
0%
Did not take
Low
Participation
Normal
Distinction
Results
Actual – Predicted Z Score in
Bio 93
All Math SAT scores do not respond identically
0.6
0.5
0.4
0.3
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
Math SAT >= 550
Math SAT < 550
Did not take
Low Participation
Normal
Distinction
Predicted Z score equation:
Z score = -3.54 + 0.14*[Gender] + 0.004*[MSAT Ave] + 0.002*[RSAT Ave] + 0.62*[AP 3]
Results
All ethnicities do not respond identically
Actual – Predicted Z Score
in Bio 93
0.70
0.60
0.50
0.40
0.30
Asian
0.20
White
0.10
URM
0.00
-0.10
-0.20
Did not take
Low Participation
Normal
Distinction
MOOC Participation
Predicted Z score equation:
Z score = -3.54 + 0.14*[Gender] + 0.004*[MSAT Ave] + 0.002*[RSAT Ave] + 0.62*[AP 3]
Conclusions?
California community college students do worse in
online courses.
Public Policy Institute of California: Online Learning and Student Outcomes, 2014
http://www.ppic.org/main/publication.asp?i=1096
California community college students do worse in
online courses.
Public Policy Institute of California: Online Learning and Student Outcomes, 2014
http://www.ppic.org/main/publication.asp?i=1096