2013-2014
Assessment
Report
Andrews University
School of Education
Outline of Report
• Enrollment
• Completers
• Candidate Assessment
• Program and Faculty Assessment
• Recommendations
SED Enrollment
(2013-2014)
Based on data provided by the Office
of Institutional Research across three
semesters:
• Summer 2013
• Fall 2013
• Spring 2014
SED Enrollment (’13-’14)
Level
Candidates
Undergraduate
62
MAT
17
MA/MS
Graduate Certificate
EdS
Doctoral (EdD, PhD)
91
11
26
197
71
475
Undeclared/Non-degree
TOTAL
TLC Enrollment (’13-’14)
TLC Program
Candidates
Elementary Education
64
Secondary Education
15
C & I (MA)
C & I (Doctoral)
TLC TOTAL
12
31
122
LEAD Enrollment
(’13-’14)
K-12 Ed Lead
Candidates
Graduate Certificate
11
MA
1
EdS
Doctoral
Ed Lead TOTAL
3
16
31
LEAD Enrollment (cont’d)
(’13-’14)
Higher Ed Admin
Candidates
MA
0
EdS
Doctoral
HEA TOTAL
1
10
11
LEAD Enrollment (cont’d)
(’13-’14)
Leadership
Candidates
Undergrad Certificate
MA
14
EdS
Doctoral
Leadership TOTAL
0
94
108
GPC Enrollment (’13-’14)
GPC Program
Special Education/
Learning Dis (MS)
School Counseling
(MA)
Clinical Mental Health
Counseling (MA)
School Psychology
(EdS)
Candidates
9
9
29
22
GPC Enrollment (cont’d)
(’13-’14)
GPC Program
Educational
Psychology (MA)
Educational
Psychology (Doctoral)
Counseling
Psychology (Doctoral)
GPC TOTAL
Candidates
17
14
32
132
SED Completers
(2013-2014)
Data Include:
• Graduates: Summer 2013 &
Spring 2014
• Degrees Conferred:
December 2013
Completion Rates
SED Completers (’13-’14)
Level
Completers
Undergraduate
11
MAT
3
MA/MS
Graduate Certificate
EdS
Doctoral (EdD, PhD)
TOTAL
17
2
8
25
66
Completers 6-Year Trend
(2008-2014)
60
75
65
78
56
66
Completion Rates
Accreditors want to know our completion
rates:
• How long does it take students to
graduate from our programs?
Program completion data for last 6 years
• (Office of Institutional Research).
• 1 year = 2, 3, or 4 semesters
• 2 years = 5, 6, or 7 semesters
Completion Rates in Years
(2008-2014)
Degree
Level
Mode Mean
N
Elementary Education Bach
4
3.89
54
Elementary Education MAT
6
5.00
15
Secondary Education
Bach
3
3.60
5
Secondary Education
MAT
3,5
4.00
11
Curriculum & Instruct MA
Curriculum & Instruct EdS
Curriculum & Instruct Doct
Completion Rates in Years
(cont’d, 2008-2014)
Degree
Level
Mode Mean
N
Clin Ment Hlth Couns MA
2
2.62
21
School Counseling
MA
2
2.56
18
Spec Educ/Learn Dis
MS
2,3
2.50
6
School Psychology
EdS
3
3.21
39
Educ Psychology
MA
2
2.45
22
Educ Psychology
PhD
5,6,7
6.00
3
Counsel Psychology
PhD
8,9 10.17
6
Completion Rates in Years
(cont’d, 2008-2014)
Degree
Level
Mode Mean
N
Educational Leader
MA
1,3
2.70
10
Educational Leader
EdS
8
8.00
1
Educational Leader
PhD
3
3.00
1
Educational Leader
GCert
1
1.00
1
Higher Educ Admin
MA
2,3
2.33
9
Higher Educ Admin
PhD
6
6.00
1
Completion Rates in Years
(cont’d, 2008-2014)
Degree
Level
Mode Mean
N
Leadership
GCert
1
1.00
1
Leadership
MA
4
4.44
9
Leadership
Doct
5
7.67
76
Candidate Assessment
• MTTC Scores
• Course Grades (CAS Courses)
• Course Outcomes (SED Course
Rubric Data)
• Conceptual Framework Outcomes
(SED Courses)
MTTC Scores
• Michigan Test of Teacher Certification
(subject area tests)
• Reported annually by MDE
• 3-year aggregated scores (2010-2013)
• Reporting only scores on those tests
with >/= 10 test takers
• Comparing Andrews with all
Michigan test takers
MTTC Content Area Scores
3-Year Aggregated (2010-2013)
English
ESL
Lang Arts
Spanish
Social Stud
Elem Educ
Guid Couns
ALL TESTS
Andrews
Michigan
N
N
12
11
13
10
13
31
12
154
% Pass
91.7
81.8
84.6
90.0
76.9
96.8
91.7
90.9
1,849
538
2,717
610
2,558
7,592
404
32,828
% Pass
88.9
90.0
79.1
91.1
74.3
96.0
86.9
88.3
Course Grades
College of Arts & Sciences Courses
taken by Education students
•
•
Using 5-point Likert scale
3 = Satisfactory (at least C+)
Course Grades (2013-2014)
CAS Discipline
Biology
Communication
Economics
English
History
Integrated Science
International
Languages
Mathematics
Physics
Political Science
Visual Arts
N
% at 3-5
Mean
St Dev
0.700
1.400
0.500
1.116
1.400
0.000
7
10
2
29
7
6
100%
80%
100%
90%
86%
100%
4.29
3.80
4.50
4.17
4.43
5.00
7
100%
4.86 0.350
15
12
8
9
87%
92%
75%
100%
4.13
4.42
3.38
4.67
1.024
0.954
1.111
0.471
Course Learning
Outcomes
• Teaching, Learning, & Curriculum
• Initial Teacher Education
• Curriculum & Instruction
• Foundations
• Graduate Psychology & Counseling
• Leadership
• Educational Leadership (K-12)
• Higher Education Administration
• Leadership
TLC—Teacher Ed
EDTE Rubric
165/
630
165/
630
228
Portfolio
Personal Philosophy
Paper
Clinical Observation &
Interaction
228 Clinical Observation &
Interaction (Revised)
N
% at 3-5
15
Mean
87%
4.25
9 100%
4.59
13
94%
4.59
12
99%
4.90
TLC—Teacher Ed (cont’d)
EDTE Rubric
408
408
408
408
408
408
408
408
408
Micro-Teaching
1: Worldview
2: Hum Grow & Change
3: Grps, Leader, Change
4: Commun, Technology
5: Research & Evaluat
6: Pers & Prof Growth
7: Content Area Knowl
Port. Intro & Closing
N
% at 3-5
25 100%
25 100%
25 98%
24 100%
25 96%
25 84%
25 90%
24 81%
25 96%
Mean
4.36
4.34
4.25
4.23
4.18
3.95
4.05
3.85
4.51
TLC—Teacher Ed (cont’d)
EDTE Rubric
420 Literacy Intervention
Strategies
476 Final Test Technology
480 Philosophy of
Classroom Management
480 Classroom Management
Plan
N
13
% at 3-5
Mean
96%
4.73
14 98%
24 100%
4.71
4.87
25
4.76
97%
TLC—Teacher Ed (cont’d)
EDTE
Rubric
488/588 Summative Eval by
AU Super (General)
488/588 Summative Eval by
AU Super (English)
488/588 Summative Eval by
AU Super (Math)
488/588 Summative Eval by
AU Super (Social St)
N
% at 3-5
Mean
20
98%
4.30
3
96%
4.15
2 100%
4.05
1 100%
4.44
TLC—Curriculum &
Instruction
EDCI
Rubric
547
730
889
Curriculum Foundations
Theoretical Framework
Project Evaluation
N
% at 3-5
1 100%
5 100%
2 100%
Mean
4.60
4.57
5.00
TLC—Curriculum &
Instruction (cont’d)
EDCI
Rubric
695
695
695
695
695
695
695
Port 1: Christ Philosoph
2: Learning Theorist
3: Servant Leader
4: Effective Commun
5: Capable Researcher
6: Lifelong Learner
7: Subj Matter Expert
N
% at 3-5
1
1
1
1
1
1
1
100%
100%
100%
100%
100%
100%
100%
Mean
4.80
4.25
5.00
4.50
5.00
5.00
5.00
TLC—Curriculum &
Instruction (cont’d)
EDCI
Rubric
895
895
895
895
895
895
895
Port 1: Christ Philosoph
2: Learning Theorist
3: Servant Leader
4: Effective Commun
5: Capable Researcher
6: Lifelong Learner
7: Subj Matter Expert
N
% at 3-5
2
2
2
2
2
2
2
100%
100%
100%
100%
100%
100%
100%
Mean
5.00
4.50
5.00
5.00
4.50
5.00
5.00
Foundations
Rubric
EDFN
500
EDRM
505
505
505
505
EDRM
636
Personal &
Profession Synthesis
Final Article
Evaluation
Lit Review Evaluatn
Proposal Evaluation
Resrch Design Meth
Program Evaluation
N
% at 3-5
Mean
33
91%
3.88
30
92%
4.37
28 100%
29 92%
30 94%
9 100%
4.65
4.21
4.25
5.00
GPC
GDPC
Rubric
514
520
Worldview Paper
Critical Review of HD
Theory
Contemporary Issues in
the Media
Journal Article Review
PBS Website Review
WISC IV Mastery
520
525
525
652
N
% at 3-5
Mean
14 100%
21 91%
4.34
4.37
20 100%
9.26*
8 100%
4 100%
11 80%
4.72
4.69
3.91
*This assessment used a 10-point rubric.
GPC (cont’d)
GDPC
Rubric
654
654
654
654
654
654
654
654
654
654
Eval of Practicum 2.1
Eval of Practicum 2.2
Eval of Practicum 2.3
Eval of Practicum 2.4
Eval of Practicum 2.5
Eval of Practicum 2.6
Eval of Practicum 2.7
Eval of Practicum 2.8
Eval of Practicum 2.9
Eval of Practicum 2.10
N
% at 3-5
1
1
1
1
1
1
1
1
1
1
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
Mean
4.95
4.89
4.89
4.89
5.00
4.88
5.00
5.00
5.00
5.00
GPC (cont’d)
GDPC
Rubric
810
810
810
810
810
810
810
810
810
810
School Psych Port 2.1
School Psych Port 2.2
School Psych Port 2.3
School Psych Port 2.4
School Psych Port 2.5
School Psych Port 2.6
School Psych Port 2.7
School Psych Port 2.8
School Psych Port 2.9
School Psych Port 2.10
N
% at 3-5
5
5
5
5
5
5
5
5
5
5
99%
97%
100%
100%
100%
100%
99%
100%
100%
100%
Mean
4.36
4.25
4.07
4.02
4.28
3.97
4.48
4.42
4.16
4.49
GPC (cont’d)
GDPC
Rubric
N
School Psych Portfolio
Evaluation
Counseling Psychology
Dispositions
*This assessment used a 4-point rubric.
% at 3-5
Mean
21 100% 3.09*
21
99%
4.14
LEAD—Ed Lead (K-12)
EDAL
Rubric
520
520
560
560
565
565
570
570
570
Vision Statement
Final Project
Case Study
School Board Meeting
Spiritual Goals
Worldview Paper
Case Study Evaluation
Observation
Profile
N
*This assessment used a 10-point rubric.
6
7
8
7
7
7
6
5
7
% at 3-5
Mean
83%
100%
83%
86%
100%
100%
96%
100%
100%
4.00
4.89
4.08
4.46
4.86
4.60
9.63*
5.00
5.00
LEAD—Leadership
Portfolio
Philosophical Foundations
Ethics, Values, & Spiritual
Learning & Human Devel
Effective Communication
Mentor/Coach
Social Responsibility
Resource Development;
Human & Financial
Legal & Policy Issues
N
17
17
17
17
17
17
17
% at 3-5
Mean
100%
100%
100%
100%
100%
100%
100%
3.59
3.71
4.06
4.00
4.12
3.76
3.82
17 100%
3.65
LEAD—Leadership (cont’d)
Portfolio
Organizational Behavior,
Development, & Culture
Implementing Change
Evaluation & Assessment
Reading & Evaluating
Research
Conduct Research
Reporting &
Implementing Research
[Individual Choice]
N
% at 3-5
Mean
17 100%
3.94
17 100%
17 100%
17 100%
3.76
3.59
3.94
17 100%
17 100%
3.82
3.88
17 100%
3.88
LEAD—Leadership—Brazil
Portfolio
Philosophical Foundations
Ethics, Values, & Spiritual
Teamwork
Learn, Mentor & Human Devel
Intercultural Comm & Global
Social Responsibility
Resourse Dev; Human & Finan
Organizational Devel & Chng
Organizational Behavior
Evaluat & Conduct Research
N
36
36
36
36
36
36
36
36
36
36
% at 3-5
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
Mean
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
3.00
Conceptual Framework
Outcomes
Currently unable to access data
• LiveText glitch
• Developers are working on a
solution
Program and Faculty
Assessment
• Senior Surveys
• Course Evaluations
Senior Surveys
Program/faculty (re: “your major”) were
rated similarly by Elementary and
Secondary candidates:
• Sec = Spanish, English, Music, Art
• > 4.00/5.00 mean rating on 13 out of
16 indicators
Differences between Elementary and
Secondary on 3 indicators that scored
below 4.00:
Senior Surveys (cont’d)
Program and Faculty differences:
Indicator
Elem
Advanced course and program
offerings had sufficient depth of
subject matter.
3.93
4.50
(n=14)
(n=8)
4.14
3.71
(n=14)
(n=7)
There were adequate facilities
4.50
and specialized equipment
(n=14)
(studios, computers, instruments,
lab supplies, etc.).
3.25
Library resources were adequate
for the program.
Sec
(n=8)
All
Seniors
4.13
3.89
3.81
Senior Surveys (cont’d)
Seek, Affirm, Change:
“How much has your experience at
Andrews University prepared you to…”
• SED mean rating >/= All Seniors on
11 out of 14 indicators
Again, differences between Elementary
and Secondary:
Senior Surveys (cont’d)
Seek, Affirm, Change differences:
Indicator
Elem
Affirm: Embrace a balanced 3.57
(n=14)
lifestyle
Affirm: Engage in creative
3.71
(n=14)
problem solving and
innovation
Change: Engage in generous 3.64
service to meet human needs (n=14)
Sec
4.25
All
Seniors
3.73
(n=8)
4.13
3.93
(n=7)
4.13
(n=8)
3.90
Senior Surveys (cont’d)
Spiritual Commitment:
• SED mean rating >/= All Seniors on
all 15 indicators
• Negligible differences between
Elementary and Secondary
candidates.
Senior Surveys (cont’d)
Comments about faculty:
• “The advisor was extremely helpful, and
the teachers were clearly looking out for the
best interest of the students.”
• “The enthusiasm, professionalism, and
commitment of the teachers of this
department kept the students engaged,
involved, and learning at all times.”
Senior Surveys (cont’d)
Comments about faculty:
• “High expectations allowed us to reach
higher standards.”
• “The teachers were all highly
knowledgeable in the content area.”
• “The professors are helpful and caring.”
• “The content and methods are taught well.”
• “First Days of School Experience is a
definite strength.”
Course Evaluations
• Conducted across University in
every class
• Evaluation data are reported for
all SED and disaggregated by
department
• “The Course” section
• “The Instructor” section
• “Overall Rating” section
Course Evaluations
All SED
Indicators
Global Index
The Course
The Instructor
Overall Rating
N = 962
Mean
4.15
4.24
4.25
3.97
St Dev
0.96
0.89
0.95
1.04
Course Evaluations—SED
Highest: “The instructor was
sensitive to and respectful of all
people.”
• Mean = 4.42
Lowest: “Timely, thoughtful, and
helpful feedback was provided on
tests and other work.”
• Mean = 4.07
Course Evaluations
TLC
Indicators
Global Index
The Course
The Instructor
Overall Rating
N = 285
Mean
St Dev
4.16
4.21
4.29
3.99
0.96
0.90
0.93
1.06
Course Evaluations—TLC
Highest: “The instructor helped me
to understand the course content
from a Christian perspective.”
• Mean = 4.45
Lowest: “Timely, thoughtful, and
helpful feedback was provided on
tests and other work.”
• Mean = 4.09
Course Evaluations
GPC
Indicators
Global Index
The Course
The Instructor
Overall Rating
N = 527
Mean
4.17
4.27
4.26
3.99
St Dev
0.95
0.88
0.96
1.01
Course Evaluations—GPC
Highest: “The instructor was
sensitive to and respectful of all
people.”
• Mean = 4.42
Lowest: “The instructor helped me
to understand the course content
from a Christian perspective.”
• Mean = 3.99
Course Evaluations
LEAD
Indicators
Global Index
The Course
The Instructor
Overall Rating
N = 304
Mean
4.19
4.27
4.28
4.03
St Dev
0.84
0.76
0.83
0.94
Course Evals—LEAD
Highest: “The instructor was
sensitive to and respectful of all
people.”
• Mean = 4.50
Lowest: “Timely, thoughtful, and
helpful feedback was provided on
tests and other work.”
• Mean = 4.12
Course Evaluations—
Comparison Across SED
Indicators
SED
TLC
GPC LEAD
Global Index
4.15
4.16
4.17
4.19
The Course
4.24
4.21
4.27
4.27
The Instructor
4.25
4.29
4.26
4.28
Overall Rating
3.97
3.99
3.99
4.03
Recommendations
• Based on the data we’ve just
examined, what should we do?
• What changes should we consider?
• Let’s “close the loop.”
Seven Recommendations
1. We need to get access to Conceptual
Framework outcome data.
•
Continue to work with LiveText to
retrieve it.
2. We should audit the adequacy of our facilities
and equipment (Senior Survey).
•
Space, computers, technology, etc.
Senior Surveys (cont’d)
Program and Faculty differences:
Indicator
Elem
Advanced course and program
offerings had sufficient depth of
subject matter.
3.93
4.50
(n=14)
(n=8)
4.14
3.71
(n=14)
(n=7)
There were adequate facilities
4.50
and specialized equipment
(n=14)
(studios, computers, instruments,
lab supplies, etc.).
3.25
Library resources were adequate
for the program.
Sec
(n=8)
All
Seniors
4.13
3.89
3.81
Recommendations (cont’d)
3. We should examine how we can help our
students improve their experience related to
(from Senior Survey):
•
•
•
“embrace a balanced lifestyle.”
“engage in creative problem solving and
innovation.”
“engage in generous service to meet
human needs.”
Senior Surveys (cont’d)
Seek, Affirm, Change differences:
Indicator
Elem
Affirm: Embrace a balanced 3.57
(n=14)
lifestyle
Affirm: Engage in creative
3.71
(n=14)
problem solving and
innovation
Change: Engage in generous 3.64
service to meet human needs (n=14)
Sec
4.25
All
Seniors
3.73
(n=8)
4.13
3.93
(n=7)
4.13
(n=8)
3.90
Recommendations (cont’d)
3. We should examine how we can help our
students improve their experience related to
(from Senior Survey):
•
•
•
“embrace a balanced lifestyle.”
“engage in creative problem solving and
innovation.”
“engage in generous service to meet
human needs.”
4. We should examine and verify completion data.
•
CAEP will ask for it.
Completion Rates in Years
(2008-2014)
Degree
Level
Mode Mean
N
Elementary Education Bach
4
3.89
54
Elementary Education MAT
6
5.00
15
Secondary Education
Bach
3
3.60
5
Secondary Education
MAT
3,5
4.00
11
Curriculum & Instruct MA
Curriculum & Instruct EdS
Curriculum & Instruct Doct
Completion Rates in Years
(cont’d, 2008-2014)
Degree
Level
Mode Mean
N
Clin Ment Hlth Couns MA
2
2.62
21
School Counseling
MA
2
2.56
18
Spec Educ/Learn Dis
MS
2,3
2.50
6
School Psychology
EdS
3
3.21
39
Educ Psychology
MA
2
2.45
22
Educ Psychology
PhD
5,6,7
6.00
3
Counsel Psychology
PhD
8,9 10.17
6
Completion Rates in Years
(cont’d, 2008-2014)
Degree
Level
Mode Mean
N
Educational Leader
MA
1,3
2.70
10
Educational Leader
EdS
8
8.00
1
Educational Leader
PhD
3
3.00
1
Educational Leader
GCert
1
1.00
1
Higher Educ Admin
MA
2,3
2.33
9
Higher Educ Admin
PhD
6
6.00
1
Completion Rates in Years
(cont’d, 2008-2014)
Degree
Level
Mode Mean
N
Leadership
GCert
1
1.00
1
Leadership
MA
4
4.44
9
Leadership
Doct
5
7.67
76
Recommendations (cont’d)
5. We must re-evaluate our assessment system and
calendar to ensure compliance with University.
•
Data in LiveText; analysis in Weave
6. We must ensure that we are getting data from all
programs—for CAEP, HLC, other accreditors,
and our own improvement.
7. We need to “re-invest” in LiveText.
•
Are we using it to potential for our needs?
•
LiveText refresher training (especially for
new faculty and staff).
Thank You!