OUTCOMES ASSESSMENT:
PROCESS FOR ASSESSING ONCE AT THE
COURSE LEVEL FOR PROGRAM AND
INSTITUTIONAL LEVEL ASSESSMENTS
Prepared and Presented by Keith Wurtz
Dean, Institutional Effectiveness, Research & Planning
Crafton Hills College
PRESENTATION OBJECTIVES
 Review the Crafton Hills College Assessment Process
 Understanding of the services provided by the Office of Institutional
Effectiveness, Research & Planning
 Understanding of how assessing at the course level will provide
assessment data that can be used at the course level, program level, and
institutional level
CRAFTON HILLS COLLEGE
ASSESSMENT PROCESS
1.
Outcomes
Statement
CRAFTON HILLS COLLEGE
ASSESSMENT PROCESS
1.
Outcomes
Statement
2. Means of
Assessment
CRAFTON HILLS COLLEGE
ASSESSMENT PROCESS
1. Outcomes
Statement
2. Means of
Assessment
3. Criteria
(Benchmark)
CRAFTON HILLS COLLEGE
ASSESSMENT PROCESS
1. Outcomes
Statement
2. Means of
Assessment
4. Summary
of Evidence
3. Criteria
(Benchmark)
CRAFTON HILLS COLLEGE
ASSESSMENT PROCESS
1. Outcomes
Statement
5.Use of
Results
4. Summary
of Evidence
2. Means of
Assessment
3. Criteria
(Benchmark)
ROLE OF THE OFFICE OF INSTITUTIONAL
EFFECTIVENESS, RESEARCH & PLANNING
1.
Outcome Statement

OIERP can help you with developing outcomes statements (e.g.: sounding board,
etc.)

Outcome needs to be something that you will find informative

What do you want students to know and/or be able to do
ROLE OF THE OFFICE OF INSTITUTIONAL
EFFECTIVENESS, RESEARCH & PLANNING
2.
Means of Assessment

Help with developing an approach for how to assess outcome

Will create a process for collecting and processing data

Create survey, scan survey, and enter into database

Enter data into e-Lumen

Provide summary results in table format to inform step 4, Summary of Results
ROLE OF THE OFFICE OF INSTITUTIONAL
EFFECTIVENESS, RESEARCH & PLANNING
3.
Criteria (Benchmark)

Help with setting benchmark
ROLE OF THE OFFICE OF INSTITUTIONAL
EFFECTIVENESS, RESEARCH & PLANNING
4.
Summary of Results

Provide summary results in table format
ROLE OF THE OFFICE OF INSTITUTIONAL
EFFECTIVENESS, RESEARCH & PLANNING
5.
Use of Results

Discuss the results of the data and help to understand what they might indicate
and how they might inform the program
MANAGING DATA
EXAMPLE 1
Program Level Outcome 1:
Course
Apply knowledge of ASL and Deaf Culture beyond the classroom in
interactions within multicultural communities and the world.
Course SLO 1:
Apply knowledge of ASL and Deaf Culture beyond the classroom in
interactions within multicultural communities and the world.
ASL-101
4 Point Rubric
ASL-102
4 Point Rubric
ASL-103
4 Point Rubric
ASL-104
4 Point Rubric
ASL-1
 SLO 1: Apply knowledge of ASL and Deaf Culture beyond the classroom in
interactions within multicultural communities and the world.
Rubric
Value
Rubric Description
#
%
0
Attended no events.
12
8.6
1
Student attended fewer Deaf
Community/cultural events than required.
20
14.3
2
Attended the required number of events.
99
70.7
3
Student attended more Deaf
Community/cultural events than required.
Total
9
6.4
140
100.0
Combined
%
22.9%
77.1%
100.0%
ASL-2
 SLO 1: Apply knowledge of ASL and Deaf Culture beyond the classroom in
interactions within multicultural communities and the world.
Rubric Rubric Description
Value
#
%
0
Attended no events.
6
7.3
1
Student attended fewer Deaf
Community/cultural events than required.
11
13.4
2
Attended the required number of events.
38
46.3
3
Student attended more Deaf
Community/cultural events than required.
27
32.9
Total
82
100.0
Combined
%
20.7
79.3
100.0
ASL-3
 SLO 1: Apply knowledge of ASL and Deaf Culture beyond the classroom in
interactions within multicultural communities and the world.
Rubric Rubric Description
Value
#
%
0
Attended no events.
3
10.3
1
Student attended fewer Deaf
Community/cultural events than required.
2
6.9
2
Attended the required number of events.
24
82.8
3
Student attended more Deaf
Community/cultural events than required.
0
0.0
Total
29
100.0
Combined
%
17.2
82.8
100.0
ASL-4
 SLO 1: Apply knowledge of ASL and Deaf Culture beyond the classroom in
interactions within multicultural communities and the world.
Rubric Rubric Description
Value
#
%
0
Attended no events.
0
0.0
1
Student attended fewer Deaf
Community/cultural events than required.
1
4.5
2
Attended the required number of events.
7
31.8
3
Student attended more Deaf
Community/cultural events than required.
14
63.6
Total
22
100.0
Combined
%
4.5
95.5
100.0
ASL-1, 2, 3, AND 4
 PLO (SAO) 1: Apply knowledge of ASL and Deaf Culture beyond the
classroom in interactions within multicultural communities and the world.
Rubric Rubric Description
Value
#
%
0
Attended no events.
21
7.7
1
Student attended fewer Deaf
Community/cultural events than required.
34
12.5
2
Attended the required number of events.
168
61.5
3
Student attended more Deaf
Community/cultural events than required.
50
18.3
Total
273
100.0
Combined
%
20.2
79.8
100.0
MANAGING DATA
EXAMPLE 2
Program Level Outcome 2: Accurately identify vocabulary,
comprehend passages assigned, interpret meaning, and sentences
follow ASL word order and prepositions are not used.
SLO 2: ASL Receptive: Students can
accurately identify vocabulary,
comprehend passages signed to
them, and interpret the meaning into
written English.
SLO 3: Word Order/Classifiers:
Sentences/story follow ASL word order
according to corresponding ASL level and
English articles and prepositions are NOT
used. For ASL 103/4: Classifiers are used
to enhance the story.
ASL-101
5 Point Rubric
5 Point Rubric
ASL-102
5 Point Rubric
5 Point Rubric
ASL-104
5 Point Rubric
5 Point Rubric
Course
ASL-1, 2, AND 4
 PLO 2: Accurately identify vocabulary, comprehend passages assigned,
interpret meaning, and sentences follow ASL word order and prepositions
are not used.
Rubric Rubric Description
Value
#
%
0
No demonstrated achievement
39
7.7
1
Minimal evidence of achievement – below
expectations
55
10.8
2
Adequate evidence of achievement – met stated
outcome or expectations
163
32.1
3
Adequate evidence of achievement – met stated
outcome or expectations
181
35.6
Significant evidence of achievement – surpassed
stated outcome, mastery or near mastery of
learning expectations
70
13.8
Total
508
100
4
Combined
%
18.5
81.5
100.0
CAN COMBINE ACROSS DISCIPLINES IF
HAVE THE SAME NUMBER OF RUBRIC
LEVELS
 ASL-103 has four rubric levels (0-3)
 SPAN-101, 102, and 103 has four rubric levels (1-4)
Rubric
Value
Broad Rubric Description
#
%
0,1
No demonstrated achievement
22
2.2
1,2
Minimal evidence of achievement – below
expectations
61
6.0
2,3
Adequate evidence of achievement – met
stated outcome or expectations
304
30.1
3,4
Significant evidence of achievement –
surpassed stated outcome, mastery or near
mastery of learning expectations
Total
Combined
%
8.2
91.8
624
61.7
1,011
100
100.0
INSTITUTIONAL LEARNING OUTCOMES
 Adopting the same number of rubric levels will enable Crafton to only
do one assessment at the course level to assess progress at both the
Program and Institutional Levels
COURSES WITH 4 LEVEL RUBRIC
 17 courses had an assessment with a 4 level rubric in Fall 2012 and/or
Spring 2013
ASL103
MICRO102
RESP138X4
CHC100
MICRO150
SPAN101
ENGL101
MICRO247X4
SPAN102
ENGL102
MICRO248X4
SPAN103
MATH095
RESP050
SPEECH100
MATH102
RESP136
ILOS ASSESSED IN THOSE COURSES
 Four ILOs were assessed in the 17 courses
Course
ASL103
CHC100
ENGL101
ENGL101H
ENGL102
ENGL102H
MATH095
MATH102
MICRO102
MICRO150
MICRO247X4
MICRO248X4
RESP050
RESP136
RESP138X4
SPAN101
SPAN102
SPAN103
SPEECH100
SPEECH100H
1 Critical Thinking
(8 Courses)
0
36
0
0
132
1
421
189
0
0
44
24
0
31
90
0
0
0
0
0
ILO
2 Written and
Oral
Communication
(9 Courses)
116
18
0
0
0
0
0
95
0
0
0
0
150
62
0
563
200
45
702
15
4 Society and
Culture
(1 Course)
87
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5 Information
Literacy
(7 Courses)
0
0
1436
59
132
1
0
0
170
104
0
0
0
31
0
0
0
0
235
5
ASSESSMENT OF WRITTEN AND ORAL
COMMUNICATION ILO
 Data for the 9 courses is provided in table below where Written and Oral
Communication was assessed in Fall 2012 and Spring 2013
Rubric Broad Rubric Description
Value
#
%
0
No demonstrated achievement
83
4.2
1
Minimal evidence of achievement – below
expectations
386
19.6
2
Adequate evidence of achievement – met
stated outcome or expectations
919
46.7
3
Significant evidence of achievement –
surpassed stated outcome, mastery or
near mastery of learning expectations
Total
Combined
%
23.8
76.1
578
29.4
1,966
100
100.0
NEXT STEPS
 Campus needs to agree on number of rubric levels
 Each faculty and department can define each level in the rubric in the
way that is most appropriate for their area/discipline/course
 Assessment only occurs at the course level and OIERP provides data in
table format at the course, program, and institutional levels
 Programs work with OIERP to develop methods of assessment where
OIERP is compiling data into database (eLumen, SPSS, surveys, scannable
forms, etc.)