Assessment of Student Learning
Materials for Convening Groups
Assessment: Assessment seeks to determine if students have learned what our learning
experiences are designed for them to learn and if the improvements that we have made to
learning experiences (based on assessments) have had the desired effects. To make this
determination, an academic program responds to three interrelated questions:
●
What should our students know or be able to do? To answer this question, a
program lists its learning outcomes.
● Where do students learn what we hope they will know or be able to do? To
answer this question, a program maps its outcomes to its curriculum.
● How do we know that they have learned what we hope they will know or be able
to do? To answer this question, a program assesses student learning.
● How do we know that improvements have had desired effects? To answer this
question, a program reassesses the relevant outcome and compares past with
present assessment findings.
Assessment Plan: Academic programs (i.e., majors, minors, and graduate programs)
submit draft assessment plans to their school assessment coordinators by September 15.
By October 1, programs submit their final plans to their assessment coordinators, copied
to their deans (for review) and to the vice provost (for archiving). A plan includes:



A (revised) list of student learning (goals and) outcomes;
A (revised) curriculum map;
A plan for the academic year (outcomes, measures, and achievement targets).
Assessment Report: Programs submit assessment reports on June 1. A report includes, in
addition to the items in the plan:



(Comparative) findings for the measures;
Current actions for unmet targets;
Report on past actions (if applicable).
Convening Group Responsibilities:
●
●
●
●
●
1
Review academic program (goals and) outcomes and align them to institutional
learning goals;
Map program (goals and) outcomes to program curricula;
Implement assessments of academic programs, review results of these
assessments, and use them for improvement;
Work with GECCo, SACs, and Library to implement assessments of general
education and information literacy as needed; and
Submit assessment plans and reports for peer review by SACs and for archiving.
Student Learning Outcomes
Student Learning Outcome: A student learning outcome (sometimes called an objective)
is a succinct statement of what you would like students to know or be able to do. For
example: Culinary arts majors will be able to describe the chemical reaction of foods. An
outcome always focuses on outputs (what students will know or be able to do at the end),
not inputs (what an instructor or a course will do). We can assess outputs at the end of a
learning experience, at the end of a course, at different points of a program, or at
graduation. Typically, program assessment, which you are conducting, examines student
learning at different points of a program. Finally, a student learning outcome addresses
student learning, not program effectiveness. For instance, assessing what students have
learned at the end of their internships is different from tracking the number of student
internships.
Goals: A learning outcome should not be confused with a learning goal, which is a more
general statement about student learning. For example: Culinary arts majors will
understand the fundamentals of food science. Some programs may wish to establish only
outcomes without goals. In any case, a program assesses student learning at the outcome
level.
Helpful suggestions and guides:
●
●
●
●
●
2
Bloom’s Taxonomy may help you think about the domains and levels of student
learning. A simple internet search will bring up many relevant sites.
Lists of strong action verbs may help you formulate your outcomes. For example:
http://www.acu.edu/academics/adamscenter/course_design/syllabus/verbs.html.
You may conceivably list dozens of outcomes for your program—but a long list is
probably not practical for assessment purposes. Consider focusing on a short set
of outcomes. You do not need to assess all your outcomes every year. Instead,
create a multi-year assessment cycle in which you assess a couple of outcomes
each year.
It is often tempting to list several skills or knowledge areas in a single outcome
(usually linked by commas). However, you may later find it difficult to assess a
multi-pronged outcome.
Every course syllabus in your program should include the outcome(s) that your
program’s curriculum map (see below) aligned with that course in addition to any
other outcomes that the instructor wishes to add.
Name of Academic Program
Student Learning (Goals and) Outcomes
Goal:
Outcome 1 (year assessed):
Outcome 2 (year assessed):
Goal:
Outcome 3 (year assessed):
Goal:
Outcome 4 (year assessed):
Outcome 5 (year assessed):
Outcome 6 (year assessed):
Etc.
3
Curriculum Map
Curriculum Map: A curriculum map is a visual tool showing the alignment between
outcomes and courses. Typically, a curriculum map is a matrix with courses arranged
vertically and outcomes arranged horizontally (or the reverse). Assume that a culinary
arts program has the following outcomes:
1.
2.
3.
Students will be able to describe the chemical reaction of foods.
Students will be able to create an original recipe.
Students will be able to apply standard food-safety regulations.
The following map simply shows which courses treat which outcomes:
CA 234
CA 345
CA 456
Outcome 1
X
X
Outcome 2
X
X
Outcome 3
X
X
X
Sometimes we want to know not only where students receive opportunities to acquire
knowledge and skills but also whether the courses introduce skills, allow students to
practice those skills, or give them opportunities to show mastery. (Other ways to phrase
those developmental levels are possible.) This map uses codes (I=introduce, P=practice,
and M=master) to indicate the levels:
CA 234
CA 345
CA 456
Outcome 1
I
M
Outcome 2
P
P
Outcome 3
I
P
M
Practical Uses of a Curriculum Map: A map will tell you immediately where you should
assess an outcome. For instance, the first map above tells you that you should go to CA
234 and CA 456 to assess Outcome 1. The second map suggests an assessment project
that measures student progress on Outcome 1 from CA 234 to CA 456. A map also can
provide you information about your curriculum. For example, according to the second
map, students are introduced to Outcome 1 in CA 234 and given the chance to show
mastery in CA 456, but they have no opportunities to practice this skill between the two
courses. In this arrangement, students may lose the skill from sophomore to senior year.
Moreover, students practice Outcome 2 in two courses—but the skill is never introduced.
Helpful suggestions and guides:
The next page provides a template for a curriculum map similar to the second map above.
You may want to use course symbols and numbers as well as outcome numbers
(corresponding to your outcome list) to simplify the map.
4
Name of Academic Program
Curriculum Map
Outcome 1
CRSE 012
CRSE 345
CRSE 678
CRSE 901
I = introduce
P = practice
M = master
5
Outcome 2
Outcome 3
Outcome 4
Assessment Plan Template
The assessment plan includes three parts:
1. A list of all current (goals and) outcomes followed by the year that each was last
assessed and the year that each will next be assessed.
2. A current curriculum map aligning outcomes to courses.
3. Specific plans for this academic year, using the template on the next page of this
document. Although you should feel free to adapt and expand the template to suit
the needs of your program, please make sure that the plan for this year includes
the following:
●
Outcomes
● Assess at least two outcomes.
●
Measures
● Use at least two measures for each outcome.
● Include at least one direct measure for each outcome.
● Include brief answers to the following questions in the paragraphs about
the measures:
o
o
o
o
o
o
o
●
Who will assess the outcome (e.g., faculty not teaching the
course)?
What instrument or method will they use (e.g., rubric)?
What student product will they use (e.g., final exam question)?
What assessment process or methodology will they use (e.g., blind
review after an inter-rater reliability session)?
When will they measure (e.g., fall 2015)?
In what courses or which student populations will they measure
(e.g., CA 456)?
How many student products will they assess (e.g., 25% of the
total)?
Achievement Targets
● Set an achievement target for every measure.
Note: Some convening groups and schools have adapted the attached assessment
plan template to address specific disciplinary or accreditation needs, although the
basic information remains the same.
6
Name of Academic Program
Assessment Plan
Academic Year
(Prior to the specific plans for this academic year, provide your current goals and
outcomes and your current curriculum map. See above for more details.)
Goal:
Outcome 1:
Measure 1:
Achievement Target:
Measure 2:
Achievement Target:
Outcome 2:
Measure 1:
Achievement Target:
Measure 2:
Achievement Target:
7
Assessment Plan Example
As you design and finalize your assessment plan, consider the example on the next page.
This example follows the structure of the assessment template and includes the content
detailed in the documents on reminders and resources.
Please note the following about the example:
● For the sake of brevity, this example does not include the full list of goals and
outcomes and the curriculum map for this fictional program.
8
●
Also for the sake of brevity, this example includes no goals (in fact, some
programs move directly to the outcome level) and only one outcome.
●
One of the best uses of this example is to check your plan for all required
elements.
●
The example does not dictate the content of the plan. For example, while 75%,
which was used in past examples, may be an appropriate achievement target for
your program, do not feel limited to 75%. Different disciplines, course levels, and
measures demand different targets. Moreover, if an achievement target was met in
the past, it can be raised in the future. If an achievement was not met in the past, it
can serve as a baseline in the future.
Culinary Arts
Assessment Plan
2015-16
Outcome:
Students will be able to describe the chemical reaction of foods.
Measure 1:
The instructors of the two fall 2015 sections of CA 456 will embed
in the final exam a common question about the chemical reaction
of foods. Before grading the exams, the instructors will extract the
answers to the embedded question from a random selection of 25%
of the total number of exams and remove student identifiers. After
an inter-rater reliability session, a team of two other culinary arts
instructors will score the extracted answers on a rubric designed by
the culinary arts faculty.
Achievement Target: 65% of students will be able to describe the chemical
reaction of foods at a satisfactory level or higher.
Measure 2:
The instructors of the two fall 2015 sections of CA 456 will
administer short surveys at the end of the semester that ask all
students in both sections to reflect on their understanding of course
material, including the chemical reaction of foods. Faculty will
tabulate the results.
Achievement Target: 85% of students will report that they have a satisfactory
understanding of the chemical reaction of foods.
9
Assessment Report Template
The assessment report includes the following:
●
Outcomes
● Assess at least two outcomes.
●
Measures
● Use at least two measures for each outcome.
● Include at least one direct measure for each outcome.
●
Achievement Targets
● Set an achievement target for every measure.
●
Findings
●
●
●
●
●
Revise the original achievement target so that it reflects the actual finding.
Consider providing a table or chart that summarizes the actual results.
Briefly interpret the findings.
If you have assessed the outcome in the past and used comparable
measures, compare past and present findings.
Actions
Past Actions


If you assessed the outcome in the past, indicate which actions you
actually implemented from that past assessment.
Discuss whether, based on current findings, those actions had a positive
effect.
Current Actions
● Provide a specific action for each unmet achievement target.
● If you assessed the outcome in the past, indicate if you plan to continue,
discontinue, or enhance past actions.
● Use the document on suggested loop-closing strategies.
Note: Some convening groups and schools have adapted the attached assessment
plan template to address specific disciplinary or accreditation needs, although the
basic information remains the same. My e-mail will prompt the assessment
coordinators, as applicable, to distribute any customized templates.
10
Name of Academic Program
Assessment Report
2015-16
Goal:
Outcome 1:
Measure 1:
Achievement Target:
Findings:
Measure 2:
Achievement Target:
Findings:
Actions:
(Past Actions):
(Current) Actions:
Outcome 2:
Measure 1:
Achievement Target:
Findings:
Measure 2:
Achievement Target:
Findings:
Actions:
(Past Actions):
(Current) Actions:
11
Assessment Report Example
As you design and finalize your assessment report, consider the example on the next
page. This example follows the structure of the assessment template.
Please note the following about the example:
● For the sake of brevity, this example does not include the full list of goals and
outcomes and the curriculum map for this fictional program.
12
●
For the sake of brevity, this example includes no goals (in fact, some programs
move directly to the outcome level) and only one outcome.
●
One of the best uses of this example is to check your report for all required
elements.
●
The example does not dictate the content of the plan. For example, while 75%,
which was used in past examples, may be an appropriate achievement target for
your program, do not feel limited to 75%. Different disciplines, course levels, and
measures demand different targets. Moreover, if an achievement target was met in
the past, it can be raised in the future. If an achievement was not met in the past, it
can serve as a baseline in the future.
Culinary Arts
Assessment Report
2015-16
Outcome:
Students will be able to describe the chemical reaction of foods.
Measure 1:
The instructors of the two fall 2015 sections of CA 456 will embed
in the final exam a common question about the chemical reaction
of foods. Before grading the exams, the instructors will extract the
answers to the embedded question from a random selection of 25%
of the total number of exams and remove student identifiers. After
an inter-rater reliability session, a team of two other culinary arts
instructors will score the extracted answers on a rubric designed by
the culinary arts faculty.
Achievement Target: 65% of students will be able to describe the chemical
reaction of foods at a satisfactory level or higher.
Findings:
At least 75% of students scored at a satisfactory level or higher on
three of the four criteria.
2013
2015
unsatisfactor satisfactor excellent totals unsatisfactor satisfactory excellen totals
y
y
y
t
knowledge
of food
5
25%
10
50%
5
25%
20
100%
5
25%
5
25%
10
50%
20
100%
knowledge
of chemical
reactions
8
40%
6
30%
6
30%
20
100%
5
25%
8
40%
7
35%
20
100%
clarity
2
10%
10
50%
8
40%
20
100%
4
20%
6
30%
10
50%
20
100%
developmen
t
4
20%
12
60%
4
20%
20
100%
5
25%
10
50%
5
25%
20
100%
Findings indicate that students met the targets for all four areas. In addition,
since 2013, students have improved their knowledge of chemical reactions.
Measure 2:
13
The instructors of the two fall 2015 sections of CA 456 will
administer short surveys at the end of the semester that ask all
students in both sections to reflect on their understanding of course
material, including the chemical reaction of foods. Faculty will
tabulate the results.
Achievement Target: 85% of students will report that they have a satisfactory
understanding of the chemical reaction of foods.
Findings:
Fewer than 75% of students reported that they had a satisfactory
understanding of the chemical reaction of foods on all relevant
questions.
2013
unsatisfactory satisfactory
2015
totals
unsatisfactory satisfactory
totals
knowledge
of food
9
45%
11
55%
20
100%
8
40%
12
60%
20
100%
knowledge
of chemical
reactions
14
70%
6
30%
20
100%
6
30%
14
70%
20
100%
2015 survey results indicate that, compared to 2013 results, more students
believe that they have a satisfactory knowledge of chemical reaction of
foods, but students still missed the target.
Actions:
14
Past actions: After assessment findings in 2013 indicated that students did
not meet targets for knowledge of chemical reactions, the faculty decided
to revise CA 345 (pre-requisite to CA 456) to include opportunities to
reinforce students’ knowledge of chemical reactions (first introduced in
CA 234) before enrolling in CA 456 (course under assessment). Current
assessment findings for the direct measure suggest that the course revision
had a positive effect on students’ knowledge. Current action: Since
students met the target for the direct measure in 2015 but nevertheless
register some disconnect between their knowledge and their perception of
that knowledge, the faculty will continue to provide opportunities for
students to practice and demonstrate knowledge of chemical reactions in
CA 345 with the hope of bolstering student confidence in preparation for
another assessment in two years.
Take Action
Suggestions for Closing the Loop
Student-Learning Assessment
The most important part of any assessment cycle is closing the loop: the moment when
the convening group collectively discusses assessment findings, identifies ways to use the
findings to improve student learning and/or assessment methodology, and indicates next
steps in the actions field of the assessment report. Although steps to improve student
learning and methods to improve assessment methodology are both valid responses to
assessment findings, the convening group should always emphasize steps to improve
student learning—even if small, incremental ones—as improving student learning is the
primary purpose of student-learning assessment. Below you will find two sets of
recommendations for closing the loop. One set is an inventory of ways to close the loop
on student learning. The other set is an inventory of ways to close the loop on assessment
methodology.
Improving Student Learning
● course-level actions
o assignments
▪ Revise the assignment.
▪ Increase the number of assignments requiring demonstration of the
relevant skill in the same course.
o course delivery
▪ Devote additional class time to discussing the relevant content or
practicing the relevant skill.
▪ Change the method of delivering the content or practicing the skill.
o student feedback
▪ Give students a copy of the rubric, and/or ask them to rate
themselves and/or their peers using the rubric.
▪ Provide opportunities for students to give feedback on their
learning (e.g., surveys, focus groups, minute papers).
▪ Ensure the alignment from program outcome to course outcome to
course assignment and communicate that alignment to students
(e.g., in the syllabus) so that they understand the purpose of the
assignment.
o other
▪ Change textbooks or other classroom materials.
▪ Add, delete, or modify a course.
▪ Identify a common problem across all sections of a course that
delivers the outcome, but give instructors freedom to implement
different but complementary solutions.
● program-level actions
o Increase the number of assignments requiring demonstration of the
relevant skill across the program.
15
o Create a signature assignment for all sections of the course to ensure
common student knowledge.
o Create additional opportunities to develop the relevant skill across the
program.
o Change pre-requisites for this and/or other relevant courses or alter course
sequencing to scaffold learning.
o Revise the program.
o Add, delete, or modify a learning outcome.
● other actions
o Attend a relevant FRC or IDC workshop.
Improving Assessment Methodology
● Change or revise the measure.
o Change the embedded assignment for the direct assessment.
o Consider alternatives for the indirect assessment.
o Revise the indirect assessment.
● Prepare scorers.
o Conduct a calibration session to enhance inter-rater reliability.
o Include the assignment instructions along with the assignment in the
packet of materials given to raters.
● Other
o Reduce or increase the sample size.
o Revise the rubric.
o Alter the display of assessment findings.
o Revise the achievement target up or down.
16
Resources and Reminders
●
●
●
Visit the Assessment Website, which offers a wealth of resources:
http://www.ramapo.edu/assessment/. Of particular note:
o
An extensive planning packet for convening groups:
http://www.ramapo.edu/assessment/process/
o
Ramapo rubrics, assessment methods, and FAQ:
http://www.ramapo.edu/assessment/resources/
Get back to the basics:
o
Write outcomes that articulate what your program truly values in student
learning.
o
Think of assessment as a research project: What are you curious about?
o
Select measures likely to yield results that you can confidently use.
o
Set achievement targets that reflect your program’s standard for
acceptable performance.
Don’t forget all the basic components of the assessment plan:
o
Assess at least two outcomes every year.
o
Use at least two measures for each outcome.
o
Include at least one direct measure for each outcome.
o
Set an achievement target for each measure.
o
Include brief answers to these questions in the paragraphs about the
measures:
Who will assess the outcome (e.g., faculty not teaching the
course)?
o What instrument or method will they use (e.g., rubric)?
o What student product will they use (e.g., final exam question)?
o What assessment process or methodology will they use (e.g., blind
o
17
review after an inter-rater reliability session)?
o When will they measure (e.g., fall 2015)?
o In what courses or which student populations will they measure
(e.g., CA 456)?
o How many student products will they assess (e.g., 25% of the
total)?
●
Consider the full range of measures available to you.
o
The Middle States Commission on Higher Education offers an inventory
of measures on page 29 of this publication:
http://msche.org/publications/SLA_Book_0808080728085320.pdf.
Ramapo’s IR office has data from course evaluations, which include
questions about major learning outcomes. If you would like to review the
data about outcomes in the aggregate and use that data as indirect
measures, contact Gurvinder Khaneja at gkhaneja@ramapo.edu for the
data and your assessment coordinator for appropriate strategies.
o Ramapo’s IR office recently completed the Alumni 2014 Survey:
http://www.ramapo.edu/ir/survey-research/alumni-survey/ (The password
is ‘IR__PPDOCS.’) Tables 4.1-4.20 towards the end of the report give
alums’ responses concerning the importance of certain all-college
outcomes and the degree to which they felt that Ramapo helped them
achieve those outcomes. This data may serve as an effective indirect
measure.
o
18
Assessment Plan/Report Review
Checklist for SACs
Plan
_____ The plan includes a full list of current goals and outcomes with years assessed.
_____ The plan includes a current curriculum map.
_____ The program will assess at least two outcomes.
_____ Each outcome has at least two measures.
_____ Each pair of measures includes at least one direct measure.
_____ The description of each measure includes the following:
_____ who will assess the outcome (e.g., faculty not teaching the course)
_____ what instrument or method will they use (e.g., rubric)
_____ what student product will they use (e.g., final exam question)
_____ what assessment process will they use (e.g., blind review after an interrater reliability session)
_____ when will they measure (e.g., fall 2015)
_____ what courses or which student populations will they measure
_____ how many student products will they assess
_____ Each measure includes an achievement target.
19
Report
_____ Each measure includes findings (minimally the target rephrased as a finding).
_____ Each finding for an outcome assessed in the past includes a comparison between
past and present findings (if similar measures were used and comparisons can be
made).
_____ Each target not met includes a specific current action.
_____ Each action for an outcome assessed in the past includes a report on whether
actions from the previous assessment were actually implemented and whether
they appear to have had a positive effect.
20