Student Learning:
What Does Middle States Really
Want?
Most information was obtained from Linda
Suskie,
Vice President
Middle States Commission on Higher
Education
Slides and Information were used with
permission from the author
3624 Market Street, Philadelphia PA 19104
Web: www.msche.org E-mail: LSuskie@msche.org
Brief History of Assessment
(Joe)
What Does Middle States Want?
Organizing and Implementing
Assessment
What Should You Document?
• Clear statements of goals
• Organized, sustained assessment process
– Principles, guidelines, support
– What assessments are already underway
– What assessments are planned, when, & how
• Assessment results documenting progress
toward accomplishing goals
• How results have been used for improvement
“Shoulds”
• Assessment should be community effort
– Middle States wants all adjunct contracts to reflect their participation in
ongoing assessment
• Learning outcomes should result directly from the mission
statement of the program
– Learning outcomes may include goals, values, or skills
– Choose 2-3 different ones to assess each year (or period)
• Each area of assessment should include at least 2
measures—one of which must be DIRECT
• Assessment must include how results have been used for
change (close the loop)
• Save a few copies of student work:
– Exemplary, adequate, inadequate
• Keep assessment summaries useful to you and your
colleagues
“Shoulds Nots” and “Not
Necessaries”
• Assessment should NOT be used for
instructor evaluation
– Middle States wants assessment of classes
taught in multiple sections by multiple instructors
– All data from all sections should be pooled
• Middles States does not want all instructors
to teach alike or all instructors to give the
same exams and/or assignments; what they
want is to use the same questions for
assessment; Ex. Critical thinking
• Assessment does not have to be value-added
The Value of a Well-Designed
Rubric
• Easy to see where students are having
trouble
• Rubric must be explicit enough to allow
consensus
• Students can see more easily that they are
being fairly graded
• Can be used for grading, course-level
assessment, and program-level assessment
Levels of Assessment
Student Level
Assessment
Course Level
Assessment
Assesses
Assesses students as
individual students a whole on courselevel goals
Includes tests &
Aggregate across
assignments
students
Results in a grade Points out strengths
and weak areas of
course
Program Level
Assessment
Across students
as a whole on
program
Addresses
important goals
that faculty
decide on
Uses more than
one course
Why Not Use Grades?
• Grades are content driven
• There are times when grades can be used if
they are used over multiple courses with
different content
The Capstone Experience:
A Good Place For Program
Assessment
• Should be designed so students demonstrate
the goals of the department
• Students who have this experience learn to
synthesize material
• Include a direct method (e.g. standardized
test) and at least one other method (focus
group or survey)
Keep assessment summaries
short and simple.
• Fast and easy to read and understand
– Use short, simple charts, graphs, and lists.
• Use PowerPoint presentations.
• Avoid narrative text.
– First aggregate (sum up) data, then drill down into details
as needed.
– Round results.
– Sort results from highest to lowest.
– Percentages may be more meaningful than averages.
• Avoid complex statistics.
– As you collect results over time, show trends.
Tell a story.
• Key questions to address:
– What have you learned about your students’ learning?
– What are you going to do about what you have learned?
– When, where, and how are you going to do it?
• Focus on “big news.”
– Have a statistician identify meaningful vs. insignificant
differences.
• Find someone skilled at finding the stories in
reams of data.
Examples
(Joe)
Economics Electives Term Paper Guidelines
Term papers will be evaluated on the following criteria:
Does the paper have a cogent thesis?
 is there a thesis statement
Is the thesis well researched?
 (use of professional journals, primary sources, current periodicals)
Is the thesis proved or disproved?
 shows analysis using economic reasoning
 examines causes of patterns of societal change
Is the paper well written?
 Clarity of thought
 Organized presentation
 Spelling, grammar, punctuation
 Citations are proper
Academically Honest?
Coherent Oral Presentation to class
 Concise and organized
 Uses available technology/media appropriately
Economics Electives Term Paper Rubric Assessment
Context:
This assessment will be used to assess term papers in economics upper division
electives and their use of the tools of the liberal arts (critical thinking and analysis),
ability to examine society using economics reasoning, and demonstrate effective oral
and written communication.
Purpose:
The purpose of this assessment is to provide feedback to students about their term
papers.
(see next slide)
Performance Assessment
Exemplary
Thesis
Paper has a cogent
thesis statement
with argument in
reference to
economics
literature.
Thesis/Topic
Well-researched
Uses professional
journals, primary
sources
(data/interviews),
current scholarly
periodicals.
Draws conclusion
from weight or
evidence found,
uses economic
analysis to examine
societal patterns
chosen.
Clear and concise
writing. Organized
presentation (topics
flow properly). Use
of standard
English, spelling,
grammar and
punctuation.
References
appropriately using
MLA/APA
standards for
notation and
bibliography.
Concise and
organized
presentation with
good voice
moderation and
uses
technology/media
appropriately, uses
Thesis Proved or
Disproved
Quality writing
References and
Citations
Oral Presentation
Proficient
Included
Mostly describes
paper; but linked
to economics
Needs Work
Not
Describes Paper
Relies on some
professional
publications and
sources, along with
synthesized sources
(textbooks)
Able to synthesize
material and state
weight of evidence ,
supports/rejects
hypothesis.
Overly reliant on
synthesized
material.
May contain some
(1-3) grammatical,
spelling,
punctuation errors.
Errors interrupt
reader’s ability to
understand and
finish.
Cites most
reference material
correctly with
minor errors of
omission or
construction.
Presents topic
Does not follow
any standard
Unable to apply
economic reason
to prove thesis or
draw proper
conclusions.
Too many to list.
August 31, 2009
Economics Electives Term Paper Rubric Assessment
Context:
This assessment will be used to assess term papers in economics upper
division electives and their use of the tools of the liberal arts (critical
thinking and analysis), ability to examine society using economics
reasoning, and demonstrate effective oral and written communication.
Results:
In the Spring 2009 semester, 23 students were in ECO 340 Labor
Economics class. They were all required to complete a term paper which
was evaluated using the attached rubric. Students were given instruction
as well as the rubric early in the semester as to what the assignment
entailed. They were evaluated on the six areas with the emphasis being
placed on the first five areas. The course ended up not having enough time
for a proper oral presentation for the papers given the class size so the
sixth goal has been omitted.
From the summary statistics included, the following patterns emerged for
this group. Only half of the students understood how to include a thesis
statement or how to link their paper topic to the literature in economics at
an acceptable level. Less than half were proficient in using professional
economics publications and overly relied on synthesized material. Easily
the area in most need was the ability to set up and prove or disprove the
thesis from the weight of the evidence learned in the paper. A real strength
the students possessed however was in the actual mechanics of writing.
Citation was solid with over half performing at the proficient level or above.
Conclusion:
From this assessment, it is clear that students taking upper division
economics electives will need a greater degree of explicit instruction on the
proper constructing of a thesis statement and how to properly show
evidence from professional economics literature in the proving or
disproving of it. This is an important skill from the liberal arts of reasoning
and rhetoric and will be addressed in upcoming classes.
Joseph DaBoll-Lavoie
Director of Economics, SOM
What Does Middle States Want?
Using Assessment Results
Why are you assessing
your program or curriculum?
– Validate it to others (accountability)
– Improve it
– Make sure it isn’t slipping
What decisions might your
assessment help with?
• Learning goals
– Are our learning goals sufficiently clear and focused?
• Curriculum
– What is the value of service learning?
– Should our courses have more uniformity across sections?
• Teaching methods
– Is online instruction as effective as traditional instruction?
– Is collaborative learning more effective than lectures?
– Are we developing a community of scholars?
• Assessments
– Have our assessments been useful?
• Resource allocations
– Where should we commit our resources first?
Using Assessment Results
to Improve Teaching & Learning
When Assessment Results
Are Good
Celebrate!
Publicize!
When assessment results are
disappointing…
Look at your learning goals.
• Are your goals inappropriate or overly
ambitious?
• Do your goals need to be clarified?
• Do you have too many goals?
Look at your curriculum.
– Including placement and
developmental education.
• Does the curriculum
adequately address
each learning goal?
Curriculum Mapping
• Big chart that is used to determine which
courses address which goals
Look at your teaching methods.
• How do students learn best?
Look at your assessments.
• Are they poorly written and
misinterpreted?
• Do they match your key learning goals?
• Are they too difficult for most responsible
students?
Recognize that Every Measure of
Insitutional and Program
Effectiveness is Imperfect
•
•
•
•
Retention & graduation rates
Published instruments
“Comparable” measures
Value-added measures
Summary:
What Does Middle States
Really Want?
Questions
an MSCHE Reviewer Might Ask
Is the Institution Engaged in
“Good” Assessment?
Used
Cost
effective
Valued
Clear &
important
goals
Reasonably
accurate &
truthful results
For Each Goal…
• How is the goal being assessed?
• What are the results of those assessments?
• How have those results been used for
improvement?
Goals
Assessments
Improvements
“Improvements” Can Only Be Made After
Assessment Has Been Done
How Much Has Been Implemented?
• Are there any missing pieces?
What Do Assessment Results Tell Us?
• Do results
demonstrate…
– Achievement of
mission and
goals?
– Sufficient
academic rigor?
Have Assessment Results Been Used?
• Have they been appropriately shared &
discussed?
• Have they led to appropriate decisions?
– Curricula and pedagogy
– Programs and services
– Resource allocation
– Institutional goals and plans
Is the Process Sustainable?
• Simple
• Practical
• Detailed
• Ownership
• Appropriate
timelines
In Summary…
Five “Rules” for Assessment
1. Keep it useful.
2. Tie assessments to important goals.
3. For student learning, include some
“direct” evidence.
4. Use multiple measures.
5. Keep doing something everywhere,
every year.
During Assessment
Between Assessments
During Assessments
Between Assessments