Assessment of
High Impact Practices:
Student Learning Portfolios
and Other Tools
Monica A. Devanas
devanas@rutgers.edu
http://CTAAR.rutgers.edu
AAC&U, LEAP, HIPs, VALUE Rubrics
American Association of Colleges and Universities
Liberal Education and America’s Promise – 2005
21st Century Liberal Education
Essential Learning Outcomes
High Impact Practices
Authentic Assessment - Valid Assessment of
Learning in Undergraduate Education
= VALUE Rubrics
High Impact Practices
What? techniques and curricular designs
for teaching and learning
Why? proven to be beneficial
for student engagement and successful learning
students from many backgrounds
How? intentional program design and pedagogy
Why? enhance student learning
narrow gaps in achievement across populations
High Impact Practices
•
•
•
•
•
•
•
•
•
•
First-Year Seminars and Experiences
Common Intellectual Experiences
Learning Communities
Writing-Intensive Courses
Collaborative Assignments and Projects
Undergraduate Research
Service Learning, Community-Based Learning
Diversity/Global Learning
Internships
Capstone Courses and Projects
List
Summarize
Solve
Analyze
Design
Evaluate
Name
Explain
Illustrate
Organize
Hypothesize
Choose
Identify
Interpret
Calculate
Deduce
Support
Estimate
Show
Describe
Use
Contrast
Schematize
Judge
Define
Compare
Interpret
Compare
Write
Defend
Recognize
Paraphrase
Relate
Distinguish
Report
Criticize
Recall
Differentiate
Manipulate
Discuss
Justify
State
Demonstrate
Apply
Plan,
Visualize
Classify
Modify
Devise
Assessment of Learning in HIP
What is the goal of High Impact Practice?
What is the context of student learning?
What assignments will be used to grade?
How will these assessments give feedback
on progress of student? … of the program?
Assessment Vocabulary
• Formative – ongoing feedback
to improve learning, areas for improvement
“low stakes” few points, but frequent
Ex: first draft, questions on reading, concept map
• Summative – evaluate learning for final grade
compare to standards or benchmark
“high stakes” high point value
Ex: midterm, final, final project, paper, performance
Assessment Vocabulary
Qualitative - data fits interpretive criteria,
not easily analyzed by quantitative methods
Quantitative – numerical data that can be
analyzed with statistical methods
Assessment Vocabulary
Direct – students
knowledge,
skills, attitudes, data measures achievement
of expected outcomes
Indirect – students
on knowledge,
skills, attitudes, give their opinion, easy to
administer, focus on attitudes, values,
beliefs
Direct Methods of Assessment
Pre and posttests
Course-embed assessment
Portfolio evaluation
Case studies
(homework assignment, essay, Reflective journals
locally developed tests)
Capstone projects
Comprehensive exams
Juried performance, exhibits
National Major Field
Internship, clinical evaluation
Achievement Tests
External examiners
GRE subject exams
Peer review
Certification & licensure
Senior thesis
exams
Major project
Grading criteria or rubrics
(QUALITATIVE)
(QUANTITATIVE)
Direct Methods of Assessment
Indirect Methods of Assessment
•
•
•
•
•
•
•
•
•
•
Departmental survey
Student satisfaction surveys
Self-reported gains
Exit interviews
Alumni survey
Employer survey
Focus groups
Job placement statistics
Percentage of students who study abroad
Graduation and retention rates
High Impact Practices
Documentation: assignments, reports, essays
Collaboration: instructors, peers, advisors
Reflection: can student see “impact” on
learning, on skill development, on beliefs,
attitudes and values?
Components of Learning Portfolio
Learning Portfolio: focus process of learning
Documentation = products for evaluation
Collaboration = mentor to understand
process of reflection and documentation
Reflection = think about process of learning,
metacognition, thinking about thinking
Components of Learning Portfolio
Learning Portfolio: focus process of learning
Zubezarreta’s “order” effect of planning
Reflection = think about process of learning,
metacognition, thinking about thinking
Documentation = products for evaluation
Collaboration = mentor to understand
process of reflection and documentation
Documentation
• Organization, selectivity driven by purpose
– Course, activity, program, curriculum
• Concise reflective narrative on documents
– Relevance, impact on learning, applications
• Selected “evidence” described in narrative
and link in appendix
– OpenLab, web-based, paper-based
Mentor is Important
• “Collaborative” Mentor: to keep focus,
manageable, current, accurate, organized,
relevant, time on task, deadlines, coach
• Need some objective review
– Peer mentor
– Advisor
– Draft reader, editor
– Portfolio “coach”
Size, Scope of Learning Portfolio
Represent one set of assignments and
activities from one course
OR
experiential learning activity or over a
longer period of time,
core courses
courses in major
undergraduate curriculum
Student Portfolio =
Learning Portfolio without reflection
Even process of collecting artifacts has value
Students see evidence of significant learning
• connections between learning experiences,
i.e. in courses, in co-curricular activities.
• Valuable as summation of work in learning in
some context as a product
Useful for assessment of course, program
Flavors of Portfolios
Student Portfolio: Product
– collection of student work in some context
Learning Portfolio: Product AND PROCESS
reflective learning = process of reflection
On Learning Portfolios
Zubizarreta…
“organized documentation of growth and
achievement that provides tangible evidence
of the attainment of professional knowledge,
skills and disposition. Each portfolios is goaldriven, original, and reflective.
The Learning Portfolio value lies in student
reflection, construct the notion of learning as
a coherent, unified, developmental process,
that it is active and lifelong.”
Students ask,
"Do e-Portfolios Have
To Be "E"?
Fundamental First
Steps For Successful
Learning Portfolios.
http://eportfolioca.org/index.php?option=com_content&view=article&id=141&Itemid=159
The E-Portfolio Forum: Deepening High Impact Learning Sessions
January 29, 2011 - San Francisco, CA
Assessment of High Impact Practices
Assortment of Assessments Methods
Formative – Summative
Qualitative – Quantitative
Direct – Indirect
Balance and Variety
Assessment of High Impact Practices
Assortment of Assessments Methods
Balance and Variety
Difficult to Assess Activities
Diversity/Global Learning
Internships
Capstone Courses and Projects
Student Portfolios – Learning Portfolios
Portfolios at City Tech
Student Portfolios - Reflection with Portfolios
– Utilized work from open learning environment
– Link to past to represent what you have
developed through the college years
– Binding evidence from past, alternately reflect
back on how past work informed current work,
has past work driven interest
– Is there maturity over time / enrichment
– Reflection what student learned through guidance
Karen Goodlad, notes from 12/2/13
High Impact Practices
Goals important for assessment
Kinds of assessment
Student Portfolios, Learning Portfolios
useful for student learning and assessment
~~Wrap UP~~
Closing Ideas from Participants
Questions??