Overview
 Portfolio Basics
• Portfolio purposes
• Learning objectives to be addressed
• Roles of student and faculty in portfolio
development, implementation and use
• Institutional strategies for data use
 A case study
 Promise and Pitfalls
 Lessons learned
What is purpose of the
portfolio?
 What outcomes are desired from
portfolio use?
• Clear purpose
• Design of portfolio driven by its use
and the knowledge and skills to be
assessed
 Growth model
 Showcase
 Hybrid
What are the MEASURABLE
learning objectives?
 Facilitates meaningful evaluation
 Provides a common language
 Use tools to facilitate the
process
 Involve your key constituents
 Communicate with students
Program Learning Outcome:
“Students will
demonstrate effective
oral communication
skills”
Performance Criteria:
Acceptable Standard of Performance
Effective oral communication
Personal appearance is appropriate
Speaks clearly and with sufficient
volume
Achieves rapport with audience
Uses engaging vocalization
Responds effectively to questions and
comments
Uses audience-appropriate vocabulary,
content, and style
Public Speaking Evaluation Sheet
Student: __________________________________ Date: ________
Title of Presentation: _____________________________________
Evaluation Scale: Yes, a lot () 4 3 2 1 0 () No, not at all
Criteria
Score
Totals
Presentation Style:
1. Personal appearance is appropriate
2. Speaks clearly and with effective volume
3. Achieves rapport with audience
4. Uses engaging vocalization
5. Responds effectively to questions and comments
6. Uses audience-appropriate vocabulary, content, and style
_____
_____
_____
_____
_____
_____
Presentation style total:
______
Content:
7. Uses the grammar of standard English
8. Presentation includes introduction, body, and conclusion
9. Organizes content logically and sequentially
10. Presents ideas and arguments clearly and logically
11. Uses appropriate audiovisual materials
12. Cites sources appropriately
_____
_____
_____
_____
_____
_____
Student and Faculty roles in
portfolio design and use
 Be clear about how faculty are involved in
the design and use of portfolio regardless
of portfolio platform
• Meet their needs for information
• Sensitivity to workload
• Non-intrusive as possible
 Involve students in meaningful ways
• Be clear about what’s in it for students
• Process should reinforce and be aligned with the
educational process
Assessment
 Development of scoring rubrics
• Linked to performance criteria
• Known to students
• Scales consistent with purpose of assessment
(i.e., student/program)
 Involve students in meaningful ways
• Be clear about what’s in it for students
• Process should reinforce and be aligned with the
educational process
What strategies for data use?
• Know what you are going to do with the data
 Who is going to review/rate portfolio
contents?
 Who is going to be responsible for
analysis of data?
 What mechanisms do you have in place
to evaluate the data?
 How do the results relate to your
educational delivery strategies – curricular
and co-curricular?
Close the loop
● Importance of feedback that provides
meaningful information
● Target those who can do something
about it
● Focus on decision-making
• Improvement
• Accountability
• Celebration
● Inform those who have been involved
about how results have been used
● Communicate to stakeholders
AAHE Principles of Best Practice for
Assessment of Student Learning
 Use the “best practices” to guide
development of assessment process
 Recognition of local constraints
OUTCOMES
INPUTS
Rose-Hulman
Institute
Of Technology
 Terra Haute, Indiana
 1650+ undergraduate students
 B.S. degrees in engineering,
science, and mathematics
 85%+ engineering students
Rose-Hulman’s Mission
To provide students with the
world’s best undergraduate
education in engineering, science,
and mathematics in an environment
of individual attention and support.
Input
Recruit highly qualified students,
faculty, and staff
Quality
Provide an excellent learning
environment
Climate
Encourage the realization and recognition of
the full potential of all campus community
members
Outcomes
Instill in our graduates skills appropriate to
their professions and life-long learning
Provide resource management &
Resources development that supports the
academic mission
Outcomes
Instill in our graduates skills appropriate to
their professions and life-long learning
 Recognition of ethical and professional responsibility
 Understanding of contemporary issues
 Recognition of the role of professionals in the global
society
 Understanding of diverse cultural and humanistic
traditions
 Ability to work effectively in teams
 Ability to communicate effectively
 Apply the skills and knowledge necessary for
mathematical, scientific, and engineering practices
 Ability to interpret graphical, numerical, and textual
data
 An ability to design and conduct experiments
 An ability to design a product or process to satisfy a
client’s needs subject to constraints
Assessment of student material
 Faculty work in teams
 Each team assesses one learning
objective
 Score holistically
 Emerging rubrics
• Does the reflective statement demonstrate
or argue for the relevance of the submitted
material to the criterion?
• Is the submitted material at a level
expected of a student who will graduate
from Rose-Hulman?
Example of Results
1. Reflection
relevant to 100%
criterion?
2. Expected for 80%
R-HIT
60%
graduate?
Communication Skills
40%
20%
0%
1
2
Rater Question
Example of Results
1. Reflection
relevant to 100%
criterion?
2. Expected for 80%
60%
R-HIT
graduate?
40%
Communication Skills - Criterion 5
Gramatically correct
20%
0%
1
2
Rater Question
Example of Results
Is the submitted material at a level expected of a
student who will graduate from Ross-Hulman?
1.Appropriate
for audience
100%
2.Organization
80%
3.Content
60%
factually
correct
40%
4.Test audience
20%
response
5.Grammatically 0%
correct
Communication Skills All Criteria
1
2
3
Rater Question
4
5
Linking results to Practice
 Development of Curriculum Map
Linking curriculum content/pedagogy
to knowledge, practice and
demonstrating learning outcomes
Curriculum Map results
Fall 1999-2000 (181 courses/labs)
Communication Skills
100%
86%
80%
70%
69%
60%
40%
20%
7%
0%
Explicit
Competence
Feedback
Not Covered
80%
Curriculum Map results
Fall 1999-2000 (181 courses/labs)
Ethics
59%
60%
40%
29%
26%
17%
20%
0%
Explicit
Competence
Feedback
Not Covered
Closing the loop
DEC
NOV
OCT
JAN
Eval committee
receives and
evaluate all data;
makes report and
refers recommendation to
appropriate areas.
SEP
AUG
Institute assessment
center prepares
reports for
submission to Dept
Heads of the
collected data (e.g.
surveys, portfolio
ratings)
JUL
Institute acts on the
recommendations of
the Eval Comm.
Reports of actions
taken by the
institute and the
targeted areas are
returned to the Eval
Comm. for iterative
evaluation.
MAR
APR
MAY
JUN
FEB
Primary focus
Implement
 It is not about electronic portfolios.
 It is about:
• teaching and learning
• faculty and student development
• support for the transformation of the
teaching/learning environment
Develop
Benefits to teaching
 Faculty are asked to reflect on learning
outcomes in relation to practice
• Consider the value of stated outcomes
• Right ones?
• Right performance criteria?
• Individual faculty ownership in creating the
context for learning
 Develop a common language and understanding
of program/institutional outcomes
 Promotes interdisciplinary discussions/
collaborations
 Explicit accountability for learning outcomes
(e.g., state mandates/accreditation)
Benefits to learning
 Students review their own progress as it
related to expected learning.
 Portfolios provide a way for students to
make learning visible and becomes the basis
for conversations and other interactions
among students and faculty.
 Learning is viewed as an integrated activity
not isolated courses.
 Students learn to value the contributions of
out-of-class experiences.
 Student reflections are metacognitive as
they appraise their own ways of knowing.
 Promotes a sense of personal ownership over
one’s accomplishments.
Students
 Rewards
 Relevance
 Commitment
Lessons Learned
 Assessment language – developing a
discourse community
 Allocate resources
 Living, iterative process
 Focus is on improvement and celebration
 Start early
 Prepare for the need for “in-service” for
faculty and students
 Decouple from faculty evaluation
DEMO
Site
http://www/rose-hulman.edu/ira/reps
http://www.rose-hulman.edu/irpa
AAHE/HLC
gloria.rogers@rose-hulman.edu