Using ePortfolios for EvidenceBased Research
Julie Hatcher,
Director of Undergraduate Programs, Associate Professor of Philanthropic Studies
Kristin Norris,
Instructional Technology Specialist, Center for Service & Learning
Kathy Steinberg,
Assessment Specialist, Center for Service & Learning
Special Thanks to our Sponsor, Epsilen
General questions
•
How familiar are you with e-portfolios?
▫
•
•
•
•
Very …..
…..
… Not at all
How familiar are you with e-portfolios and service
learning?
How familiar are you with e-portfolios and civic
learning outcomes?
How familiar are you with advising others on how to
use e-portfolios in service learning research?
What are a few KEY issues to address?
Agenda
•
•
•
•
•
•
•
•
•
•
1:00-1:10 Introductions & General questions
1:10-1:25 Overview of ePortfolios
1:25-1:45 Activity (Review example ePortfolios)
1:45-1:55 Tools to assess the evidence
1:55-2:25 Activity (Read/Watch Call of Service)
2:25-2:40 Overview of Digital Storytelling
Break (10-15 min)
3:00-3:30 Research, Reflection, Assessment
3:30-3:45 Choice Points
3:45-4:00 Take Aways & Next Steps
Overview of ePortfolios
Harness the power of the pedagogy
ePortfolios is both a
process and product
• Process: a series of events (time & effort) to produce a
result
▫ Portfolio as a workspace
▫ Working Portfolio (digital archive, repository of artifacts,
collaboration space, reflective journaling)
▫ Primary purpose: learning or reflection
▫ Organization: chronological
• Product: the outcome/results of an activity/process
▫ Portfolio as a showcase
▫ Presentation Portfolios (the “Story” or narrative, multiple
views, varied audiences & purpose)
▫ Primary purpose: Accountability or showcase for
employment
▫ Organization: thematic
Barrett, Eifel, July 2011(www.slideshare.net/eportfolios)
Processes
Portfolios
Social Networking
Technology
Collection
Connect (‘friending’)
Archiving
Selection
Listening (reading)
Linking/Thinking
Reflection
Responding
(commenting)
Digital Storytelling
Direction/Goals
Share (linking/tagging)
Publishing
Presentation
Feedback
Example of a Product ePortfolio
Example of a Process ePortfolio
What is an ePortfolio?
• “A portfolio tells a story. It is the story of
knowing. Knowing about things…Knowing
oneself…Knowing an audience….Portfolios are
students’ own stories of what they know, why
they believe they know it, and why others should
be of the same opinion.” (Paulson & Paulson,
1991, p. 2)
Power of the Pedagogy
“Stories help us organize our experience and
define our sense of ourselves”
(Roger Schank, Tell Me a Story)
Why ePortfolios?
•
•
•
•
•
•
•
•
•
•
Reflecting
Celebrate learning
Personal planning & goal setting
Employment applications
Accountability (prove what you have learned)
Capture and store evidence (repository)
Give and receive feedback
Collaborate
Present what you know to an audience
Exploring your personal and professional identity
ePortfolios as a Purpose
the “overarching purpose of ePortfolios is to
create a sense of personal ownership over
one’s accomplishments, because ownership
engenders feelings of pride, responsibility,
and dedication.”
Paris & Ayers (1994)
Managing Oneself
•
•
•
•
•
•
What are my strengths?
How do I perform?
What are my values?
Where do I belong?
What should I contribute?
Responsibility for Relationships?
“Success in the knowledge economy comes to those
who know themselves – their strengths, their values,
and how best they perform” –Peter Drucker (2005,
Harvard Business Review)
Why should you use ePortfolios
• Facilitate Reflection
• Enable students to create something that
demonstrates their knowledge, skills, abilities,
dispositions – often times the things they have
yet to articulate in any other way
• Gather evidence of student knowledge (and
maybe growth)
• Because it is of value to the students
Choosing an ePortfolio Platform
https://sites.google.com/site/choosingeportsoftware/
Latest Blog by Trent Batson
http://campustechnology.com/articles/2011/10/12/a-survey-of-the-electronic-portfolio-market-sector.aspx#
Activity:
Examine sample ePortfolios
•Pay attention to structure (tabs, headings, organization)
•What evidence of civic learning do you see?
•Key elements to look for – ‘About me’, artifacts vs reflections
(is it ‘critical reflection’?), intended audience
•What does an ePortfolio allow you to see that a post-test or a
written reflection would not?
Examples
• https://stonybrook.digication.com/egimenez/How_did
_the_Soup_Kitchen_influence_my_life
• https://stonybrook.digication.com/americorps_justin_t
homas/Home
• https://oncourse.iu.edu/osp-presentationtool/viewPresentation.osp?id=5DCDA2A93B4FE85E93
F80F4DAE621672
Assessing the Evidence
Is there a magic rubric?
Goal of Exercise
•
Introduction to VALUE – Civic Engagement
Rubric
▫
▫
▫
▫
▫
AAC&U purposes of meta rubrics with eportfolios
What are the domains of civic learning?
Does the CE Rubric work? What’s missing?
Does the CE Rubric work for different types of
evidence (reflection paper, digital story)?
Keep in mind issues that may surface in your
own context, adaptations
Public Purposes of Higher Education
• AAC&U



LEAP – Personal and Social Responsibility
VALUE project – Civic Engagement Rubric
Bridging Theory to Practice
• AASC&U

American Democracy Project
• Carnegie Foundation

Voluntary Classification for “Community
Engagement”
• Professional Associations
• Accreditation
AAC&U Essential Learning Outcomes
•
Personal and social responsibility
•
•
•
•
Civic knowledge and engagement- local and global
Intercultural knowledge and competence
Ethical reasoning and action
Skills for lifelong learning
“Anchored through active involvement with diverse
communities and real world challenges.”
AAC&U VALUE Initiative
•
What learning should “look like”; 15 rubrics
•
Meta rubric – adaptable to campus context
•
Institutional assessment for e-portfolios
•
Based on existing rubrics …. However….
Civic Engagement
•
•
•
•
Few existing rubrics in civic engagement
Convening of “experts” in the field, by phone
Common readings
Shared experiences – “what does a student
look like”
Definitional terms
•
▫
▫
▫
AAC&U
AASC&U
Ehrlich definition – see rubric
VALUE Rubrics
•
Simple language – used by variety of readers
•
Progressively more “robust” learning
•
Levels of sophistication; zero can be used
•
Single, most vital performance in each cell
•
Core, shared dimensions of learning, for all
students across all majors
Dimensions of CE Rubric
•
Diversity of communities and cultures
•
Analysis of knowledge
•
Civic-identity and commitment
•
Civic communication
•
Civic action and reflection
•
Civic contexts/structures
Activity: Types of Evidence
1. In groups of 3, read a student paper
2. Using the AAC&U VALUE Rubric for Civic
Engagement, rate the student submission
3. Together, we will watch a digital story by the
same student, rate using the rubric
4. Repeat the exercise for the 2nd paper and
digital story
Digital Storytelling
The power of voice
Power of Voice
“When words are infused by the
human voice, they come alive” –
Maya Angelou
When the writer is engaged personally with a
topic, they impart a personal tone and flavor to
the piece that is unmistakably his/hers alone. It
is the heart and soul of the writing, the magic,
and wit, the feeling, the life and breath.
(http://educationnorthwest.org/resource/503)
What is a digital story?
• An illustration of learning
• A way of documenting an experience(s)
• One way to facilitate the reflection process
▫
▫
▫
▫
▫
2-4 minute digital video clip
First person narrative
Told in your own voice
Illustrated (mostly) by still images
Additional music added to evoke emotions
Examples of Digital Stories
• Using www.screenr.com
▫ http://screenr.com/6ASs
• Using PhotoStory3 (Windows)
▫ http://www.youtube.com/watch?v=nqDu4uYtNJM&f
eature=feedu
• ISL example
▫ http://streaming.osu.edu/knowledgebank/digstory08
/JacksonFINAL/flash/f.htm
• Using iMovie (created by 5th graders)
▫ http://www.infotoday.com/MMSchools/jan02/digital
place.mov
What makes for a great digital story?
•
•
•
•
•
•
•
•
•
•
Overall purpose of the story
Narrator’s point of view (emotional content)
A dramatic question
Choice of content
Clarity of voice
Pacing of the narrative
Meaningful soundtrack
Quality of the images
Economy of the story detail
Good grammar and language usage
Steps to develop a digital story
1. Write a script: write, get feedback, rewrite, and
work with others (maybe in a group) to develop
ideas
2. Capture and process the images to further
illustrate the story
3. Record the author reading the story
4. Combine audio and images onto a timeline, add
music
5. Present/publish
Example Storyboard
Script/Sound Effect/Music
Image/video
Mary had a little lamb
Drawing of Mary with her lamb in a
field of flowers
(sound – birds singing, girl humming
the tune
Whose fleece was white as snow
Drawing of a snowflake
(Sounds – girl continues to hum the
tune)
And everywhere that Mary went
(Sound – girl continues to hum the
tune)
Drawing of Mary Walking into the
mall
Copyright Laws
• These days, almost all things are copyrighted the moment
they are written, and no copyright notice is required.
• Copyright is still violated whether you charged money or
not, only damages are affected by that.
• Fair use is a complex doctrine meant to allow certain
valuable social purposes. Ask yourself why you are
republishing what you are posting and why you couldn't have
just rewritten it in your own words.
• Copyright is not lost because you don't defend it.
• Fan fiction and other work derived from copyrighted works is
a copyright violation.
• Don't rationalize that you are helping the copyright holder;
often it's not that hard to ask permission.
http://www.templetons.com/brad/copymyths.html
Just remember…
• copyright has two main purposes, namely the protection
of the author's right to obtain commercial benefit from
valuable work, and more recently the protection of the
author's general right to control how a work is used.
• While copyright law makes it technically illegal to
reproduce almost any new creative work (other than
under fair use) without permission, if the work is
unregistered and has no real commercial value, it gets
very little protection. The author in this case can sue for
an injunction against the publication, actual damages
from a violation, and possibly court costs. Actual
damages means actual money potentially lost by the
author due to publication, plus any money gained by the
defendant.
So what can you use without worrying
about Copyright issues?
IMAGES
• Flickr (Narrow search to ‘The
Commons’)
• Google Images (Labeled for
reuse)
• Your own!
• www.iupui.edu/galleries/
• Photos.iupui.edu (need an
account – I’d be more than
happy to share – not the same as
my CAS login)
MUSIC
• http://freeplaymusic.com
• www.jamendo.com
• Podsafe Audio
(www.podsafeadio.com)
• Creative Commons
(http://creativecommons.org/l
egalmusicforvideos/)
• Yahoo Music
(http://new.music.yahoo.com)
• Your own!
Putting it all together
• Import all images, video, voice recording, and
musical elements
• Lay your narration track onto the timeline
• Add your images to match your narration
• Create an initial rough cut before adding
transitions or special effects
• Add titles, transitions, special effects sparingly
▫ Note: these steps may vary depending upon the
program you are using.
Tools (Putting it all together)
Mac
Web 2.0
Windows
Write Script
Google Docs
Write Script: Word
Record Audio:
Audacity/GarageBand
Myna (Aviary)
Record Audio: Audacity
Edit images: iPhoto
Aviary Tools
Edit Images: Picasa3
Edit Video: iMovie
Animoto or Voice Thread Edit Video: Photostory3
Prezi
or Windows Movie
Maker
But, there are 50+ free tools available and capable of doing something
just slightly different. Check these blogs to learn more about the various
tools and their capabilities
http://cogdogroo.wikispaces.com/StoryTools (5-+ Web 2.0 Ways to Tell a
Story)
http://50ways.wikispaces.com/ (More recent version of the site above)
Share/Publish
•
•
•
•
•
•
•
Motionbox (http://www.motionsbox.com)
YouTube (http://www.youtube.com/)
TeacherTube (http://teachertube.com/)
SchoolTube (http://schooltube.com/)
Blip.tv (http://blip.tv)
Vimeo (http://www.vimeo.com)
Facebook video
• Your ePortfolio!
BREAK
Come back at 3:00
ePortfolios for Research
Assessment vs. Evaluation
Student Assessment
• Individual or group of
learners
• Understand learner through
performance of a specific
learning task/standard
• Provide feedback to
students
• Diagnostic tool for
instruction
• Formative
• Ex: Reflections
Student Evaluation
• Individual/group of learners
• Understand learner through
performance of a specific
learning task/standard
AND
• Judge the quality or worth of
the assessment results
• Provide feedback to students
• Based upon multiple sources
of assessment information.
• Formative/Summative
Research vs. Evaluation
Research, especially fundamental or basic
research, differs from evaluation in that
its primary purpose is to generate or test
theory and contribute to knowledge for
the sake of knowledge. Such knowledge,
and the theories that undergird
knowledge, may subsequently inform
action and evaluation, but action is not
the primary purpose of fundamental
research. (Patton, 2002, pp. 10-11)
Value of ePortfolios for Service
Learning
• Most assessment tools are self-report
instruments (nationally and locally)
• Eportfolios provide “authentic” assessment
evidence/data
• Draw on strengths of Service Learning
▫ critical reflection
• Eportfolios can be used for research
▫ also for course use and program assessment
▫ designs can be simple or complex
Important Considerations
• Confidentiality (FERPA)
• Ownership of materials; permissions
• Access to materials and platform after student
leaves or graduates
• IRB Issues
▫ Informed consent
Examples of Research on Service
Learning using ePortfolios
• Project 1:
Civic development of students
in a Service Learning Assistant
program
• Project 2:
Development of civic learning
in freshmen taking service
learning classes versus nonservice learning classes
Project 1
Service Learning Assistant Program
Overview
• Scholarships recognize IUPUI students selected by
faculty or professional staff to:
▫ assist in the implementation of a service learning class,
▫ collaborate with faculty in their community-based
research,
▫ expand the capacity of campus departments to
increase the number of students who participate in
service-learning, or
▫ complete a service project in and with the community.
• Faculty development focus makes it unique.
Guiding Research Questions
• To what extent do ePortfolios support the civic
development of students?
• What types of ePortfolios best facilitate student
civic learning and assessment?
• What factors contribute to student civic
development as a result of participation in a
service-based scholarship program? (Kristin’s
research)
Program Specific Research Questions
• To what extent do students who participate in
service scholarship programs (compared to their
SL peers or non-SL peers):
▫ develop a greater capacity/ability to articulate an
integrated sense of their personal, civic,
professional identity?
▫ develop enhanced civic learning KSAs, compared
to other students who do not participate in these
types of interventions?
Using ePortflios to gather evidence
• Civic-Minded Graduate Scale
• Complete reflection on a professional
development activity
• Complete end-of-award period reflection (CMG
Narrative/Scale)
• Faculty mentor rates the end-of-award reflection
(CMG Narrative Rubric)
Project 2
• Development of civic learning in freshmen
taking service learning classes versus nonservice learning classes
• High-impact practices:
▫ First-year seminars
▫ Themed Learning Communities
• ePortfolios are both a high-impact practice and a
tool for gathering evidence
• Evaluate ePortfolios using rubrics
Workshop Reflection:
Choice points
• What ePortfolio platforms are available?
• What type of ePortfolio do you want?
▫ Free-form vs. structured (how?)
▫ Process/assessment purposes (matrix style) vs.
development (presentation style)
student
• What types of evidence do you intend to gain, and how?
▫ Are you wanting to use a ePortfolio as a repository of the
student’s work -OR- to help facilitate reflection after having
completed their assignment?
• How might you structure your course or program (learning
outcomes, assignments, activities) to gather ePortfolio
evidence?
▫ ePortfolios CANNOT be an add-on
• How will you evaluate the evidence?
▫ Are you grading an assignment, then evaluating their ePortfolio?
Take-Aways & Next Steps
•What is one thing you will take
away from this workshop today?
•What is your next step(s)?
Questions? Comments?
Feel free to contact us!
Julie Hatcher (jhatcher@iupui.edu)
Kristin Norris (norriske@iupui.edu)
Kathy Steinberg (ksteinbe@iupui.edu)
Thank you Epsilen!