Seeking Evidence for Impact:
Lessons from the Inter/National Coalition for
Electronic Portfolio Research
Darren Cambridge
ELI Webinar
July 11, 2011
Overview
• Conceptual foundations of approach
• Structure of the Inter/National Coalition for
Electronic Portfolio Research
• Findings about three types of learning
• Contributing factors
The Importance of Having Problems
In scholarship and research, having a "problem" is at
the heart of the investigative process; it is the
compound of the generative questions around which
all creative and productive activity revolves. But in
one’s teaching, a "problem" is something you don’t
want to have, and if you have one, you probably
want to fix it. … How might we make the
problematization of teaching a matter of regular
communal discourse? How might we think of
teaching practice, and the evidence of student
learning, as problems to be investigated, analyzed,
represented, and debated?
—Randy Bass
Three curricula
Kathleen Yancey, Reflection in the Writing Classroom
Research into the Swamp
“There is a high, hard ground where practitioners
can make effective use of [traditional] researchbased theory and techniques, and there is a
swampy lowland where situations are confusing
‘messes’ incapable of technical solution. The
difficulty is that the problems of the high ground,
however great their technical interest, are often
relatively unimportant to clients or to the larger
society, while in the swamp are the problems of
greatest human concern.”
Donald Schön
Transactional Research
• Practitioners generate research questions
• Goal is to influence practice
• Methodologies chosen based on knowledge
about learning, not exclusively current
disciplinarily-accepted methodologies
• Agency for answering the questions resides in
multiple constituents
– practitioner researchers
– learners
– peer practitioner researchers
• Diversity provides robustness
DISCUSSION
COALITION STRUCTURE
Coalition Exigency
• Rapid growth in use of electronic portfolios in
the United States (and beyond)
• Wide diversity of models
• Considerable potential to impact learning and
engagement
• Evidence uneven and unintegrated
Inter/National Coalition for Electronic
Portfolio Research
Coalition Structure
• Inter/National Coalition for Electronic Portfolio
Research established in 2003
• Led by Barbara Cambridge (AAHE/NCTE), Kathleen
Yancey (Clemson/FSU), Darren Cambridge
(EDUCAUSE/GMU/AIR)
• Six cohorts of about ten campuses that work
together for three years
Inter/National Coalition for Electronic
Portfolio Research
Cohort 1
Cohort 2
Cohort 3
Alverno College
Bowling Green State
University
Indiana UniversityPurdue University
Indianapolis (IUPUI)
LaGuardia Community
College
Northern Illinois
University
Portland State University
Stanford University
University of Washington
Virginia Tech University
Clemson University
Kapi’olani Community
College
George Mason University
Thomas College
The Ohio State University
University of Georgia
University of Illinois
University of Nebraska
Omaha
Washington State
University Arizona State
University
California State Universities
Florida State University
Framingham State
University
George Mason University
Minnesota State Colleges
and Universities
Penn State University
University of San Diego
Seton Hall University
Sheffield Hallam University
University of Waterloo
University of
Wolverhampton
Cohort 4
Cohort 5
Cohort 6
University of Bradford
University of Cumbria
University of Groningen
London Metropolitan
University
University of Manchester
Medical School
University of Michigan
University of
Northumbria
University of Nottingham
University of
Wolverhampton
Queen Margaret
University College
Kapi’olani Community
College
Louisiana State
University
University of Akron
University of Cincinnati
University of Denver
University of North
Carolina Wilmington
University of Oregon
Virginia State University
Virginia Tech
Bowling Green State
University
Curtin University of
Technology (Australia)
Goshen College
Indiana University
Purdue University
Indianapolis
Lamar University
Northeastern University
Portland State University
University of Georgia
University of Michigan
University of Mississippi
Virginia Military Institute
Westminster College
Coalition Activities
•
•
•
•
•
•
Individual questions and collaborative themes
Two meetings a year
Blog, newsletter, and Ning
Interaction between cohorts
Consultations with Coalition leadership
Coordinated dissemination
Inter/National Coalition for Electronic
Portfolio Research
Intra-campus Practices
•
•
•
•
Diverse team
Space for forming
Narrow but open question
Balance between intellectual and pragmatic
purposes
Inter/National Coalition for Electronic
Portfolio Research
Diverse Team
• Both people who have research in their job
title and those who don’t
• Reflective of the range of people involved in
portfolio practice on the campus
– Include administrators
– Include students
• Portland State: Administrators, students,
faculty from multiple disciplines
Inter/National Coalition for Electronic
Portfolio Research
Space for Forming
• Need sufficient time and space to develop
– Shared expectations
– Shared conceptual framework
– Personal relationships within team
Inter/National Coalition for Electronic
Portfolio Research
Narrow but Open Question
• Well-focused research question
• Openness to the data taking you elsewhere
Inter/National Coalition for Electronic
Portfolio Research
Intellectual and
Pragmatic Purposes
• Clear sense of audiences and purposes of research
• Practitioner research doesn’t have to be just
evaluation
• Balance between what you need to justify your work
and what’s intellectually meaningful
• Practice as inquiry
Inter/National Coalition for Electronic
Portfolio Research
Diversity and Balance
• Who might you ask to join your team you’ve
not previously considered?
• What aspects of your project can you expand
or emphasize to balance intellectual and
pragmatic value?
Inter/National Coalition for Electronic
Portfolio Research
Inter-campus Practices
• Senior administrative support
• Triangulation rather than replication
• Collaborative exploration of
methodologies
• Regular conversations with neutral
experts
• Multiple genres of reporting out
Inter/National Coalition for Electronic
Portfolio Research
Senior Administrative Support
• Three-year commitment of travel funding
from institutional budget
– Confirmation of commitment to portfolio practice
• Regular updates and notes of thanks
• Ideally, member of the team
Inter/National Coalition for Electronic
Portfolio Research
Triangulation
• Triangulation rather than replication
• Enough structure to focus and connect, but not
restrict
– No one strict definition of “research”
– Shared themes but not a mandated research question
• Cohorts One and Two: Catalog and taxonomy of reflective artifacts
• Critical friends
Inter/National Coalition for Electronic
Portfolio Research
Collaborative
Exploration of Methodology
• Guided exploration of research methodologies and
methods
• Both a way to plan the project and a way to develop
shared understanding of research
• Breaking out of received notions of research through
conversations
– Across disciplines
– Across campuses
Inter/National Coalition for Electronic
Portfolio Research
Conversations with Experts
• Quarterly conference calls with a Coalition
leader
• Periodic occasions for reviewing and asking
questions
• The questioning is probably more important
than the advice
October 10, 2006
Inter/National Coalition for Electronic
Portfolio Research
Multiple Reporting Genres
• Variety of forms of reporting
– One-pagers
– Blue Skies questions
– Thick descriptions of artifacts
– Presentations of evidence
– Chats
• Helps to stimulate creativity and
accommodate multiple styles
October 10, 2006
Inter/National Coalition for Electronic
Portfolio Research
DISCUSSION
EMERGENT RESULTS
Dimensions of Learning
• Reflective Learning
• Integrative Learning
• Learning to Establish Identity
Reflective learning
• Eportfolios can document reflective ability
• Eportfolios reveal a positive correlation
between the quality of reflection and
evidence
• The relationship between reflection and
evidence is more complex than previously
considered
Northern Illinois University
Semester
Reflection
Evidence
Fall 2004
2.76
3.31
Spring 2005
3.11
3.12
Fall 2005
3.55
3.70
Spring 2006
3.17
3.23
Transactional Benefits
• Alverno
– Beginning with faculty conceptual frameworks
leads to better integration into practice
• Northern Illinois
– Teaching assistants as researchers leads to
stronger investment in reflective practice
• George Mason
– Student affairs educators as researchers leads to
expansion of knowledge in both domains
Integrative Learning
Eportfolio use correlates with
increased student engagement.
LaGuardia CCSSE Results
How much has your coursework emphasized synthesizing & organizing ideas,
information, or experiences in new ways?
1 = Very Little, 2 = Some, 3= Quite a Bit, 4 = Very Much
LaGuardia ePortfolio & Retention
78
76
74
72
70
68
66
64
College
ePortfolio
Fa05->Sp06
Sp06-> Fa06
2 Smstr Mean
71
76.5
69
74.7
70
75.5
Kapi’olani Community College
Transactional Benefits
• LaGuardia
– Multiple methodologies for different
constituencies
– Students as co-inquirers essential to interpretation
• Kapi’olani
– Impact and interpretation situated in cultural
context
Learning to Establish Identity
Eportfolios can help engender strong and
complex professional identities.
University of Waterloo
“Again, this is also describing the relationship between employees
in audit engagements as well as school assignments. The only
difference I noticed is that in school assignments, everyone has
around the same educational and technical background, whereas
during an audit engagement, there are different levels of
employees (senior managers, managers, senior staff and junior
staff) grouped together to provide a larger variety of mindset during
the engagement. I believe that creating a group with different levels
of employees is the most efficient method because junior staff will
be learning from more experienced staff during the engagement,
senior staff will be able to concentrate on the more difficult aspects
of the audit while junior staff could complete the small, simple and
tedious tasks, and finally the audit team can get a larger variety of
ideas due to the diverse members.”
Clemson University
Virginia Tech
Transactional Benefits
• Cross-disciplinary collaboration yields
expanded methods for understanding
profession
• Students serve as co-inquirers through
reflective representation of their experience
Contributing factors
• Matrix thinking
• Ownership and expressive range
• Structure and support
Matrix Thinking
Freedom and Structure
• Expressive range
– Visual design
– Linking
– Use of multiple media
• Structure and support
– Levels of structure appropriate to student ability
– Language tailored to discipline and profession
– Peer mentoring and mentors as peers
Electronic Portfolios 2.0: Emergent
Research on Implementation and Impact
• Collection of 24
chapters detailing
research from the first
three years of the
Coalition
• Published by Stylus in
2009
• More about the
Coalition at ncepr.org
Eportfolios for Lifelong Learning and
Assessment
• Connects the work of
the Coalition to a
broader theoretical
framework and wider
range of research
• Published by JosseyBass in 2010
• More about my work
at ncepr.org/darren