Assessing for Learning:
Focusing an Interdisciplinary
General Education Revision
on Student Outcomes
Kate Marley, Associate Vice President for Academic Affairs
John M. Burney, Vice President for Academic Affairs
Doane College
Ida Asner, Director of Consulting Services
LiveText
Presentation Overview
Participants will
• Understand the value of sequencing learning skills and
outcomes across four years.
• Review a curriculum design that reduces broad distribution
requirements and provides three levels of liberal arts seminars
• Learn about a focus on outcomes both for faculty
development and for assessment of student learning
• Understand issues of implementing and sustaining an
interdisciplinary curriculum that builds authentic assessment
within the course structure
Your issues
• In a one minute paper jot down the
biggest obstacles to designing
programs based on LEAP principles
and using authentic assessment at
your own institutions. Then share
those obstacles with a colleague.
Timeline for developing a new
general education program
2011 -2012: Mission reviewed and revised
2012: Initial writing of new mission-driven general learning outcomes
2012-2013: Slowly built new Gen Ed curriculum
• Focus on scaffolding student development toward the outcomes
• Process of draft and comment in faculty forums
• Ongoing faculty education on active learning through workshops and
shareshops
May 2013: Revised curriculum passed by faculty
2013-2014: Planning and development
• Faculty development
• Design of new courses
• Revision of existing courses and new approval processes
• Development of assessment plan, rubric building and testing
• Review of assessment management software systems
2014-2015: Implementation of the first year of the new curriculum and
assessment
Doane’s Mission
Doane College’s mission is to provide an
exceptional liberal arts education in a creative,
inclusive, and collaborative community where
faculty and staff work closely with
undergraduate and graduate students preparing
them for lives rooted in intellectual inquiry,
ethical values, and a commitment to engage as
leaders and responsible citizens in the world.
Doane Core Connections (DCC)
• Foundational Areas of Knowledge – 1 course in each of 7
interdisciplinary areas
•
•
•
•
•
•
•
Community and Identity
Mathematical Reasoning
Rhetorical Communication
Global and Cultural Contexts
Scientific Perspectives
Human Creativity
In Search of Meaning and Values
• Liberal Arts Seminars (LAR)
• LAR 101 Inquiry Seminar: Learning the Art of Inquiry (freshmen)
• LAR 202 Integrative Seminar: Democracy and Diversity (sophomores)
• LAR 303 Impact Seminar: Connecting Knowledge to Choices and
Actions (juniors and seniors)
• Experiential Studies
33-34 credits
Traditional outcomes
Through the liberal arts, students will:
• Understand foundational areas of knowledge.
•
•
•
•
•
•
•
•
•
•
•
Community and Identity
Mathematical Reasoning
Rhetorical Communication
Global and Cultural Contexts
Scientific Perspectives
Human Creativity
In Search of Meaning and Values
Develop crucial intellectual skills
Communicate effectively
Use information wisely
Pursue a healthy lifestyle
Advanced outcomes
• Build connections of knowledge across various disciplines.
Students will learn to:
• synthesize knowledge across foundational areas and specialized
studies;
• develop creative and imaginative insights and expressions; and
• apply and integrate knowledge collaboratively to solve complex
problems.
• Adapt their liberal education to serve and to lead at all levels
of citizenship. Students will learn to:
• pursue a refined, empathetic understanding of a multifaceted
world;
• orient their own ethical compasses to act accordingly; and
• engage with people of varying perspectives to build just societies.
Hallmark: Progressively building to advanced skills
across four years through the Liberal Arts Seminars
First Year
Fall
Spring
LAR 101
Second Year
Fall
Spring
LAR 202
Third Year
Fall
Spring
Fourth Year
Fall
Spring
LAR 303
• Traditional skills introduced in LAR 101
• Traditional skills practiced, advanced outcomes
introduced in LAR 202
• Traditional and advanced skills applied in LAR 303
LAR 101 Inquiry Seminar: Learning the
Art of Inquiry – First Year
• A course designed to introduce students to
college-level writing, discussion, critical thinking,
and critical reading. Faculty will choose a topic
for each section in order to help students learn
information research skills, to work
collaboratively, and to gain an appreciation for
interdisciplinary study and multiple perspectives.
LAR 202 Integrative Seminar: Democracy
and Diversity – Sophomore Year
• In a collaborative setting, students will apply and
integrate knowledge and experiences to examine
complex questions related to democracy and
diversity from multiple perspectives.
• This course can address far-reaching issues that
are enduring or contemporary in areas such as
culture and values, science and society, global
interdependence, citizenship, or human dignity
and freedom.
LAR 303 Impact Seminar: Connecting Knowledge
to Choices and Actions – Junior or Senior Year
• This course will develop students’ teamwork and
leadership skills to prepare for citizenship or work as
they connect theory, practice and experience.
• Students, drawn from multiple fields of specialized
study, will collaboratively analyze a complex realworld problem, develop an empathetic
understanding of multiple perspectives needed to
comprehend the issue, and propose possible
solutions.
• Students will be engaged through experiential
pedagogies such as service learning.
Key to Effective Scaffolding:
Pedagogy is as important as content
• Active Learning
• Collaborative Learning
• Peer-to-peer education
• Role Playing Simulations – Reacting to the
Past
• Inquiry-Based and Problem-Based Learning
• Service Learning
• Reflective Writing
Faculty Development
Workshops:
Feb 2012: Technology and Liberal Education – Bryan Alexander
Aug 2012 Meeting Students Where They Are – Jennifer McCrickerd
Aug 2012: Trends in General Education – John Burney
Aug 2013: Developing Key Skills and Courses for the Core Curriculum –
Engebretson, Kalbach, Marley and Burney
Jan 2014: Student Reflection and Student Portfolios – John Zubizarreta
May 2014: Rubric norming with LAR faculty – Marley and Vertin
Aug 2014: Cooperative Learning as an Effective Pedagogy – Karl Smith
Jan 2014: Active learning strategies, exploring initial assessment data –
Marley and Vertin
Faculty teams sent to conferences:
• AAC&U General Education
• Collaboration for the Advancement of College Learning
• HLC Assessment
• Reacting to the Past Annual Institute
Need for New Direct Assessments
•
•
•
•
Had no direct evidence of student achievement in LAR 101
Now adding two more seminars
Faculty buy-in for implementation would be essential
Assessment data could provide evidence of seminar
effectiveness
• New Assessment Team of faculty from across college –
• Charged to develop direct assessments for DCC – 9 faculty and dean
• Team attended a fall HLC assessment workshop, met weekly
• Scaffold of Liberal Arts Seminars
• Tremendous opportunity for DEVELOPMENTAL assessment
of outcomes in seminar series
Assessment Team Identified Common
Outcome Threads in Seminar Scaffold
• Critical Thinking
Intellectual Skills
• Communication
• Reflection, Synthesis, Integration Intellectual
Engagement
• Collaboration
Threads for New Direct Assessment:
• Intellectual Skills
• Critical Thinking
• Communication
• Intellectual Engagement
•
•
•
•
•
Learner Self-Awareness
Intellectual Curiosity
Engagement with Diverse Cultures and Communities
Connections Across Disciplines
Connections Among Academic and Co-Curricular Experiences
Rubric building
• Two teams of 6 faculty were tasked with building and testing the
rubrics
• Rubric criteria and descriptions of development written and revised
• Two rounds of using rubrics with student papers and revision
• Intellectual Engagement team also developed reflective writing
prompts for each level of seminar
• Rubrics and norming process shared periodically with faculty
• January faculty workshop on reflective writing with John Zubizarreta
• May faculty workshop to scale the norming to all LAR and interested
faculty
• Teams provided select sample papers for replication of scoring and
discussion in small groups
Doane College Intellectual Skills Rubric
Criteria
Exceeding (4)
Achieving (3)
Engage in discovery: Student explores a main topic and articulates a position.
Progressing (2)
Beginning (1)
Does not
meet min.
req. (0)
The central problem or issue is
The central problem or issue is
The central problem or issue is
The problem or issue is identified in
Problem or
defined clearly along with subsidary clearly articulated along with
Problem or issue
stated although complexity of the
vague terms and/or distracting sub
issue is not
questions that explore the
indication of the complexity of
issue may not be addressed.
problems are identified.
identified.
complexity of the problem or issue.
the issue.
Specific position is stated and
Position - argument,
sustained throughout. It is
Position is stated and sustained Position is stated but may not be Position is stated, but may be simplistic, No position is
claim, hypothesis, thoughtful, creative or unique, and
throughout.
sustained consistently.
vague and/or confusing.
stated.
thesis takes into account the complexities
of the issue and limitations.
Gather and evaluate facts and assumptions: Student provides context and evidence in support of the position.
Student provides adequate context Student provides some context for the
Student provides an exceptional
Student provides a thorough
for the problem or issue, though
problem or issue, though may lack
treatment of the context for the
treatment of the context for
may be somewhat lacking in
critical information or irrelevant
No context is
Context
problem or issue. Relevant
problem or issue and is able to
information and/or include
information may be included.
provided.
assumptions are presented along articulate relevant assumptions
irrelevant information. Some
Assumptions are not likely to be
with potential biases.
and potential biases.
assumptions are identified.
identified.
A minimal amount of relevant evidence
Relevant, credible evidence is
Relevant evidence is gathered and
Relevant, credible evidence is
is gathered and reported but
gathered, reported, and interpreted
reported accurately. There may be
gathered, reported, and
interpretation is probably not provided. No evidence is
Evidence clearly, fairly, and accurately. In
limited interpretation of evidence or
interpreted clearly, fairly, and
Some irrelevant or distracting evidence
provided.
addition student utilizes evidence in
some evidence may not be from
accurately.
may also be included, or evidence may
a novel and/or esp. effective way.
credible sources.
not be from credible sources.
Develops conclusion with relevant evidence: Student arrives at overall conclusion and articulates implications and consequences.
Student clearly and articulately
Student develops overall
Overall conclusion stated. Some Overall conclusion stated. May not flow
develops overall conclusion from
conclusion from position and
Conclusion and
statement of implications and
clearly from position and evidence, No conclusion is
position and evidence. Provides evidence. Explores implications
Implications
consequences may be included,
implications and consequences not
offered.
nuanced exploration of implications
and consequences of
but may be poorly explored.
explored.
and consequences.
conclusion.
Practices Effective Communication: Student considers purpose, structure, organization, and appropriate conventions to effectively convey meaning to an intended audience.
Demonstrates an exceptional
Lacks
Demonstrates an occasionally
understanding of audience and
Demonstrates understanding of
Demonstrates an inconsistent
understanding
inconsistent understanding of
Purpose and Audience purpose that focuses all elements
audience and purpose that
understanding of audience and purpose of audience and
audience and purpose that
of the work to effectively convey
conveys meaning.
that frequently obscures meaning.
purpose.
occasionally obscures meaning.
meaning.
Lacks adequate
Structure and
Demonstrates a sophisticated
Demonstrates coherent
Demonstrates organization but Demonstrates inconsistent organization, organiz. or
organization
organization to achieve maximum organization with consistent flow may lack overall coherence, flow, lacking overall coherence, flow, and/or
structure to
appropriate to genre,
coherence and momentum.
and continuity.
and/or continuity.
continuity.
convey
discipline, medium
meaning.
Use of
Conventions approp. to
Employs skillful communication
appropriate
genre, discipline, Skillfully communicates meaning
Uses straightforward
that conveys meaning
Uses communication that may include communication
medium with particular clarity, fluency, and
communication that conveys
effectively. It may have errors,
significant number of convention errors conventions is
- such as in writing - possibly creativity. It is virtually error
meaning. It may include errors that
but they do not interfere with
which frequently obscure meaning.
so poor as to
Doane College Intellectual Engagement Rubric
Criteria
Learner selfawareness
Intellectual
curiosity
Connections
across
disciplines
Exceeding (4)
Demonstrates the skills of a
mature learner, able to reflect in
depth, yielding a nuanced or
complex awareness of their
learning process intrinsically
motivating them to guide their
own development as a learner.
Independently motivated to learn
for the sake of increasing own
knowledge and understanding,
often tenacious, innovative or
creative and tend to extend
curiosity beyond the classroom
and apply learning in other
settings.
Integrates and synthesizes
examples, facts, or theories from
more than one field of study or
takes a multi-disciplinary
approach to draw well-reasoned
conclusions or creative insights.
Achieving (3)
Demonstrates the ability to
candidly reflect on own
learning process yielding
awareness and takes some
responsibility to guide own
development as a learner.
Progressing (2)
Beginning (1)
Does not meet
minimum requirements
(0)
Demonstrates the ability to Demonstrates a willingness to
reflect on own learning
reflect, may inadequately
process but superficial
identify strengths and
No willingness to reflect
awareness and action
weaknesses, may not
on own learning.
regarding own development understand own role in own
as a learner.
learning.
Demonstrates interest in
Demonstrates little to no
Motivated to acquire
acquiring knowledge for
Demonstrates some interest in
interest in or is apathetic
knowledge beyond practical
practical ends such as
acquiring knowledge for
toward the acquisition,
ends, appreciates
improving own performance, practical ends such as exams
application or
understanding the significance shows interest in applying
or grades but not in its
significance of
and application of knowledge. knowledge or understanding
application or significance.
knowledge.
its significance.
Connects examples, facts, or
Connects examples, facts, or
theories from more than one Provides examples, facts, or
theories from more than one
point of view or discipline. theories from one point of view
point of view or discipline to
May contain inaccurate or
or discipline.
gain insight.
oversimplified statements.
Does not present any
facts, examples or
theories.
Connections
Meaningfully synthesizes
Identifies connections among Identifies connections among
Can identify significant
among
connections among academic,
academic, co-curricular,
academic, co-curricular,
academic, co-curriculuar,
Does not identify
academic, co- co-curricular, professional and/or
professional and/or life
professional and/or life
professional and/or life
signficant academic, cocurricular,
life experiences to enhance
experiences to enhance
experiences and can state the experiences but has difficulty curricular, professional
professional
understanding, broaden own
understanding and broaden
importance of those
identifying connections among
or life experiences.
and life
perspectives, and translate
own perspectives.
connections.
them.
experiences learning gains to new situations.
Actively engaged with diverse
cultures and communities in order Demonstrates an empathetic Reflects on how own attitudes
Expresses attitudes and
Engagement
to develop an empathetic
understanding of a
and perspectives may be
perspectives as an individual,
with diverse
understanding of a mutlifaceted
multifaceted world and is
different from those of other
respects differences between
cultures and world and is eager to continually
willing to examine own
cultures and communities and
themselves and those of other
communities examine own perspectives in the perspectives in the face of is willing to consider what can
cultures and communities.
face of new experiences or
new experiences or learning.
be learned.
learning.
Expresses attitudes and
perspectives as an
individual and
recognizes differences
between themselves and
those of other cultures
and communities.
Evaluation of assessment and
portfolio management systems
• Team of faculty explored multiple systems
• Identified critical parameters – ease of use, cost
structure, reporting capabilities
• Scheduled open demonstrations for faculty and
staff
• Collected survey evaluations and recommended
LiveText
• Implementation of the system began with
LiveText in spring 2014
Assessment for learning
Intellectual Skills
• Faculty have students submit an assignment that is already
part of their course plan targeting student development
toward these outcomes
• Faculty assess in LiveText using Intellectual Skills Rubric
Intellectual Engagement
• Faculty engage students in critical reflection throughout term
to deepen student self-awareness, responsibility for own
learning
• Faculty have students respond to a reflective prompt/portfolio
that is common across all sections at each seminar level
• Faculty assess in LiveText using Intellectual Engagement Rubric
LAR 101 Reflection Prompt
Critical Reflection assignment instructions:
First, read the LAR 101 Doane Core learning outcomes…
Second, read William Cronon’s essay “‘Only Connect…’ The Goals of a
Liberal Education.”
Finally, in an essay of at least three pages address the following:
• Identify two Doane Core learning outcomes that you think will be
important in your own development and connect each to one or more
Cronon characteristic. Reflect on the personal importance of these
connections, supporting your reflection with specific evidence from
your own life.
• Looking to the future, what will you do next to continue growing? How
will you act, in choosing courses or activities or experiences, in ways
that will help you to identify or explore your interests, clarify your
values, and strengthen your intellectual skills?
LAR 202 Reflection Prompts
DIVERSITY AND DEMOCRACY:
• What do you see as the critical issues that we need to face to
create a well-functioning democracy…?
• Where in your range of academic or co-curricular experiences
have democratic processes played a role? What have you
learned…
• Describe an experience when you have engaged with people from
diverse background that are different from your own. In this
case, how are you defining diversity? What assumptions did you
have going into the experience? How did the experience
challenge your perspectives and assumptions? If it did not…
EDUCATION:
• Reread your LAR 101 critical reflection on the essay by William
Cronon “‘Only Connect…’ The Goals of a Liberal Education” and
reflect on how your perspective(s) on liberal education may have
changed based on the work you have done…
LAR 303 Showcase Portfolio
The comprehensive project in this course will require leadership
and collaboration. In order to make the collaboration as
effective as possible it is important to know the strengths, skills,
and values that each member brings to the team.
To open your portfolio, write a cover letter in application for a
team that summarizes your strengths, skills, and values.
• What are you most passionate about?
• What are your goals for the future?
• What can you bring to the team to help it construct an
understanding of a critical issue…
• How will your engagement with the team over the critical
issue being addressed in your section both further your own
goals and help the team to succeed?
LAR 303 Portfolio Evidence
Your portfolio should showcase examples of your work that provide
evidence of your development in the areas described below. Each sample
should be introduced with a short reflection addressing the prompts below
and any other relevant ideas.
• Engage in intellectual inquiry – what is your best example from
previous work of developing an argument supported by relevant
evidence in order to analyze a critical issue?
• Communicate effectively – what is/are your most effective
achievement(s) in writing, speaking, and/or creative work while in
college?
• Build insights across disciplines – where have you connected
knowledge from two or more courses in different disciplines…?
• Be a productive team member and work collaboratively to solve a
problem – what specifically stands out in your contributions to past
team efforts…?
• Pursue an understanding of a multifaceted world – where have you
engaged with people from different cultures, beliefs, appearances,
abilities, backgrounds, or perspectives, etc in order to challenge and
expand your world view…?
LiveText training, implementation,
analysis of first term of data
Training
• LiveText staff came to campus during August Faculty week
• Faculty received accounts and got into system to explore student
and faculty views
Implementation
• Fall term 2014: LAR 101 students on our traditional campus and
adult campuses successfully submitted assignments, instructors
assessed them
Analysis
• Jan 2015 Rubric Review: Team of faculty reviewed initial data,
conducted inter and intra-rater reliability scoring
• Jan 2015 Workshop on Outcomes and Assessment: shared initial
results and debrief of processes
Initial results for intellectual skills development
in LAR 101 at traditional and adult campuses
Initial results for intellectual engagement in LAR 101
at traditional and adult campuses
Trad.
Only
Adult
Only
Next steps
Revise and practice
• Revise rubrics, prompts
• Foster faculty practice and collaboration regarding rubrics
Support development
• Support development of LAR 202 sophomore seminars and LAR 303
junior/senior seminars
• Support development of reflective writing and student-centered
teaching practices
Look ahead
• Set targets for student development
• Collect evidence of development across the sequence of seminars,
analyze, respond
ASSESSMENT & E-PORTFOLIO SOLUTIONS
• A versatile and extensible learning tool from LiveText
Extracurricular
Experience
Academic
Experience
THE FUTURE OF LEARNING
Professional
Experience
Prior Studies
Hallmark Assignments
Resume / Profile
Personal Achievements
Key Assessments
Organizational Achievements
Hobbies and Interests
Instructor/Faculty Assessment
Professional Membership
Learning Journals
Achievement of University Goals
Journal / Life-wide Learning
3
3
Assignment
A SIMPLE WORKFLOW
3
4
Correlation of Standards &
Outcomes
Custom Rubrics Aligned to Outcomes
Assessors Dashboard
Ease of use for Faculty and Assessors
Customizable Assessment Tools
Assignment
MORE WORKFLOWS
4
0
INFINITE POSSIBILITIES
4
1
Aggregate data on Core
Competencies
Disaggregated by College &
Gender
Trend Reports
Scheduled Reporting
Dashboards
CurateD Portfolios
Questions
• Kate Marley
• kate.marley@doane.edu
• John Burney
• john.burney@doane.edu
• Ida Asner
• ida.asner@livetext.com