Thinking Critically
about Critical Thinking
Jun 26, 2015
9:00 am – 2:30 pm
UCEN-107
Special Guests:
Julie Stein, CSU East Bay
Gregory Maximilian
Aisemberg, COC
Philosophy
Andy McCutcheon
Rebecca Eikey
Goals
What value do Institutional Learning
Outcomes have?
Why develop measurements for the ILOs?
What is Critical Thinking?
What would a Critical Thinking Rubric for
COC look like?
How do rubrics support student learning?
COC Mission
As an innovative institution of excellence, College of the
Canyons offers an accessible, enriching education that
provides students with essential academic skills and
prepares students for transfer education, workforce-skills
development, and the attainment of learning outcomes
corresponding to their educational goals. To fulfill its mission,
College of the Canyons embraces diversity, fosters
technical competencies, supports the development of
global responsibility, and engages students and the
community in scholarly inquiry, creative partnerships, and
the application of knowledge.
“A collaboration
between educators,
students, policymakers,
and business and
community leaders.”
How is the Workplace Changing?
“Human work will increasingly shift toward two
kinds of tasks:
• solving problems for which standard operating
procedures do not currently exist,
• and working with new information—acquiring it,
making sense of it, communicating it to
others….”
Frank Levy and Richard Murnane, “Dancing with Robots” (2013)
Learning Agility
The LEAP Initiative Promotes
• Essential Learning Outcomes
A Guiding Vision and National Benchmarks for College Learning and Liberal Education
in the 21st Century
• High Impact Practices
Helping Students Achieve the Essential Learning Outcomes
• Authentic Assessments of Student Learning
Probing Whether Students Can APPLY Their Learning – to Complex Problems and RealWorld Challenges
• Seven Principles of Excellence, including Inclusiveness
Diversity, Equity, Quality of Learning for All Groups of Students
Goal: Raise Quality of Education
 Large-scale
collaboration
 Transformational
change
 Educational
alignment
The Essential Learning Outcomes
Narrow Learning is Not Enough
 Knowledge of Human Cultures and the Physical and Natural World
Focused on engagement with big questions, enduring and contemporary
 Intellectual and Practical Skills
Practiced extensively across the curriculum, in the context of progressively more
challenging problems, projects, and standards for performance
 Personal and Social Responsibility
Anchored through active involvement with diverse communities and real-world challenges
 Integrative and Applied Learning
Demonstrated through the application of knowledge, skills, and responsibilities to new
settings and complex problems
Overview of ILO Development
at CSU East Bay
2010
2011
Development of
Institutional
Learning Outcomes (ILOs)
2012
2013
2014
ILO
Adoption
Blackboard
Outcomes
Implementation
Campus-Wide
Assessment
Critical
Thinking
11
Overview of ILO Development
at CSU East Bay
The California State University East Bay Institutional Learning Outcomes (ILOs) express a
shared, campus-wide articulation of expectations for all degree recipients. Graduates
of CSUEB will be able to:

think critically and creatively and apply analytical and quantitative reasoning to
address complex challenges and everyday problems;

communicate ideas, perspectives, and values clearly and persuasively while
listening openly to others;

apply knowledge of diversity and multicultural competencies to promote equity
and social justice in our communities;

work collaboratively and respectfully as members and leaders of diverse teams
and communities;

act responsibly and sustainably at local, national, and global levels;

demonstrate expertise and integration of ideas, methods, theory and practice in
a specialized discipline of study.
12
Outcomes Build Upon Each Other
Proposed Six ILOs
1.
2.
3.
4.
5.
6.
Effective Communication
Critical Thinking
Working with Others
Information Literacy
Quantitative Literacy
Community Engagement
COC ILO Survey Results
Invited to
Responses
Participate
Overall
Response
Rate
122
844
15%
Adjunct Faculty
64
595
11%
Full-Time Faculty
37
179
21%
College Planning
Team
6
22
27%
Division Deans
4
7
57%
Learning Resources
6
8
75%
Student Services
5
16
31%
In General - Average Response Rate for Email Surveys = 24.8%
COC ILO Survey Results
Survey Prompt
Response
Familiar with proposed ILOs?
44% = yes
56% = not
Satisfaction with proposed ILOs?
~70% = satisfied or very satisfied
Agreement that ILOs reflect
COC?
~80% = agree or strongly agree
Consider remaining categories?
~20% = yes
Areas missing in the ILOs?
~15% = yes
Activity: Ranking of ILOs
 Working as individuals
 Rank the proposed ILOs according to importance.
 Where 1 = most important and 8 = least important.
ILO Activity – Definitions
 Working in teams on definitions of ILO
 Each group will have a definition of a proposed ILO
 Edit/refine the definition and post into Discussion Board in Bb
 Present to entire group
 Debrief
Break
Break
11:00 am -11:10 am
How Rubrics Measure
Student Attainment of Outcomes
A rubric is a faculty-developed scoring guide for use in assessing student work
along specific dimensions. It contains a set of criteria specifying the
characteristics of a learning outcome (e.g. the ILO) and the levels of
achievement for each characteristic.
Level of
achievement
Criteria
How Rubrics Measure
Student Attainment of Outcomes
 Articulates what learning faculty want their
students to achieve actually looks like
 Helps clarify “fuzzy” outcomes: e.g. “demonstrate
effective critical thinking”
 Can measure virtually any student work (e.g.
paper, e-portfolio, project, audio or video
presentation, performance, blog, etc.)
 Helps students: clear expectations, specific
feedback, better potential future performance
The Difference Between Course Grading,
PLO Assessment and ILO Assessment
SLO
Who designs the
assessment(s)?
What is purpose?
Assessment affects
student grade?
What happens with
the results?
PLO
ILO
The Development of the
Critical Thinking Rubric at CSU East Bay
27
The Development of the
Critical Thinking Rubric at CSU East Bay
Results and Faculty Response
 Improved student learning when students
were provided with a rubric
 Recognition of the value of involving faculty
in all steps of the process
 Reinforcement of the importance of
designing well-crafted, meaningful
assignments with clear, carefully crafted
prompts
 Concern about the applicability of one rubric
across disciplines
“During the course of the critical
thinking rubric project, the quality of
work submitted by the students was
much higher than in quarters past. I also
feel that the rubric helped me to grade
the papers more consistently and helped
me to hold the students to a higher
standard, which helps them to reach
higher levels of achievement in their
future courses.”
Hospitality, Recreation, & Tourism
faculty
28
AAC&U VALUE Rubrics
 As part of its VALUE (Valid Assessment of Learning in Undergraduate
Education) project, AAC&U worked with faculty and other academic
and student affairs professionals in an exhaustive process of
gathering, analyzing, synthesizing, and drafting institutional-level
rubrics for the Essential Learning Outcomes.
 32,729 individuals participated in consortia approach
 5661 institutions use the VALUE rubrics
VALUE Rubrics
• Contain the most common and broadly shared
criteria or core characteristics considered
critical for judging the quality of student work
in that outcome area.
• Reflect faculty expectations for essential
learning across the nation regardless of type of
institution, mission, size or location.
Faculty Activity & Debrief
Goal:
Develop a Critical Thinking Rubric using
AAC&U’s as a staring point.
Write changes on CT Rubric Posters.
How Rubrics Support Assignments
and Help Deepen Student Learning
At what levels can rubrics be used?
 To evaluate student work demonstrating a particular
student learning outcome (SLO) = individual faculty
member use in grading
 To assess selected student work demonstrating a
particular program learning outcome (PLO)=program
faculty use for curriculum improvement
 To assess selected student work demonstrating a
particular institutional learning outcome (ILO)=university
faculty committee use for institution-wide assessment
32
How Rubrics Support Assignments
and Help Deepen Student Learning
 For faculty, rubrics:
 Can be developed and measure virtually any student work (e.g.
paper, project, audio or video presentation, performance, eportfolio, blog, etc.)
 Provide a clearer picture of strengths and weaknesses across a class
 Can make faculty life easier and grading more consistent, accurate,
and unbiased
 Reduce the time spent grading by referring to substantive quality
descriptors, without writing long comments
 For students, rubrics:
 help to better understand faculty expectations and standards; this can
result in reduced anxiety and improved learning
 Monitor progress as they work towards clearly indicated goals
 Use instructor feedback to improve their performance
 Discourage arguments about grading practices
Activity: Critical Thinking Assignments
 Peer-Review & Sharing
 Working with a partner share your example of a Critical
Thinking Assignment.
 Explain to your partner why you chose this assignment as an
example of Critical Thinking.
 How does this assignment demonstrate Critical Thinking?
 Identify two specific terms from the VALUE Rubric that this
assignment addresses.
 Would you change any elements of the assignment or the
value Critical Thinking VALUE Rubric?
 Report out
Signature Assignment Update
 How might Signature Assignments help strengthen
connections between course level SLOs and ILOs?
 For those who have used Signature Assignments,
 How has it gone?
 Best advice to share?
 Did you use your own rubric?
 Would you consider modifying a VALUE Rubric?
 Plans for the future?
Signature Assignment Examples
 Pan’s Labyrinth
https://pathbrite.com/portfolio/PbqH9cPV49/4multimodal-project
 eMolecule http://morphinemadness.weebly.com/ or
http://jinlutchmanemoleculeproject.weebly.com/ or
Identify some golden nuggets
you have learned today.
Share with a neighbor and then we will report out.
Questions?
Next Steps
Revision of ILO descriptions
Revision of ILO Rubrics
Additional feedback
Survey
Disciplinary experts
Use of ePortfolios
http://facultyeportfolioresource.weebly.com/eportfolio-examplesto-show-your-students.html