Welcome to the CIRTL Network’s Virtual Coffee Hour
FOSTERING CRITICAL THINKING
Nancy Ruggeri
Susanna Calkins
Justin Notestein
Session begins at 1PM CT
When you join the room please run the Audio Setup Wizard: Tools Menu->Audio->Audio Setup Wizard
While we wait for the session to begin, please feel free to test your microphone and webcam
If you are experiencing problems and/or have questions, please type into the chat window
www.cirtl.net
Ways to Interact during the Coffee
Hour Discussion
• Turn on/off your microphone:
• Turn on/off your video:
• Raise your hand if you have a question or
comment
• Use the chat window to add comments, ask
questions, or request help
www.cirtl.net
Susie Calkins, Assoc. Director, Searle
Center for the Advancement of
Learning & Teaching
Justin Notestein, Asst. Prof, Chemical
and Biological Engineering
Nancy Ruggeri, Assoc. Director, Searle
Center for the Advancement of
Learning & Teaching
When I hear the term “critical thinking,” I have a
good sense of what this means in my
discipline.
A.
B.
C.
D.
Strongly agree
Agree
Disagree
Strongly disagree
What is Critical Thinking?
Classical Emphasis
Evaluate Arguments
and Conclusions
Reasoning
Expanded Contemporary Emphasis
Evaluating Ideas
And Plans
Evaluate One’s Own
Understanding
Problem Solving
Communication
Developing Life-Long
Learning Skills
Creativity
Thanks to Barry Stein
Critical knowledge
(argument validity,
theory, socio-political
consequences)
Critical Skills
(challenge
assumptions &
biases, evaluate
evidence, analyze
arguments etc)
Critical
Disposition
(willingness to be
critical thinker,
analyze own
faults & biases,
self-reflection)
James, Hughes & Cappa, 2010)
Dimensions
of critical
thinking
To what extent do you try to incorporate methods to
promote critical thinking into your teaching?
(Or if you haven't taught, to what extent do you see
other instructors promoting critical thinking in their
teaching?)
A.
B.
C.
D.
To a great extent (in most or all class sessions)
On occasion, but not in every session
Not very much at all
Unsure
Critical Thinking Initiative in STEM (CTIS) Research
Collaboration
Research Questions
Can we use the Critical Thinking Assessment Test (CAT) to:
1. Improve how faculty teach (and think about) critical thinking?
2. Increase the gains that students make in critical thinking?
DUE-0942404
What we are doing…
Present faculty with assessment data on
student gains in critical thinking in their class…
…To inspire faculty to make changes
to their teaching to enhance critical
thinking…
…To see if changes in teaching result in
greater gains in their students’ critical
thinking
Skills Evaluated by CAT Instrument
Separate factual information from inferences.
Evaluating
Information
Interpret numerical relationships in graphs.
Understand the limitations of correlational data.
Evaluate evidence and identify inappropriate conclusions
Identify alternative interpretations for data or observations.
Creative
Thinking
Identify new information that might support or contradict a
hypothesis.
Explain how new information can change a problem.
Learning &
Problem
Solving
Separate relevant from irrelevant information.
Integrate information to solve problems.
Learn & apply new information.
Use mathematical skills to solve real-world problems.
Communication
Communicate ideas clearly
Developing Discipline Specific Analogs
Aligning course assessments with critical thinking skills
Ex. Identify alternative
interpretations for evidence
Ex. Identify new information
needed to support an idea
Example from Geology Analog
Students consider scatter plot graph indicating Exosolar Planets
Step 1: Summarize information of graph
Step 2: Evaluate inference from NASA scientists, describing the
extent to which the data support the inference.
Step 3: Offer other plausible explanations for the data (OTHER than
the scientists’ inference)
Step 4: Identify other data or techniques needed to more fully
evaluate the inference
*planets that revolve around a star that is not our Sun.
Enhancing critical thinking skills in Chemical Engineering
Chemical engineers are…
exactly what you think they would be like.
Curriculum focuses on solving problems (equations) in
math, science, and engineering using modern tools of
the field.
• Calculus, simulation, and hands-on experiment.
• Generalities of course content must be approved
by external accreditation board.
It is underappreciated that the real challenges
are in critical thinking:
• How to formulate ill-defined problems
• How to interpret ambiguous results
• How to chose which of many possible
(constraints, fits, equations, solution
methods, data sources) apply to a given
situation
ABET Requirements
Core
curriculum
elements
Math
Science
Math
214;
215
Gen
Chm
Phys
1352,3
a. math, sci,
engg
*
*
*
*
*
b. expt;
data anal
c. design
d.
teamwork
*
Des & Com
Engineering Anal., Comp., Basic Eng.
EDC1,2
Eng
Anal
1-4
MSE
301
Stat
ChE
210
ChE
211
ChE
321
Chm
2101,2
Chm
3422
ChE
212
ChE
275
ChE
307
ChE
322
ChE
323
ChE
341
ChE
342
ChE
351
ChE
352
*
*
*
3
3
3
*
*
2
3
3
2
3
3
1
2
1
*
*
i
1
*
1
1
1
3
*
1
1
3
3
*
1
1
e. engg
problems
*
f. prof;
ethics
*
g. commun.
*
h. broad
education
*
Com
*
*
*
3
*
1
1
j. contemp
issues
"3" = Major
component
(>25%)
*
*
"2" = significant component
(10-25%)
*
*
2
2
2
1
3
i
1
3
i
i
1
1
i
i
i
1
2
i
1
1
i
3
i
1
i
i
2
i
1
1
i
3
i
i
2
2
1
2
"1" = minor component
(5-10%)
1
1
3
i
i. lifelong
learning
k. modern
tools
3
Major Program: Required Classes
*
"i" = incidental component
(<5%)
2
3
1
1
2
1
2
1
1
1
1
1
1
1
3
1
2
i
1
i
i
i
1
i
1
i
1
1
1
1
1
i
2
2
2
2
"*" indicates contributions to ABET
(a) - (k) from curriculum elements or
courses outside the ChBE
Significant obstacles
The community feels engineering education is a zero-sum game.
“The PI has begun to develop non-technical communication exercises for ChemE
undergraduates, and has two years of evaluation results suggesting that students
appreciate their value. The technical content from the introductory mass-and-energybalance course that these exercises have or will displace is not specified.”
(comment on 2010 NSF CAREER proposal)
I argue that enhancing critical thinking is essential,
and actually the core of engineering education
Here, discuss ChE 210
First ChE course, covers underlying principles of the field
Minimal math: IN – OUT = ACUMULATION
First ‘engineering’ course: math with a context, overspecified/underspecified.
Required to synthesize information, find ‘creative’ solutions
Performance in this course often uncorrelated with the rest of the curriculum.
One example of the appearance of critical thinking
Calculating PVT relationships is a core competency
• PV=nRT
• MANY equations of state for ‘real gases’
• Theoretical derivations
• Empirical Fits
• Generalized expressions
 Offered a bonus question: A prize to first person to calculate P of CH4 at some T
to 10% accuracy, to 1% accuracy
 Many people gave very precise values using inaccurate relations
A critical thinking assessment (pre-test week 1, post-test week 8)
% conversion
% conversion
The following plots are obtained when a researcher is monitoring the conversion of compound A into
compound B as a function of time at a set pressure (left) and for a constant time with increasing pressure
(right). The researcher hypothesizes that conversion should increase linearly with time and should be
independent of pressure.
Time (min)
Pressure of A
Describe the plots.
A) The plots show a set of experiments where the conversion of a reaction is monitored at different
pressures and times.
B) The plots show a set of experiments monitoring a chemical reaction. Generally, the data could be
consistent with conversion increasing with time and relatively independent with pressure.
C) The plots show that conversion increases with time and no correlation can be drawn to the effect
of pressure.
D) Conversion increases linearly with time, except for the third point, which is wrong. It is hard to
draw conclusions about the effect of pressure. If the 4th point is wrong there may be a sinusoidal
dependence.
Further questions requiring written responses
A. Describe the plots.
B. Does this plot support the hypothesis that % conversion is linearly dependent on
time and independent of the pressure of A?
C. Looking back at their work, the researcher suspects there may be an error in the
collection of one or more data points in each experiment. Do these plots support
the hypothesis that there is an error in the collection of one or more data point?
D. Does your answer to part C change your answer to part B? Why or why not?
Critical Thinking Skills Tested:
A: Summarize a pattern of results without making inappropriate inferences
B/C/D: Evaluate how strongly correlation-type data supports a hypothesis.
Provide alternative explanations for a pattern with many possible causes.
Provide relevant alternative interpretations of information.
Provide alternative explanations for spurious relationships.
Identify additional information needed to evaluate a hypothesis.
Summary of student responses
• About 30% of students were able to summarize without making inappropriate inferences.
• Students had a strong tendency to see a linear trend in graph 1. Students not clear on what it
means for data to be ‘independent of pressure’. Many students were looking for any possible
correlation – sinusoids, higher order polynomials, etc.
• Most students assumed from the get-go that one data point was wrong, but no appreciation
of error bars in data, no appreciation that axes are devoid of scale, etc.
• Data treated as canonical – their job is to find the ‘right’ fit.
 For the rest of the course, dealt with messy and ill-defined data sets. Many instances
where multiple data sources give you similar, but certainly not the same results.
 Saw no improvement with post-test, just a broadening of the distribution.
 Students excel at the math of data fitting, but don’t know how or why.
 Suggestions: address ‘real’ data early and often. Show that results can be interpreted
in multiple ways, which may or may not be equally valid/general. Importance of
‘consistent with’ vs. ‘is’
A second type of critical thinking question (Week 2 and week 9)
You are an engineer at Dow Chemical looking to make a new product by one of two processes A and B
described in patents and summarized below. Both processes use the same reactants, have the same
products, and generate the same overall yields.
Process A: Patent 1,234,567, year 1978
Reactant
100 m3 reactor
T = 200°C
P = 50 psig
Byproduct
20 m tall
separator
T = 200°C
P = 50 psig
Product
Process B: Patent 4,567,890, year 2010
Reactant
1 m3 reactor
Filled with catalyst
T = 200°C
P = 500 psig
Byproduct
compact
separator
T = 200°C
P = 500 psig
Product
Additional info:
•The reactant is toxic.
•Compressors are expensive to operate (electricity cost).
•Temperatures above 150°C require special materials.
•The catalyst and compact separator units are proprietary
(separately patented and licensed)
•12 examples of process A have been built at scale; no
examples of process B have been built at this scale.
•The product is worth $1200/ton
•The reactant costs $800/ton
To help you decide between the two processes, you hire a consultant with many decades of experience. For
two processes that have the same overall specifications (use the same reactants, generate the same
products), the major deciding factor for Dow will be total cost to build and operate the process.
• Consultant recommends process A for a company that wishes to make 10,000 tons/yr of product. Does
this mean that process A is lowest cost to construct?
• Circle the most relevant pieces of information.
• List at least three reasons why consultant would have recommended this process, other than it being the
lowest cost to construct.
• What additional info may have the consultant used to reach their recommendation?
A second type of critical thinking question (Week 2 and week 9)
You are an engineer at Dow Chemical looking to make a new product by one of two processes A and B
described in patents and summarized below. Both processes use the same reactants, have the same
products, and generate the same overall yields.
Process A: Patent 1,234,567, year 1978
Reactant
100 m3 reactor
T = 200°C
P = 50 psig
Byproduct
20 m tall
separator
T = 200°C
P = 50 psig
Product
Process B: Patent 4,567,890, year 2010
Reactant
1 m3 reactor
Filled with catalyst
T = 200°C
P = 500 psig
Byproduct
compact
separator
T = 200°C
P = 500 psig
Product
Additional info:
•The reactant is toxic.
•Compressors are expensive to operate (electricity cost).
•Temperatures above 150°C require special materials.
•The catalyst and compact separator units are proprietary
(separately patented and licensed)
•12 examples of process A have been built at scale; no
examples of process B have been built at this scale.
•The product is worth $1200/ton
•The reactant costs $800/ton
Critical Thinking Skills Tested:
 Evaluate whether spurious relationships strongly support a claim.
 Determine whether an invited inference in an advertisement is supported by information.
 Provide relevant alternative interpretations of information.
 Separate relevant from irrelevant information when solving a real-world problem.
 Identify suitable solutions for a real-world problem using relevant information.
Noted improvements in this question over the course of the year, especially in the creativity of responses.
Suggestions – talk about the real-world issues behind the engineering problems. Acknowledge conflicts of
interest and controversy. The motivations of Chaucer and a $21B gas-to-liquids plant are both subject to
interpretation.
Consider a critical thinking skill you would like to
develop in your students.
Create a new assignment, project or activity that
could develop that skill
FOSTERING CRITICAL THINKING
Dr. Nancy Ruggeri,
Associate Director of
Graduate Programs,
Searle Center for
Advancing Learning and
Teaching, Northwestern
University
Dr. Susanna Calkins,
Associate Director, Faculty
Development, Searle
Center for Advancing
Learning and Teaching,
Northwestern University
Dr. Justin Notestein,
Assistant Professor of
Chemical and Biological
Engineering, McCormick
School of Engineering,
Northwestern University
www.cirtl.net
NEXT MONTH’S COFFEE HOURS
BUILDING AN ACADEMIC
CAREER SERIES
TEACHING AND LEARNING
IN THE STEM CLASSROOM
SERIES
Tenure and Promotion: What you should
know, what you should ask
April
25th,
2013, 12-1pm CT
No More Coffee Hours in Series
Please check back next September for next
year’s Coffee Hours series
Facilitated by:
Don Foss, Professor, Dept of Psychology, University of
Houston
Philip Cohen, Vice Provost for Academic Affairs, Dean,
Graduate School, Professor, Dept of English, The
University of Texas at Arlington
Daniel Mosse, Professor, Dept Chair, Dept of Computer
Science, University of Pittsburgh
To sign up to hear about these and other CIRTL events, email
info@cirtl.net.
www.cirtl.net